We are looking at different combinations of neurophysiological tests, and trying to find out which combinations of variables are best suited for clinical use (diagnosing diabetic polyneuropathy). To combine the data, we compute Z-scores and combine these into aggregated scores, and we classify >2SD as abnormal.
Ideally we would just look at sensitivity/specificity/ROC. The problem is that we do not have gold standard data, i.e. we do not have access to the medical files of the participants to investigate at a later date whether they actually were diagnosed with the disease. We still want to try to use this data as well as we can to inform future studies. So since we do not know the true prevalence in the groups, we need to find some sort of proxy for sensitivity/specificity that makes sense..
One group consists of diabetics (n=68), most of whom are likely to have the disease, and the other group are pretty healthy controls (n=38), likely to contain none or very few. The standard deviations used are based on another dataset, with even stricter criteria (super healthy people, n = 600 ish), and are adjusted for known covariates (age, sex and height), so we think that the >2SD limits should be pretty good.
So what we end up with are "prevalence rates" (or at least positive/negatives) of the disease in both groups (we do not know exactly what rate is "true", seems like its somewhere around 75% for the diabetics based on some ceiling-effect here, and between 5-23% for the controls – could be actual findings or false positives). So far we looked at these ways of interpreting the data, but are not entirely sure which makes the most sense, or if maybe something else would be much better suited?
Get help from others!