Two simple scoring systems, (based on the number of symptoms/risk factors reported on a respiratory questionnaire) produced high positive predictive values when compared with majority opinion of three respiratory physicians regarding possible obstructive airways disease.
Two important sources of selection bias could have occurred. First, non-respondents to the questionnaire survey could have been materially different from respondents. This was examined after the first questionnaire survey in 1993, when a comparison was made between the practice medical records of a random sample of 100 respondents and 100 non-respondents. No important differences were found with respect to age, gender, and total number of consultations and consultations for respiratory problems in the previous year. There is no reason to suspect that the situation was different in 1995. Second, the adults attending for clinical review may have been different from those who were invited but who did not attend. These two groups were compared with respect to their age, gender and the number of cigarettes smoked as reported on the questionnaire. No significant differences were found between the two groups.
The purpose of a screening test is to identify individuals with a good chance of having disease and who require further clinical assessment to confirm or refute the diagnosis. When defining the threshold at which a screening test is deemed to be positive, consideration has to be taken of the balance between false positive results (which can lead to extra distress because of unnecessary further investigations) and false negatives (which result in some cases of disease being missed). The positive predictive value of a test reflects the frequency of disease in those with a positive screening test. The cost effectiveness of a screening programme will depend on the cut-off values chosen for the screening procedure as these determine the number of new cases detected and requiring treatment, unnecessary investigations undertaken etc.
When assessing a screening test it can be difficult to know what diagnostic standard to use, particularly for conditions such as obstructive airways disease which do not have a universally accepted clinical definition. In our study, we defined adults as having possible obstructive airways disease if the majority opinion of three consultant physicians was that a trial of treatment for obstructive airways disease was merited (majority decisions of specialists has been used in previous studies to diagnose asthma) [1, 5]. Many clinicians would probably agree that it is reasonable to assess an adult who might merit a trial of obstructive airways disease treatment. Comparing the two scoring systems against this "standard", we were able to determine the test characteristics (sensitivity, specificity and positive predictive value) of each system. The more stringent system (four or more symptoms/risk factors plus a marker of severity) had a greater positive predictive value and greater specificity (so gave fewer false positive results) than the less stringent system (four or more symptoms/risk factors alone), but at the cost of reduced sensitivity (more cases were missed).
Any system, which increased sensitivity, would almost certainly have reduced positive predictive value probably resulting in and unacceptable rate of false positives. In contrast with screening for malignant conditions when detection of all cases is of prime importance (i.e. high sensitivity), it could be argued that in population screening for diseases such as asthma/COPD the PPV carries more weight than sensitivity.
A recent article by Grimes and Schultz  illustrates this point:
"Although sensitivity and specificity are of interest to public-health policymakers, they are of little use to the clinician. Stated alternatively, sensitivity and specificity (population measures) look backward (at results gathered over time). Clinicians have to interpret test results to those tested. Thus what clinicians need to know are the predictive values of the test (individual measures, which look forward)".
In a previous paper , 1112 patients (13.8% of responders) were identified positively by the screening questionnaire. The PPV would indicate that approximately 834 of these had obstructive airways disease. Further, 529 of those who screened positively had no recorded diagnosis of obstructive airways disease or received inhaled medication in the previous 12 months. Thus despite the disadvantage of a relatively low sensitivity for the scoring system significant numbers of patients with obstructive airways disease would be identified. It should be noted that the prevalence of adults "meriting a trial of obstructive airways disease therapy" in our survey was calculated  to be 22% (95% CI 15.5–31.4). This may be an overestimate of the true prevalence of obstructive airways disease as it will include patients who would not benefit from a trial of treatment.
The three specialists reviewing the clinical information relied on detailed written data supplied to them by the research team; they did not have direct contact with patient. Neither the experts nor the examining team were aware of the results of the postal questionnaire at the time of their involvement in the study. Total agreement by the consultants on whether a subject warranted a trial of medication was reached in 59.2% of adults. Difficulties in making a diagnosis without personally seeing the patient may also have contributed to disagreement between consultants although it is likely that some difference of opinion would have persisted even with personal examination. Expert opinion concerning asthma diagnosis has been used to define asthma in previous epidemiological studies [1, 11, 12]., whilst others have used patient recall of asthma diagnosis or treatment when measuring prevalence of the disease or its underdiagnosis . The majority of diagnostic decisions are made in primary care and we acknowledge that choosing three consultant respiratory physicians as the diagnostic gold standard therefore has limitations. They were however chosen to reflect a spread of secondary and tertiary respiratory opinions.
An advantage of the scoring systems used was their simplicity. Techniques such as discriminant analysis and logistic regression could have been used to assess the predictive value of each question and a different scoring system developed from them. This would have the disadvantage that validation would then have been based on the same data as was used to derive the scoring system and therefore would tend to be over optimistic. In addition the scoring system might be more difficult to implement.
In clinical practice, a simple scoring system to identify patients requiring further review is attractive. Choices about which system to use will depend on a number of factors; the balance between positive predictive value and sensitivity of each system and available health care resources. It is important to remember that if this questionnaire were to be used in another setting the positive predictive value would have to be recalculated as this value is dependent on the prevalence of disease in the population studied.