Family physicians’ diagnostic gut feelings are measurable: construct validation of a questionnaire
© Stolper et al.; licensee BioMed Central Ltd. 2013
Received: 19 July 2012
Accepted: 17 December 2012
Published: 2 January 2013
Family physicians perceive that gut feelings, i.e. a ‘sense of reassurance’ or a ‘sense of alarm’, play a substantial role in diagnostic reasoning. A measuring instrument is desirable for further research. Our objective is to validate a questionnaire measuring the presence of gut feelings in diagnostic reasoning.
We constructed 16 case vignettes from real practice situations and used the accompanying ‘sense of reassurance’ or the ‘sense of alarm’ as reference labels. Based on the results of an initial study (26 family physicians), we divided the case vignettes into a group involving a clear role for the sense of reassurance or the sense of alarm and a group involving an ambiguous role. 49 experienced family physicians evaluated each 10 vignettes using the questionnaire. Construct validity was assessed by testing hypotheses and an internal consistency procedure was performed.
As hypothesized we found that the correlations between the reference labels and corresponding items were high for the clear-case vignettes (0.59 – 0.72) and low for the ambiguous-case vignettes (0.08 – 0.23). The agreement between the classification in clear sense of reassurance, clear sense of alarm and ambiguous case vignettes as derived from the initial study and the study population’s judgments was substantial (Kappa = 0.62). Factor analysis showed one factor with opposites for sense of reassurance and sense of alarm items. The questionnaire’s internal consistency was high (0.91). We provided a linguistic validated English-language text of the questionnaire.
The questionnaire appears to be valid. It enables quantitative research into the role of gut feelings and their diagnostic value in family physicians’ diagnostic reasoning.
KeywordsGut feelings Family medicine General practitioners Diagnostic reasoning Questionnaire Construct validation Medical decision-making Medical problem-solving Intuition Clinical reasoning
Uncertainty about diagnosing is a typical characteristic of general practice [1, 2]. Gut feelings and some important aspects of context knowledge, i.e. all a family physician knows about a patient apart from the signs and symptoms, are not discussed in traditional textbooks on medical decision-making, but often used by family physicians to support their diagnostic reasoning [3–5]. Chest pain, for instance, may indicate cardiac diseases, gastro-esophageal or pulmonary disorders, but can also be a sign of musculoskeletal or mental illnesses. Since it is impossible in a family medicine setting to define signs and symptoms that prove the presence of myocardial problems,  gut feelings may contribute to the family physician’s medical decision-making. In the case of chest pain such feelings appear to be rather accurate  and in the case of diagnosing serious infections in children, the family physician’s gut feeling ‘that something is wrong’ is in fact the best predictor among all signs and symptoms [8, 9]. The gut feeling ‘this is not normal’ when a physician observes a child has proved to be a sign that makes the physician question the child’s well-being . A qualitative study concluded that certain gut feelings, which could be referred to as a ‘sense of reassurance’ and a ‘sense of alarm’, play a substantial role in family physicians’ diagnostic reasoning . A sense of reassurance means that a family physician feels secure about the further management and course of a patient’s problem, even though he/she may not be certain about the diagnosis: ‘everything fits in’. A sense of alarm implies that a family physician worries about a patient’s health status, even though he/she has found no specific indications yet; it is a sense of ‘there’s something wrong here’. The sense of alarm as a diagnostic tool has been taken seriously by disciplinary tribunals and is even regarded as an element of the professional standards for doctors . Gut feelings are based on the interaction between patient information and a physician’s knowledge and experience,  and can be considered as a kind of skilled intuition . Dual process theories may explain how physician’s analytical reasoning like the use of Bayes theorem and algorithmic decision tools, and their non-analytical reasoning like pattern recognition and gut feelings continually interact as two modes of knowing and thinking [13, 15]. Affect defined as a feeling of ‘goodness’ or ‘badness’ guides the decision making process [16–19]. Cognitive neuroscience research provides support for the view that emotions are a vital component of the decision making process, helping people to thread their ways through the huge amount of information and knowledge [20–22].
In medicine, quantitative research into the actual contribution of gut feelings to physicians’ diagnostic processes is still lacking. Appropriate study designs assessing the diagnostic value of gut feelings, the significance of determinants like experience and context knowledge and the effects of teaching students about gut feelings need an instrument that will be able to determine the presence of gut feelings in diagnostic reasoning [23, 24]. In nursing, some questionnaires have been developed to explore retrospectively the significance of gut feelings in diagnostic reasoning. These questionnaires, however, do not determine the presence or absence of gut feelings in actual clinical reasoning in real practice and have only been applied in research into nursing [25–30]. Moreover, they are not practicable in a general practice setting due to their extensiveness.
Questionnaire based on the consensus statements
A ‘sense of reassurance’ means that a family physician feels secure about the further management and course of a patient’s problem, even though he/she may not be certain about the diagnosis: everything fits in.
I feel confident about my management plan and/or about the outcome: it all adds up.
A ‘sense of alarm’ implies that a family physician worries about a patient’s health status, even though he/she has found no specific indications yet; it is a sense of ‘there’s something wrong here’.
I am concerned about this patient’s state of health: something does not add up here.
A ‘sense of alarm’ activates the diagnostic process by stimulating a family physician to formulate and weigh up working hypotheses that might involve a serious outcome.
In this particular case, I will formulate provisional hypotheses with potentially serious outcomes and weigh them against each other.
A ‘sense of alarm’ means that a family physician perceives an uneasy feeling as he/she is concerned about a possible adverse outcome.
I have an uneasy feeling because I am worried about potentially unfavorable outcomes.
A ‘sense of alarm’ means that, if possible, the family physician needs to initiate specific management to prevent serious health problems.
This case requires specific management to prevent any further serious health problems.
This patient’s situation gives me reason to arrange a follow-up visit sooner than usual or to refer him or her more quickly than usual to a specialist.
Please indicate what kind of gut feeling you had at the end of the consultation:
* Something is wrong with this picture.
* Everything fits.
* Impossible to say, or not applicable.
The objective of the present study was to assess the construct validity of this questionnaire in a group of experienced general practitioners and to determine its factor structure and internal consistency.
Selection of case vignettes
Based on real patient cases encountered by the 5 family physicians in the project group, we constructed vignettes of cases in which a sense of reassurance or a sense of alarm had played a role in the diagnostic approach of the attending family physician. After discussion, we accepted 15 case vignettes (5 sense of reassurance and 10 sense of alarm vignettes). Each case vignette had a format starting with the contextual information the family physician already has about the patient, followed by the patient’s complaints and finally additional information gathered during the encounter.
Examples of case vignettes
Case vignette 8 (sense of reassurance)
Patient is 34 years old and works as a sales assistant in a bakery. She is married with two children. Her medical history is uneventful. She does not smoke and drinks little alcohol. The only medication she is on is Microgynon 30 (oral contraception). No significant matters in the family history.
Patient visits her family physician because of a burning sensation she’s had for the last two days when urinating. She also feels pain and itch in her labias. She reports some discharge from the vulva. She was given amoxicillin for a lower airways infection two weeks ago. She has had the same complaints after a previous course of antibiotics, but cannot remember the name of the drug she was given then.
Patient does not appear ill and has a normal complexion. External gynecological examination shows no abnormalities. There is vaginal discharge which resembles curdled milk. No further abnormalities are visible. Urine test strips for leukocytes, blood and nitrite are negative.
Case vignette 16 (sense of alarm)
A 49-year-old woman phones the ‘triagist’ at an out-of-hours medical service at 21:40 h to report pain in her left side which has been increasing over the past 4 days. The pain is linked to her breathing and feels like sore muscles. The pain appears to be episodic to some extent; there are times when it is clearly less severe. The patient does not feel an urge to move. She currently has no pain elsewhere, and has had none during the past few days. Apart from a caesarian section ten years ago, she has no medical history. The only medication the patient is on is oral contraception. She has no known allergies. The triagist decides to invite her to visit the out-of-hours medical service post the same night.
As the physician collects her from the waiting room, the pain makes her walk with a stoop and she seems to experience shooting pains with each breath. She occasionally cries out for pain.
Blood pressure is 128/84 mm Hg, pulse rate 90 a minute, regular and even. Saturation rate is 97%. Auscultation of the heart reveals no abnormalities and the lungs present vesicular breath sounds. A striking feature is the marked local tenderness of the musculature on the left side of the thorax. Calves are supple. A chest X-ray made within the past week shows no abnormalities. The family physician decides to administer an intramuscular injection of diclofenac 75 mg combined with 2 mg diazepam, which seems to be reasonably effective.
First general hypothesis, relating to diagnostic characteristics with regard to clear-case vignettes:
The correlation between the reference labels of the clear-case vignettes (i.e. sense of reassurance or sense of alarm) and the answers given by the study population to questionnaire items reflecting a sense of reassurance or a sense of alarm is moderate to high.
Specific hypotheses derived from this:
Comparing labels: there is a moderate to high correlation and agreement between the reference labels of the clear-case vignettes and the final sense of reassurance or sense of alarm judgment given by the study population (item 7).
Comparing items of the study population: there is a moderate to high negative correlation between the sense of reassurance item (item 1) and the final sense of reassurance or sense of alarm judgment (item 7) for the clear-case vignettes. There is a moderate to high positive correlation between the sense of alarm items (items 2–6) and the final sense of reassurance or sense of alarm judgment (item 7) for the clear-case vignettes.
Comparing items of the study population: there are moderate to high negative correlations between the sense of reassurance item (item 1) and the sense of alarm items (items 2–6) for the clear-case vignettes, whereas the intercorrelations between the sense of alarm items (items 2–6) are moderate to high.
Second general hypothesis, relating to diagnostic characteristics with regard to the ambiguous-case vignettes:
The correlation between the reference labels of the ambiguous-case vignettes (i.e. sense of reassurance or sense of alarm) and the answers given by the study population to items reflecting a sense of reassurance or sense of alarm is weak to absent.
Specific hypotheses derived from this:
Comparing correlations: there is a weaker correlation between the reference labels of the ambiguous-case vignettes and the final sense of reassurance or sense of alarm judgment (item 7) given by the study population compared to the clear-case vignettes.
Comparing items of the study population: there is a weak correlation between the sense of reassurance item (item 1) or sense of alarm items (items 2–6) and the reference labels, but a moderate to high correlation between the sense of reassurance or sense of alarm items (items 1–6) and the final sense or reassurance or sense of alarm judgment (item 7).
Finally, we purposively selected 52 experienced family physicians from different regions in the Netherlands, as an earlier study had suggested that the level of experience is positively associated with the use of gut feelings  and invited them by phone to participate. To avoid selection bias, they were told that we were studying the diagnostic reasoning process of family physicians, without mentioning that the purpose of the study was to get information about gut feelings. Each of them received by post a different set of 10 case vignettes, selected so as to balance the number of clear and ambiguous cases, and varying the presentation order. We asked the participants to evaluate the vignettes by stating the most likely diagnosis and the treatment approach they envisioned, and to complete the questionnaire for each vignette. Forty-nine family physicians (63% female, mean number of years of experience = 20, SD = 9) of the 52 invited family physicians completed 10 questionnaires each. The reason to decline was lack of time.
First, to examine whether the items concerning sense of alarm and sense of reassurance formed one or two dimensions, we examined the factor structure employing a Principal Component Analysis. The internal consistency was measured by calculating Cronbach’s alpha, after reversing the polarity of the sense of reassurance item (item 1) .
To examine the hypotheses in our construct validity procedures, we used Spearman’s rank correlations because of the categorical, ordinal nature of the data. We used the usual interpretation of the correlation values: weak < 0.3, moderate 0.3–0.6 and high > 0.6 .
We then used a cross tabulation to visualize the extent of agreement per vignette between the reference labels of the case vignettes and the final judgment (item 7) given by the study population. We also calculated a weighted Kappa using quadratic weights to assess the chance-adjusted agreement between the classification of the case vignettes (3 categories based on the results of the initial study: clear sense of reassurance; ambiguous; clear sense of alarm case vignettes) and the final judgment given by the study population in item 7 (3 categories: sense of reassurance, impossible to say or not applicable, sense of alarm). We used the following interpretation of the Kappa values: < 0.2 = slight, 0.2–0.4 = fair, 0.4–0.6 = moderate, 0.6–0.8 = substantial and > 0.8 = almost perfect . Statistical calculations were performed using SPSS-16.
For this publication and to allow a broad application of the Dutch-language questionnaire we translated the questionnaire into American-English using a formal linguistic validation procedure . This involved two separate forward-backward translations, a consensus meeting and a check on cultural misunderstandings by ten US family physicians [39–41]. The text of the questionnaire is the result of this procedure (see Table 1) (see Endnote).
Since no patients were involved, this research did not fall under the Dutch Medical Research Involving Human Subjects Act (WMO) or the Embryos Act, so that no ethical permission was required.
Factor analysis on items 1-7
Correlations between questionnaire items for the group of ‘clear-case’ vignettes (n= 49 family physicians)
Correlations between questionnaire items for the group of ‘ambiguous-case’ vignettes (n= 49 family physicians)
Number of participants of the study population that agreed upon the SR or SA reference label (% of agreement in bold) in their final judgment (item 7) for each vignette in both the ‘clear-case’ group and the ‘ambiguous’ group
Impossible to say or not applicable
The purpose of this study was to assess the construct validity of a questionnaire to measure gut feelings in general practice, which was based on consensus statements obtained in a Delphi procedure. The construct validity proved to be good. The correlation patterns were consistent with our hypotheses and there was a substantial agreement in judgments between the family physicians in the initial and the validation study. Factor analysis showed that the questionnaire clearly determined one gut feelings factor, with the sense of reassurance and the sense of alarm as two opposites, and the questionnaire had a high internal consistency. Our questionnaire to determine gut feelings in general practice thus passed an important validation test. The careful linguistic validation procedure makes it appropriate for research across countries. As far as we know, this is the first validated instrument providing a measure of the extent to which a sense of reassurance or a sense of alarm is present during the process of diagnosis. The results of the Delphi consensus procedure can be regarded as a sound basis for the content of the questionnaire. The successful construct validation procedure confirms that gut feelings play a role in family physicians’ diagnostic reasoning process, so now quantitative research can be started to examine their actual contribution.
We tried to compare our gut feelings questionnaire with comparable instruments by performing an English-language search in PubMed and Cinahl using the search terms ‘questionnaire’ [Mesh] OR ‘weights and measures’ [Mesh] AND ‘intuition’ [Mesh] OR ‘gut feelings’ OR ‘sense of reassurance’ OR ‘sense of alarm’. This search only yielded some extensive questionnaires examining the meaning of intuition in the field of nursing [25, 26, 28–30, 42] retrospectively exploring not only the use of intuition and the extent of acknowledgment of intuition, but also other phenomena like physical sensations, premonitions, and reading of cues, cognitive behaviors, experience, skills and clinical thinking. In contrast, our gut feelings questionnaire only aims to determine the presence of a sense of reassurance and a sense of alarm in actual family physicians’ diagnostic reasoning, in observational as well in experimental research designs . The questionnaire is short, easy to complete and feasible for use during routine office hours.
A limitation of our validation approach is that we used the outcomes of the diagnostic reasoning process of the 26 participants of the initial study as a reference to interpret the outcomes in the present validation study. Obviously, we needed such a procedure because there is no gold standard for gut feelings in general practice and we wanted to extend our reference beyond the agreement of the 5 family physicians who were part of the project group. The classification of case vignettes into a clear-case vignette group and an ambiguous one was consistent with actual practice with sometimes vague presentations of disease pictures and diagnostic uncertainty. It allowed us to hypothesize in which cases the correlations between the items in the questionnaire and the reference labels of the case vignettes (i.e., sense of reassurance or sense of alarm) would be moderate to high, or weak to absent. The hypothesized patterns were clearly confirmed by the data, although there were three case vignettes where the classification of the case vignettes as clear sense of reassurance, clear sense of alarm, or ambiguous case vignettes based on the initial study were not consistent with the judgment by the participants of the validation study. Case vignette 12 would now be classified as an ambiguous case, while cases vignettes 9 and 15 would now be classified as clear cases. The shift might be due to differences of experience between the family physicians participating in the initial and validation study and perhaps to the minor adaptations to the phrasing of the case vignettes made after the initial study.
Nevertheless, overall agreement in judgments between the 26 family physicians of the initial study and the 49 family physicians of the validation study was substantial. Moreover, when we based the classification (in further analysis) on the judgments of the 49 experienced family physicians the correlations were even higher for the clear-case vignettes and virtually absent for the ambiguous-case vignettes. We cannot expect full agreement, as gut feelings are based on the interaction between a family physician’s knowledge and experience and the patient information available . As we represented real life situations in the case vignettes in our study, the participants could not further investigate their gut feelings by questioning or examining a patient. Research, therefore, should be extended to real practice situations where additional information can be obtained from patients.
After the validation, our questionnaire can now be used to start further research into the role of gut feelings in diagnostic reasoning, as well as research to assess the significance of the determinants and the effects of educational interventions related to gut feelings. The questionnaire may be useful for studies among specialists and nurses too. A European research agenda on gut feelings  gives the highest priority to questions about the prevalence of a sense of reassurance and a sense of alarm in routine practice and their diagnostic accuracy. Gut feelings may function as a compass in uncertain and complex situations, and now outcomes research may reveal to what extent this tool guides family physicians towards correct decisions and a proper diagnostic management and whether there are differences between experienced family physicians and trainees. Comparisons of situations in which family physicians manage patients they know and patients they see for the first time (e.g. when they are on call) may be used to assess the role of context knowledge. In terms of medical training, an interesting topic would be to study how students, trainee doctors and even experienced practitioners can be trained to develop, recognize and use gut feelings while avoiding pitfalls. In more experimental research, standardized patients or written clinical cases varying in the amount of contextual and patient knowledge could be presented for diagnosis to trainees and family physicians, to study the contribution of major diagnostic cues.
Family physicians’ diagnostic gut feelings are measurable with the validated questionnaire. Diagnostic reasoning belongs to the core business of family physicians, and qualitative research has shown that gut feelings play a substantial part in this process. Our questionnaire should now make it possible to take the next crucial steps toward more quantitative research.
A British-English version has been obtained in a similar way and is available too (http://www.gutfeelingsingeneralpractice.eu).
The authors are very grateful to the 75 participating family physicians and to Silvia Bours, research assistant.
- Dinant GJ, et al: Diagnosis and decision. Undifferentiated illness and uncertainty in diagnosis and management. Oxford Textbook of Primary Medical Care. Edited by: Jones R, Britten N, Gulpepper L, Gass D, Grol R, Mant D. 2004, Oxford: Oxford University Press, 201-203.Google Scholar
- Griffiths F, Green E, Tsouroufli M: The nature of medical evidence and its inherent uncertainty for the clinical consultation: qualitative study. BMJ. 2005, 330: 511-517. 10.1136/bmj.38336.482720.8F.View ArticlePubMedPubMed CentralGoogle Scholar
- Bruyninckx R, Van den Bruel A, Hannes K, Buntinx F, Aertgeerts B: GPs' reasons for referral of patients with chest pain: a qualitative study. BMC Fam Pract. 2009, 10: 55-10.1186/1471-2296-10-55.View ArticlePubMedPubMed CentralGoogle Scholar
- Hani MA, Keller H, Vandenesch J, Sonnichsen AC, Griffiths F, Donner-Banzhoff N: Different from what the textbooks say: how GPs diagnose coronary heart disease. Fam Pract. 2007, 24: 622-627. 10.1093/fampra/cmm053.View ArticlePubMedGoogle Scholar
- Johansen ML, Holtedahl KA, Rudebeck CE: How does the thought of cancer arise in a general practice consultation? Interviews with GPs. Scand J Prim Health Care. 2012, 30: 135-140. 10.3109/02813432.2012.688701.View ArticlePubMedPubMed CentralGoogle Scholar
- Bruyninckx R, Aertgeerts B, Bruyninckx P, Buntinx F: Signs and symptoms in diagnosing acute myocardial infarction and acute coronary syndrome: a diagnostic meta-analysis. Br J Gen Pract. 2008, 58: 105-111.View ArticlePubMedGoogle Scholar
- Buntinx F, Truyen J, Embrechts P, Moreel G, Peeters R: Chest pain: an evaluation of the initial diagnosis made by 25 Flemish general practitioners. Fam Pract. 1991, 8: 121-124. 10.1093/fampra/8.2.121.View ArticlePubMedGoogle Scholar
- Van den Bruel A, Haj-Hassan T, Thompson M, Buntinx F, Mant D: Diagnostic value of clinical features at presentation to identify serious infection in children in developed countries: a systematic review. Lancet. 2010, 375: 834-845. 10.1016/S0140-6736(09)62000-6.View ArticlePubMedGoogle Scholar
- Van den Bruel A, Aertgeerts B, Bruyninckx R, Aerts M, Buntinx F: Signs and symptoms for diagnosis of serious infections in children: a prospective study in primary care. Br J Gen Pract. 2007, 57: 538-546.PubMedPubMed CentralGoogle Scholar
- Lykke K, Christensen P, Reventlow S: "This is not normal … "–signs that make the GP question the child's well-being. Fam Pract. 2008, 25: 146-153. 10.1093/fampra/cmn021.View ArticlePubMedGoogle Scholar
- Stolper CF, Van Bokhoven MA, Houben PHH, Van Royen P, Van de Wiel M, Van der Weijden T, et al: The diagnostic role of gut feelings in general practice. A focus group study of the concept and its determinants. BMC Fam Pract. 2009, 10: 17-10.1186/1471-2296-10-17.View ArticlePubMedPubMed CentralGoogle Scholar
- Stolper CF, Legemaate J, Dinant GJ: How do disciplinary tribunals judge the 'gut feelings' of doctors? An analysis of Dutch tribunal decisions 2000-2008. J Law and Medicine. 2010, 18: 68-75.Google Scholar
- Stolper CF, Van de Wiel M, Van Royen P, Van Bokhoven MA, Van der Weijden T, Dinant GJ: Gut feelings as a third track in general practitioners' diagnostic reasoning. J Gen Intern Med. 2011, 26: 197-203. 10.1007/s11606-010-1524-5.View ArticlePubMedGoogle Scholar
- Kahneman D, Klein G: Conditions for intuitive expertise: a failure to disagree. Am Psychol. 2009, 64: 515-526.View ArticlePubMedGoogle Scholar
- Pelaccia T, Tardif J, Triby E, Charlin B: An analysis of clinical reasoning through a recent and comprehensive approach: the dual-process theory. 2011, Online: Med Educ, 16-Google Scholar
- Slovic P, Finucane M, Peters E, MacGregor DG: The Affect Heuristic. Heuristics and biases. Edited by: Gilovich T, Griffin D, Kahneman D. 2002, New York: Cambridge University Press, 397-420.View ArticleGoogle Scholar
- Finucane M, Peters E, Slovic P: Judgement and decision making: The dance of affect and reason. Emerging Perspectives on Judgement and Decision Research. Edited by: Schneider SL, Shanteau J. 2003, Cambridge, UK: Cambridge University Press, 327-364.View ArticleGoogle Scholar
- Epstein S: Integration of the cognitive and the psychodynamic unconscious. Am Psychol. 1994, 49: 709-724.View ArticlePubMedGoogle Scholar
- Mikels JA, Maglio SJ, Reed AE, Kaplowitz LJ: Should I go with my gut? Investigating the benefits of emotion-focused decision making. Emotion. 2011, 11: 743-753.View ArticlePubMedGoogle Scholar
- Craig AD: How do you feel–now? The anterior insula and human awareness. Nat Rev Neurosci. 2009, 10: 59-70. 10.1038/nrn2555.View ArticlePubMedGoogle Scholar
- Naqvi N, Shiv B, Bechara A: The Role of Emotions in Decision Making: a Cognitive Neuroscience Perspective. Curr Dir Psychol Sci. 2006, 15: 260-264. 10.1111/j.1467-8721.2006.00448.x.View ArticleGoogle Scholar
- Damasio AR: Descartes' Error: Emotion, Reason, and the Human Brain. 1994, New York: AvonGoogle Scholar
- Buntinx F, Mant D, Van den Bruel A, Donner-Banzhof N, Dinant GJ: Dealing with low-incidence serious diseases in general practice. Br J Gen Pract. 2011, 61: 43-46. 10.3399/bjgp11X548974.View ArticlePubMedPubMed CentralGoogle Scholar
- Stolper CF, Van Leeuwen Y, Van Royen P, Van de Wiel M, van Bokhoven MA, Houben PHH, et al: Establishing an international research agenda on gut feelings in general practice using the nominal group technique. Eur J of Gen Pract. 2010, 2: 75-79.View ArticleGoogle Scholar
- Smith AJ: The development and psychometric testing of an instrument measuring the use of intuition by nursing students. 2003, University of San Diego: DissertationGoogle Scholar
- Smith A: Measuring the use of intuition by registered nurses in clinical practice. Nurs Stand. 2007, 21: 35-41. 10.1007/BF03079502.View ArticlePubMedGoogle Scholar
- Smith AJ, Thurkettle MA, dela Cruz FA: Use of intuition by nursing students: instrument development and testing. J Adv Nurs. 2004, 47: 614-622. 10.1111/j.1365-2648.2004.03149.x.View ArticlePubMedGoogle Scholar
- Miller VG: Measurement of self-perception of intuitiveness. West J Nurs Res. 1993, 15: 595-606. 10.1177/019394599301500506.View ArticlePubMedGoogle Scholar
- Rew L: Acknowledging intuition in clinical decision making. J Holist Nurs. 2000, 18: 94-13. 10.1177/089801010001800202.View ArticlePubMedGoogle Scholar
- Miller VG: The development of an instrument to measure self-perception of intuitiveness of practicing nurses. 1990, University of Texas at Austin: DissertationGoogle Scholar
- Stolper CF, Van Royen P, Van Bokhoven MA, Houben PHH, Van de Wiel M, Van der Weijden T, et al: Consensus on gut feelings in general practice. BMC Family Practice. 2009, 10: 66-10.1186/1471-2296-10-66.View ArticlePubMedPubMed CentralGoogle Scholar
- Smith GT: On Construct Validity: Issues of Method and Measurement. Psychol Assess. 2005, 17: 396-408.View ArticlePubMedGoogle Scholar
- Vet de HCW, Terwee CW, Mokkink LB, Knol DL: Measurement in Medicine. A practical guide. 2011, Cambridge: Cambridge University PressView ArticleGoogle Scholar
- Baks M: Pulmonary embolism often not immediately evident [Longembolie lang niet altijd direct duidelijk]. Huisarts 24/7. 2009, 28-29.Google Scholar
- Bland JM, Altman DG: Cronbach's alpha. BMJ. 1997, 314: 572-10.1136/bmj.314.7080.572.View ArticlePubMedPubMed CentralGoogle Scholar
- Cohen J: The significance of a product moment r. Statistical power analysis for the behavioural sciences. 1988, Hillsdale, New Yersey: Erlbaum, 75-107.Google Scholar
- Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1978, 33: 159-174.View ArticleGoogle Scholar
- MAPI Institute: http://www.mapi-institute.com/linguistic-validation. last seen 21-12-2012.
- Acquadro C, Conway K, Hareendran A, Aaronson N: Literature review of methods to translate health-related quality of life questionnaires for use in multinational clinical trials. Value Health. 2008, 11: 509-521. 10.1111/j.1524-4733.2007.00292.x.View ArticlePubMedGoogle Scholar
- Streiner DL, Norman GR: Health Measurement Scales. A practical guide to their development and use. 2006, Oxford: Oxford University Press, 3Google Scholar
- Acquadro C, Conway K, Giroudet C, Mear I: Linguistic validation manual for patient-reported outcomes (PRO) instruments. 2004, Lyon: MAPI Research InstituteGoogle Scholar
- Cone KJ: The development and testing of an instrument to measure decision making in emergency department triage nurses. 2000, Saint Louis University: DissertationGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2296/14/1/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.