Context factors in general practitioner - patient encounters and their impact on assessing communication skills - an exploratory study
© Essers et al.; licensee BioMed Central Ltd. 2013
Received: 7 January 2013
Accepted: 15 May 2013
Published: 22 May 2013
Assessment of medical communication performance usually focuses on rating generically applicable, well-defined communication skills. However, in daily practice, communication is determined by (specific) context factors, such as acquaintance with the patient, or the presented problem. Merely valuing the presence of generic skills may not do justice to the doctor’s proficiency.
Our aim was to perform an exploratory study on how assessment of general practitioner (GP) communication performance changes if context factors are explicitly taken into account.
We used a mixed method design to explore how ratings would change. A random sample of 40 everyday GP consultations was used to see if previously identified context factors could be observed again. The sample was rated twice using a widely used assessment instrument (the MAAS-Global), first in the standard way and secondly after context factors were explicitly taken into account, by using a context-specific rating protocol to assess communication performance in the workplace. In between first and second rating, the presence of context factors was established. Item score differences were calculated using paired sample t-tests.
In 38 out of 40 consultations, context factors prompted application of the context-specific rating protocol. Mean overall score on the 7-point MAAS-Global scale increased from 2.98 in standard to 3.66 in the context-specific rating (p < 0.00); the effect size for the total mean score was 0.84. In earlier research the minimum standard score for adequate communication was set at 3.17.
Applying the protocol, the mean overall score rose above the level set in an earlier study for the MAAS-Global scores to represent ‘adequate GP communication behaviour’. Our findings indicate that incorporating context factors in communication assessment thus makes a meaningful difference and shows that context factors should be considered as ‘signal’ instead of ‘noise’ in GP communication assessment. Explicating context factors leads to a more deliberate and transparent rating of GP communication performance.
KeywordsCommunication and Interviewing skills Continuing Medical Education Graduate Medical Education Assessment of Learner Performance
As communication is at the heart of good clinical practice, communication training and assessment are key components in undergraduate as well as postgraduate medical curricula. Communication levels are usually assessed by rating the performance against predefined communication skills [1–3]. Widely used communication assessment instruments such as the Maastricht History-taking and Advice Scoring list (MAAS-Global) , also applied in general practitioner’s (GP) performance assessment, determine to what extent generic communication skills, expected to be pursued in every consultation with every patient, are observed. However, generic criteria may fail to capture the contextual proficiency of the GP’s performance [2, 5–7]. The underlying assumptions that generic communication skills should be applied in any consultation and that every consultation can be treated as if it requires the same communication performance seem unjustified. Several authors have argued that context factors influence communication in health care [1, 8–10] and that communication performance is therefore context-dependent [11–16]. Moreover, from a patient-centred perspective, every consultation is unique and sets a specific context for the communication between the doctor and the patient [17–20].
So far, the role of the context in which doctor-patient communication takes place has hardly been accounted for in communication assessments [10, 21, 22]. However, the influence of context factors has been put forward as an explanation to why GPs achieve low scores on communication performance [23, 24]. If context factors can be taken into account when rating GP communication skills, we may move from a generic to a context specific assessment of GP communication performance.
In a previous explorative study, several context factors (CF) were identified on the level of the consultation that may well explain deviations from generic recommendations on communication . These context factors were related to the doctor, to the patient, and to the consultation. If, for example, the consultation has been initiated by the doctor to re-evaluate the patient’s condition, the doctor does not need to explore the patient’s request for help. Similarly, in case of an easily solved problem like ear wax blocking hearing, it seems spurious to explore emotions extensively. Yet some patients do fear the removal procedure, or may have questions concerning the consultation or treatment goal. Thus, context factors may explain why certain communication behaviour is absent but they do not justify its absence in all circumstances. This clearly reflects the dynamic way ‘context’ is to be understood . By incorporating contextual influences, communication performance assessment may gain in validity.
The current exploratory study aimed to find out how communication performance ratings change if context factors are explicitly taken into account. Our research question was: How does incorporating context factors influence the assessment of GP communication performance? In order to answer this question, we first examined which previously identified context factors were present in the currently studied GP consultations. Secondly, we explored how applying a context-specific protocol would affect communication scores. We expected GP communication scores to be significantly higher if context factors are explicitly taken into account .
The study was carried out between February and September 2010. We selected a sample of 40 consultations from a database of 808 videotaped Dutch GP consultations, recorded as part of a video-observation study performed by NIVEL (Netherlands institute for health services research) in 2007 – 2008 . The 40 GPs that participated in this NIVEL study have age, gender, practice type, and patients characteristics that are similar to the GP population characteristics in the Netherlands, although urban practices are over-represented . Firstly, out of the 40 GPs, 20 GPs were randomly selected and subsequently, from each of these GPs, the 4th and 5th consultations were then selected, which we felt was a reasonable trade-off between analyzing GPs and consultations. We excluded the first three consultations during which the GP may have had to get used to the video recording. This procedure ensured sufficient power and variation between doctors and consultations. A sample of 40 consultations would provide enough power to establish a minimal relevant difference between the two ratings of 0.45 (α = 0.05, β = 0.10) on item scores . The time interval between first (standard) and second (context-specific) rating was 5–6 months.
For the incorporation of CFs into the rating process, a context-specific rating protocol was developed by three researchers, each with different backgrounds (GP, communication researcher, communication trainer and assessor), to be used in addition to the MAAS-Global manual (Additional file 2). The protocol was developed on the basis of the relationships between CFs and the justifiable absence of communication skills found in our previous study . It accounts for the absence of certain communication skills as a result of the presence of one or more CFs and at the same time keeps the existing rating rules of the MAAS-Global Manual intact. Additional file 2 presents the protocol and the ascertained relationships. In order to stay in line with the MAAS-Global rating rules, the central assumptions in the protocol were:
If a sub-item is justifiably absent as a result of the presence of one or more CFs, it should not weigh on the item scores.
If an item is justifiably absent due to the presence of one or more CFs, it should be scored ‘Not applicable’.
List of context factors
Present in # consultations (%)
Doctor knows patient and his social context
Doctor knows patients’ medical history
Doctor knows patients’ way of communicating
Doctor is very experienced
Specific patient verbal behaviour
Specific patient non-verbal behaviour
Patient is also treated by other provider
Patient has a disease (diagnosis) or (recurrent) problem known to both doctor and patient
Patient is familiar with (physical) examination (PE)
First consultation in a series1
Follow-up consultation in a series2
Consultation in a series based on protocol (initiative by doctor)3
Consultation in preventive care (initiative doctor)4
Diagnosed problem is easily solved
Problem urgently needs medical care
Diagnosed problem is mainly psychosocial
There is more than one person (patient) present
Characteristics of physical examination:
- leaves room for talk
- requires full attention
Results of application of context-specific rating protocol
N times item is influenced (a)
Score without CF
Score with CF(b)
2. Follow-up consultation
3. Request for help‡
4. Physical Examination‡
10. Information giving†
Total mean score‡
Kappa was calculated, based on a separate sample of seven consultations taken from the dataset, to assess inter-rater variance in determining the presence of CFs between the first (GE) and the second rater (BA) (κ = 0.69). Apart from checking the presence of the previously found CFs, their frequencies were calculated in order to determine to what extent applying the context-specific rating protocol could be expected to influence ratings of the MAAS-Global items.
The rating of the GP communication performance was done both times by the same rater (GE) to exclude noise produced by heterogeneity of raters . To check for intra-rater consistency, kappa was calculated by twice scoring ten consultations that did not belong to the study sample, with a 6 month time lap between the two moments of assessment, using standard MAAS-Global rating rules (κ = 0.662).
To analyse the extent to which CFs influenced the rating process, we calculated the number of applicable MAAS-Global items per consultation, with and without accounting for CFs, as this number is used as the denominator to determine the overall score on the MAAS-Global [4, 24, 28, 33]. Moreover, because context factors may predict the absence of certain communication behaviour (and thus a (sub-)item)  but do not necessarily lead to the absence of that specific behaviour, we calculated the number of times the MAAS-Global items were potentially influenced by CFs, based on applying the rules from the rating protocol (see Additional file 2), and compared this to the actually influenced number as a result of the ratings (see Table 2).
Our expectation that the mean item scores in the context-specific rating would be higher was tested by calculating the direction of the change in scores with a paired t-test for repeated measurement in the same sample, using PASW Statistics 18, Release Version 18.0.3 (SPSS, Inc., 2010, Chicago, IL, http://www.spss.com). To determine the relevance of the difference between the two ratings, the effect size was calculated for the difference between the individual MAAS-Global item scores and for the difference between the mean sum scores per consultation, divided by their pooled initial standard deviations (SDs); a d of 0.2 was considered a small effect, a d of 0.5 as a moderate effect, and a d of 0.8 as a large effect [33–35].
The study was performed according to Dutch privacy legislation. The privacy regulation was approved by the Dutch Data Protection Authority. All participating GPs and patients signed an informed consent form before the recording of the consultation. According to Dutch legislation, approval by a medical ethics committee was not required for this study.
The 20 GPs and the patients in the research sample were comparable in gender, age, and practice type to those of the larger data set (35% female GPs, mean age 49 yrs (SD: 6.4) vs. 51 yrs (SD: 5.9)) . All context factors in the list were observed in the current sample, with frequencies varying from one time to 34 times. Table 1 presents the CFs observed in the consultations and their frequencies found in the study sample.
In 38 out of 40 consultations, CFs prompted the application of the context-specific rating protocol. In two consultations there was no CF present that required deviating from the MAAS-Global Manual. The mean number of CFs per consultation was 6.5 (range 4 – 12). As a consequence of incorporating context factors, the number of applicable items per consultation decreased from 12.2 to 11.8 (sub-items: from 40.6 to 37.7).
The potential change in scores was highest in eight out of thirteen items (items 3, 4, 5, 6, 8, 9, 10, and 12) whereas little to no change would be expected in five items (items 1, 2, 7, 11, and 13) (Table 2).
As a result of applying the context-specific rating protocol, a significant increase was found in ten out of the thirteen mean item scores whereas one item (item 7) showed a decrease (Table 2). In the items 2 (Follow-up consultation) and 6 (Management) the difference in scores was not significant. The mean overall score in the standard rating was 2.98, while in the context-specific rating it was 3.66 (p < 0.00). Effect sizes were large for three items and moderate for another four items. In the remaining six items effect sizes were low (Table 2). Effect size in the mean overall score was large (0.84).
This study indicates that explicitly incorporating context factors into communication assessment in a protocolized way leads to a significantly lower number of applicable MAAS-Global items per consultation, and to higher item scores. By applying the protocol, the mean overall score found in our study rose above the mean minimum standard score of 3.17, which is the level set for the MAAS-Global scores to represent ‘adequate GP communication behaviour’ in a study by Hobma . However, consistent with other recent findings on doctor communication patterns [36–38], the GP scores in our study on the items Request for help, Management, Exploring and Emotions are below the minimum standard. These are important aspects in GP-patient communication and need attention in postgraduate GP training and continuing professional development (CPD).
In the standard rating protocol, the absence of criterion behaviour is penalized by a low item score, whereas judging a sub-item to be justifiably absent will lead to relatively higher item scores. In our study, most changes in item scores were as we expected them but the results in items 1, 6, 7, 11 and 13 were unforeseen. The unexpected results in these items may be due to rater leniency in the second rating, although the change did not go in the same direction in all items. The significant change in item 7 (Closure) can also be explained by the lack of clarity in the MAAS-Global protocol where the assessor has to score a question near the end of a consultation: either under Closure as ‘general question‘, or under Management (item 6) as ‘asking for patient’s response‘. An explanation for the not-significant change in item 6 (Management) may be that the potential change was not acknowledged in the actual rating: absence of sub-items was not justified or sub-items were not absent. Apparently, in our sample this item did not change under the influence of context factors as much as we expected, even if closing remarks were scored more often under Management the second round. However, for the results to be corroborated, a more robust study is necessary. As our study is exploratory, the numerical changes we found must definitely be interpreted with diligence.
The presence of contextual factors identified before is also confirmed, as all previously identified CFs in GP consultations were also found in the current sample . However, CF frequencies found in this study cannot be generalized, as the sample is not sufficiently large. Although the representativity of GPs and patients in the sample is good, the frequencies only represent the consultations in the sample and were needed in this study to explore the magnitude of the effect on item scores. In our previous study, CFs were identified on the basis of inductive reasoning, using several rounds of systematic analysis to establish what factors could explain low scores.
Although there is wide recognition of the fact that professional competence is context-dependent, this aspect has so far been neglected in assessment of GP communication with patients in authentic consultations [6, 7, 10, 11]. Now that we have found indications that, especially in workplace-based assessment, CFs can and need to be incorporated explicitly in judging communication performance, this way of assessment may enhance the credibility of communication training and assessment, not only for GPs but also in GP specialty training. Studies on GP trainee experiences [39, 40] show that there is a need for this. The application of a context-specific protocol can do justice to clinical practice as it acknowledges the context-specificity of GP (trainee) communication in their surgeries. It may also contribute to removing the artifact that is created by merely looking at the presence of generic communication skills.
From recent research, we know that experts, when assessing trainee performance in practice, implicitly take contextual information into account [41–43]. By explicating CFs and by designing a context-specific assessment protocol, we may have unveiled part of the internal and implicit process of weighing contextual information. We have made this process explicit and thus open to empirical research. However, although the context-specific rating protocol was based on this study and developed by three researchers with different backgrounds, a limitation is that the protocol has not yet been reviewed by other GPs.
Explicitly accounting for CFs in workplace-based communication assessment can not only make performance scores more transparent, it may also raise their external validity. The characteristics of the various consultation-related CFs reflect current developments in family medicine in which a growing number of follow-up and preventive consultations concerning chronic disease management is seen for which protocols have been developed [44–46]. However, although the rating process using the context-specific protocol leads to a more refined outcome, it also encompasses a long list of items to ‘tick’. Adding extra criteria to the assessment process may render it less feasible in practice. Raters, however, need to be sensitized for context influences and do it justice in their assessments, and they can be trained to do so.
To determine the presence of context factors, some subjectivity is necessarily involved. As is shown in other studies on assessment of clinical performance, expert raters recognize context as an important factor modulating their assessment of, for instance, resident performance [41, 43, 47]. Although we chose optimal rater consistency by having the same rater for both the first and second rating of communication performance and substantial inter-rater agreement between two raters was found in determining the presence of CFs in GP consultations, the inherent limitation is that this may have caused a bias to corroborate the hypothesized findings, both in the first and in the second rating. Therefore, keeping in mind the exploratory nature of the study, the results should be interpreted with care.
More research is needed to validate the currently established context factors and the protocol that we developed more rigorously. Moreover, as contexts change with time, also the way it will influence GP communication in their patient encounters will change. Therefore, context factors as well as a context-specific rating protocol will need to be updated regularly.
Assessment of professional performance is a complex enterprise, as so many behaviours seem to depend on contextual factors . We think that context factors should rather be considered as ‘signals’ than as ‘noise’ in GP communication assessment and that the results of our study in incorporating them are promising. Now, a more robust study can be carried out to find out if our results are generalisable. The context-specific rating protocol should be reviewed by other GPs, and other raters should apply the protocol to assess GP consultations to further validate our findings. Furthermore, for validation of the protocol, research should focus on experts rating communication and the way they incorporate contextual information in assessing communication performance.
Although we do not claim to have found all relevant context factors in GP communication, the presence of CFs we did find and their influences on GP communication plead for a more context-specific approach of communication assessment, as has been advocated before [7, 10]. Evidently, communication competence “is not defined solely by the presence or absence of specific behaviour, but rather by the presence and timing of effective verbal and non-verbal behaviour within the context of individual interactions with patients or families” .
We thank the patients and GPs who participated in the study. We also thank the Dutch Ministry of Health for funding the collection and videotaping of routine GP consultations, and the SBOH Foundation for funding this study.
Presentation in professional meetings: EACH congress September 2012 St. Andrews.
Funding: The study was funded by the SBOH Foundation, employer of Dutch Family Medicine Residents and funder of the national residency program for GPs in the Netherlands.
- Hulsman RL: Shifting goals in medical communication. Determinants of goal detection and response formation. Patient Educ Couns. 2009, 74 (3): 302-308. 10.1016/j.pec.2008.12.001.View ArticlePubMedGoogle Scholar
- Schirmer JM, Mauksch L, Lang F, Marvel MK, Zoppi K, Epstein RM, Brock D, Pryzbylski M: Assessing communication competence: A review of current tools. Fam Med. 2005, 37 (3): 184-192.PubMedGoogle Scholar
- Zoppi K, Epstein RM: Is communication a skill? Communication behaviors and being in relation. Fam Med. 2002, 34 (5): 319-324.PubMedGoogle Scholar
- Ram P, Grol R, Rethans JJ, Schouten B, van der Vleuten C, Kester A: Assessment of general practitioners by video observation of communicative and medical performance in daily practice: issues of validity, reliability and feasibility. Med Educ. 1999, 33 (6): 447-454. 10.1046/j.1365-2923.1999.00348.x.View ArticlePubMedGoogle Scholar
- Cegala DJ, Lenzmeier Broz S: Physician communication skills training: a review of theoretical backgrounds, objectives and skills. Med Educ. 2002, 36 (11): 1004-1016. 10.1046/j.1365-2923.2002.01331.x.View ArticlePubMedGoogle Scholar
- Egener B, Cole-Kelly K: Satisfying the patient, but failing the test. Acad Med. 2004, 79 (6): 508-510. 10.1097/00001888-200406000-00003.View ArticlePubMedGoogle Scholar
- Salmon P, Young B: Creativity in clinical communication: from communication skills to skilled communication. Med Educ. 2011, 45 (3): 217-226. 10.1111/j.1365-2923.2010.03801.x.View ArticlePubMedGoogle Scholar
- de Haes H, Bensing J: Endpoints in medical communication research, proposing a framework of functions and outcomes. Patient Educ Couns. 2009, 74 (3): 287-294. 10.1016/j.pec.2008.12.006.View ArticlePubMedGoogle Scholar
- Feldman-Stewart D, Brundage MD, Tishelman C, Team SC: A conceptual framework for patient-professional communication: An application to the cancer context. Psycho-Oncol. 2005, 14 (10): 801-809. 10.1002/pon.950.View ArticleGoogle Scholar
- Bensing J, van Dulmen S, Tates K: Communication in context: new directions in communication research. Patient Educ Couns. 2003, 50 (1): 27-32. 10.1016/S0738-3991(03)00076-4.View ArticlePubMedGoogle Scholar
- Eva KW: On the generality of specificity. Med Educ. 2003, 37 (7): 587-588. 10.1046/j.1365-2923.2003.01563.x.View ArticlePubMedGoogle Scholar
- Baig LA, Violato C, Crutcher RA: Assessing clinical communication skills in physicians: are the skills context specific or generalizable. BMC Med Educ. 2009, 9: 22-10.1186/1472-6920-9-22.View ArticlePubMedPubMed CentralGoogle Scholar
- Veldhuijzen W: Challenging the Patient-Centred Paradigm. 2011, Maastricht: Maastricht UniversityGoogle Scholar
- Winefield HR, Murrell TG, Clifford JV, Farmer EA: The usefulness of distinguishing different types of general practice consultation, or are needed skills always the same?. Fam Pract. 1995, 12 (4): 402-407. 10.1093/fampra/12.4.402.View ArticlePubMedGoogle Scholar
- Salmon P, Mendick N, Young B: Integrative qualitative communication analysis of consultation and patient and practitioner perspectives: towards a theory of authentic caring in clinical relationships. Patient Educ Couns. 2011, 82 (3): 448-454. 10.1016/j.pec.2010.10.017.View ArticlePubMedGoogle Scholar
- Allen J, Gay B, Crebolder H, Heyrman J, Svab I, Ram P, Evans P: The European definitions of the key features of the discipline and core competencies. Br J Gen Pract. 2002, 52 (479): 526-527.PubMedPubMed CentralGoogle Scholar
- Kenny DA, Veldhuijzen W, Weijden T, Leblanc A, Lockyer J, Legare F, Campbell C: Interpersonal perception in the context of doctor-patient relationships: a dyadic analysis of doctor-patient communication. Soc Sci Med. 2010, 70 (5): 763-768. 10.1016/j.socscimed.2009.10.065.View ArticlePubMedGoogle Scholar
- Stewart M: Towards a global definition of patient centred care - The patient should be the judge of patient centred care. Brit Med J. 2001, 322 (7284): 444-445. 10.1136/bmj.322.7284.444.View ArticlePubMedPubMed CentralGoogle Scholar
- Pilnick A, Hindmarsh J, Gill VT: Beyond ‘doctor and patient’: developments in the study of healthcare interactions. Sociol Health Illn. 2009, 31 (6): 787-802. 10.1111/j.1467-9566.2009.01194.x.View ArticlePubMedGoogle Scholar
- Crossley J, Eiser C, Davies HA: Children and their parents assessing the doctor-patient interaction: a rating system for doctors’ communication skills. Med Educ. 2005, 39 (8): 820-828. 10.1111/j.1365-2929.2005.02230.x.View ArticlePubMedGoogle Scholar
- Di Blasi Z, Harkness E, Ernst E, Georgiou A, Kleijnen J: Influence of context effects on health outcomes: a systematic review. Lancet. 2001, 357 (9258): 757-762. 10.1016/S0140-6736(00)04169-6.View ArticlePubMedGoogle Scholar
- Durning SJ, Artino AR, Pangaro LN, van der Vleuten C, Schuwirth L: Perspective: redefining context in the clinical encounter: implications for research and training in medical education. Acad Med. 2010, 85 (5): 894-901. 10.1097/ACM.0b013e3181d7427c.View ArticlePubMedGoogle Scholar
- Kramer AWM, Dusman H, Tan LHC, Jansen JJM, Grol R, van der Vleuten CPM: Acquisition of communication skills in postgraduate training for general practice. Med Educ. 2004, 38 (2): 158-167. 10.1111/j.1365-2923.2004.01747.x.View ArticlePubMedGoogle Scholar
- Hobma SO, Ram PM, Muijtjens AMM, Grol R, van der Vleuten CPM: Setting a standard for performance assessment of doctor-patient communication in general practice. Med Educ. 2004, 38 (12): 1244-1252. 10.1111/j.1365-2929.2004.01918.x.View ArticlePubMedGoogle Scholar
- Essers G, van Dulmen S, van Weel C, van der Vleuten C, Kramer A: Identifying context factors explaining physician’s low performance in communication assessment: an explorative study in general practice. BMC Fam Pract. 2011, 12: 138-10.1186/1471-2296-12-138.View ArticlePubMedPubMed CentralGoogle Scholar
- Noordman J, Verhaak P, van Beljouw I, van Dulmen S: Consulting room computers and their effect on general practitioner-patient communication. Fam Pract. 2010, 27 (6): 644-651. 10.1093/fampra/cmq058.View ArticlePubMedGoogle Scholar
- Jones R, Schellevis F, Westert G: The changing face of primary care: the second Dutch national survey. Fam Pract. 2004, 21 (6): 597-598. 10.1093/fampra/cmh603.View ArticlePubMedGoogle Scholar
- Reinders ME, Blankenstein AH, van der Horst HE, Knol DL, Schoonheim PL, van Marwijk HWJ: Does patient feedback improve the consultation skills of general practice trainees? A controlled trial. Med Educ. 2010, 44 (2): 156-164. 10.1111/j.1365-2923.2009.03569.x.View ArticlePubMedGoogle Scholar
- van Thiel J, Kraan HF, Van Der Vleuten CP: Reliability and feasibility of measuring medical interviewing skills: the revised Maastricht History-Taking and Advice Checklist. Med Educ. 1991, 25 (3): 224-229. 10.1111/j.1365-2923.1991.tb00055.x.View ArticlePubMedGoogle Scholar
- Ram P, van Thiel J, van Dalen J: MAAS-Global Manual 2000. 2003, In. Maastricht: Maastricht UniversityGoogle Scholar
- Veldhuijzen W, Ram PM, van der Weijden T, Wassink MR, van der Vleuten CPM: Much variety and little evidence: a description of guidelines for doctor-patient communication. Med Educ. 2007, 41 (2): 138-145. 10.1111/j.1365-2929.2006.02667.x.View ArticlePubMedGoogle Scholar
- Timmerman AAR, Cals P, J.W.L.: Competentcy-based education of doctor-patient communication in Dutch General Practitioner specialty training. Tijdschrift voor Medisch Onderwijs. 2011, 30 (4): 130-142. 10.1007/s12507-011-0037-5.View ArticleGoogle Scholar
- Reinders ME, Blankenstein AH, van Marwijk HWJ, Knol DL, Ram P, van der Horst HE, de Vet HCW, van der Vleuten CPM: Reliability of consultation skills assessments using standardised versus real patients. Med Educ. 2011, 45 (6): 578-584. 10.1111/j.1365-2923.2010.03917.x.View ArticlePubMedGoogle Scholar
- Hojat M, Xu G: A visitor’s guide to effect sizes - Statistical significance versus practical (clinical) importance of research findings. Adv Health Sci Educ. 2004, 9 (3): 241-249.View ArticleGoogle Scholar
- Dunlap WP, Cortina JM, Vaslow JB, Burke MJ: Meta-analysis of experiments with matched groups or repeated measures designs. Psychol Meth. 1996, 1 (2): 170-177.View ArticleGoogle Scholar
- Mjaaland TA, Finset A: Frequency of GP communication addressing the patient’s resources and coping strategies in medical interviews: a video-based observational study. BMC Fam Pract. 2009, 10: 49-10.1186/1471-2296-10-49.View ArticlePubMedPubMed CentralGoogle Scholar
- van den Brink-Muinen A, Caris-Verhallen W: Doctors’ responses to patients’ concerns: testing the use of sequential analysis. Epidemiol Psichiatr Soc. 2003, 12 (2): 92-97. 10.1017/S1121189X0000614X.View ArticlePubMedGoogle Scholar
- Epstein RM, Hadee T, Carroll J, Meldrum SC, Lardner J, Shields CG: “Could this be something serious?” - Reassurance, uncertainty, and empathy in response to patients’ expressions of worry. J Gen Intern Med. 2007, 22 (12): 1731-1739. 10.1007/s11606-007-0416-9.View ArticlePubMedPubMed CentralGoogle Scholar
- Veldhuijzen W, Ram PM, van der Weijden T, van der Vleuten CPM: Communication guidelines as a learning tool: an exploration of user preferences in general practice. Patient Educ Couns. 2013, 90 (2): 213-219. 10.1016/j.pec.2012.10.002.View ArticlePubMedGoogle Scholar
- Van Nuland M: Vocational trainees’ views and experiences regarding the learning and teaching of communication skills in general practice. Patient Educ Couns. 2009,10.1016/j.pec.2009.05.002.Google Scholar
- Govaerts MJB, Schuwirth LWT, Van der Vleuten CPM, Muijtjens AMM: Workplace-based assessment: effects of rater expertise. Adv Health Sci Educ. 2011, 16 (2): 151-165. 10.1007/s10459-010-9250-7.View ArticleGoogle Scholar
- Kogan JR, Conforti L, Bernabeo E, Iobst W, Holmboe E: Opening the black box of clinical skills assessment via observation: a conceptual model. Med Educ. 2011, 45 (10): 1048-1060. 10.1111/j.1365-2923.2011.04025.x.View ArticlePubMedGoogle Scholar
- Durning S, Artino AR, Pangaro L, van der Vleuten CP, Schuwirth L: Context and clinical reasoning: understanding the perspective of the expert’s voice. Med Educ. 2011, 45 (9): 927-938. 10.1111/j.1365-2923.2011.04053.x.View ArticlePubMedGoogle Scholar
- Bosch M, van der Weijden T, Grol R, Schers H, Akkermans R, Niessen L, Wensing M: Structured chronic primary care and health-related quality of life in chronic heart failure. BMC Health Serv Res. 2009, 91: 104-10.1186/1472-6963-9-104.View ArticleGoogle Scholar
- Lugtenberg M, Burgers JS, Westert GP: Effects of evidence-based clinical practice guidelines on quality of care: a systematic review. Qual Saf Health Care. 2009, 18 (5): 385-392. 10.1136/qshc.2008.028043.View ArticlePubMedGoogle Scholar
- World Health Organization: 2008–2013 Action Plan for the Global Strategy for the Prevention and Control of Noncommunicable Diseases. 2008, In. Geneva: WHOGoogle Scholar
- Young B, Ward J, Forsey M, Gravenhorst K, Salmon P: Examining the validity of the unitary theory of clinical relationships: comparison of observed and experienced parent-doctor interaction. Patient Educ Couns. 2011, 85 (1): 60-67. 10.1016/j.pec.2010.08.027.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2296/14/65/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.