Skip to main content

Mix of methods is needed to identify adverse events in general practice: A prospective observational study

Abstract

Background

The validity and usefulness of incident reporting and other methods for identifying adverse events remains unclear. This study aimed to compare five methods in general practice.

Methods

In a prospective observational study, with five general practitioners, five methods were applied and compared. The five methods were physician reported adverse events, pharmacist reported adverse events, patients' experiences of adverse events, assessment of a random sample of medical records, and assessment of all deceased patients.

Results

A total of 68 events were identified using these methods. The patient survey accounted for the highest number of events and the pharmacist reports for the lowest number. No overlap between the methods was detected. The patient survey accounted for the highest number of events and the pharmacist reports for the lowest number.

Conclusion

A mix of methods is needed to identify adverse events in general practice.

Peer Review reports

Background

Patient safety is important in primary care, as most patients and most of their health problems are treated in this setting [1]. Adverse events in primary care occur between five and 80 times per 100 000 consultations [2]. General practitioners (GPs) were positive about reporting adverse events [3], but the validity and usefulness of incident reporting systems remains unclear [[4], page 3]. A range of methods is available for identifying adverse events, such as review of medical records and case reviews of deceased patients [5–7]. The aim of the presented study was to compare five different methods for identifying adverse events in general practice with respect to the number and type of identified events, the patient subgroups affected, and the agreement of events across the methods.

Methods

Study design and setting

A prospective observational study was performed focused on five GPs in two practices in a period of five months (May to October 2006). A total of approximately 8250 patients were registered with the two practices. The ethical committee of the Radboud University Nijmegen Medical Centre approved the study.

Measures

We defined an adverse event as an unintentional event with actual or potential harm to the patients' health status. This broad definition is consistent with the International Patient Safety Event Classification of the World Health Organization: "a process or act of omission or commission that resulted in hazardous health care conditions and/or unintended harm to the patient" [8]. The five methods were: GPs reported adverse events, pharmacist reported adverse events, patients' experiences of adverse events, assessment of a random sample of medical records, and assessment of all deceased patients.

GP reported adverse events

The GPs recorded all events during a five-month period. On the basis of an existing international taxonomy for errors in general practice [9] a simplified computerised registration form was made. The GPs registered event date, birth date of patient, gender, event category (practice administration (archive; medical record; appointment; other), diagnostic (wrong diagnosis; delayed diagnosis; missed diagnosis; other), therapeutic (wrong, incomplete; delayed; none, though it should be; other), communication (with patients; with caregivers; other)), and additional remarks and/or context.

Pharmacist reported adverse events

The pharmacist recorded events from her point of view in the same period. An adjusted form was developed for this purpose. Event date, birth date of patient, gender, practice, event category (prescribing error (wrong prescription, wrong administration; wrong dose; other), adverse reaction (adverse reaction; allergic reaction; overdose; interaction; contra-indication; other), dispensing error (too late; wrong medicine; wrong dose; other)), and additional remarks or context were recorded.

Patient reported adverse events

In the waiting room of the two practices samples of 50 patients, consecutively visiting the practice, were invited to complete a questionnaire on experienced problems with safety of their health care in the previous six months. A drop box was used to collect the completed questionnaires. Questions were derived from items of the Medical Harvard Study, [10] and from questions of two survey studies. [11, 12] Questions had to be answered with yes or no; and asked whether something had gone wrong in the care they had received from their GP during the past half year (see table 1 for the complete questionnaire). The questionnaire guaranteed anonymity of participating patients.

Table 1 Feasibility, yield, usefulness and reliability per method

Assessment of medical records

Thirty medical records per GP (in total 150 medical records) were randomly selected (using a list of random numbers) from patients who had visited their GP in the observation period. Anonymous medical records, containing the information from this period were printed out. Two clinical researchers examined the information independently [RWo, RWe]. They scrutinised the records for indications of events and, when found, categorised the event (errors in office administration, diagnosis, treatment or communication with their subcategories); and added demographic data of the patient (birth date, gender). Subsequently the GPs discussed their findings and reached consensus.

Assessment of all deceased patients

Both practices had a registration of deceased patients from which the medical records were retrieved of all patients who had died in the period May-October 2006. One GP examined the medical records of these patients for events. The same registration form and analysis procedure as for the audits of medical records was used.

Data-analysis

Numbers of events per category per method and overall were added up and percentages were calculated. Age range, mean age, age category (<50 and > = 50 years old) and percentage of women were calculated per method and overall.

A qualitative analysis of events will be published elsewhere. [13]

Results

Table 2 presents the events reported by the five methods. A total of 68 events were identified using these methods. Each of the methods provided events that were not found with other methods; no overlap between the methods was detected. Mostly women were affected, in almost two-third of the registered events. Patients over 50 years and patients under 50 year were almost equally affected, but the age group below 18 was rarely involved. An exception to this general trend was the subgroup of events based patient registration, which involved mostly patients under 50 years of age.

Table 2 Overview of events, characteristics of patients involved and event categories

GP reported events

GPs registered 20 events in five months; in these months there were 4095 patients who had visited the practice, resulting in almost 5 events per 1000 patients visiting the practice. Most of these events concerned women aged 50 years or older. GPs registered little events that concerned treatment.

Pharmacist reported events

The pharmacists registered six events per 16320 prescriptions (including repeated prescriptions), all related to female patients. Of these there were five prescribing errors and one known allergic reaction.

Patient reported events

Twenty-seven patients answered positive to one or more items of the patient questionnaire. In total, 78 positive answers were given, it is unclear whether these answers referred to one or more events. Patients answering positive to questions were slightly younger compared to all responders (43,8 vs. 45,5 years old). By far most often (16/78) mentioned was a breach of confidentially of their medical record during treatment by their GP. Next, a lack of respect by GP or practice assistant was reported (8/78), followed by a delay in diagnosis (5/78), inappropriate drug prescribed (5/78) or inappropriate advice given (5/78) and a wrong appointment at the practice (5/78).

Assessment of medical records

There had been 4,095 patients consulting the practices during the study period, from which 150 medical records were randomly selected. Analysis resulted in the finding of 11 events, all errors of treatment and communication.

Assessment of deceased patients

During the study period 28 patients had died. One medical record was not available for examination, as this patient had not given permission. This method generated between one or two events per ten deceased patients.

Discussion

All five methods proved to identify a number of adverse events. The patient survey accounted for the highest number of events and the pharmacist reports for the lowest number. All methods resulted in a variety of events, except for the pharmacist reports, which only referred to pharmaceutical treatment. The identified events referred to adult male and female patients of all ages, but events on children were very seldom reported. Events based on patient registration mostly involved individuals aged 50 years or younger. There was no overlap between the methods regarding the identified events.

A systematic review of methods to identify adverse events in health care concluded that "the available methods have widely differing purposes, strengths and weaknesses and must be considered as complementing each other by providing different levels of qualitative and quantitave information" [[4], page 4]. Incident reporting systems have the advantage of being not as time-consuming as formal studies, but they are likely to underestimate the number of adverse events (numerator) while the number of opportunities for incidents (denominator) remains unknown. A recent study in hospitals also concluded that different methods should be used in order to provide an adequate assessment of clinical adverse events [14].

Our study in general practice suggested that GPs did not report all adverse events, as compared to other sources. It was difficult to assess the validity of the GP reported events because of the lack of overlap with other methods. Event reporting by GPs is probably important for raising awareness and a safety culture, but it is unlikely to be a comprehensive method for identifying adverse events. None of the GPs had a feeling of suspicion with registration. They did not think the registered events would be used against them. They all felt it was difficult to remember to register events due to daily routine working procedures and time pressure.

Almost three in ten patients reported on health safety issues, and patients as a group reported a substantial number of adverse events. It was unclear whether the checked items of the patient questionnaire referred to one or more health safety issues; we assumed these were from 27 events (thus one event per patient), in order not to overestimate number of events. In this study almost 60% of patient reported events concerned psychological harm or harm in trust or confidence. Another study showed that less than a third of reported harm was physical [15]. Patients registering events may reveal blind spots in care provided by the GP. The questionnaire seemed quite easy to complete and its analysis asked for a relatively low time investment. However, the validity and usefulness of patient reported incidents needs further research [16, 17].

Inaccuracies of medical record are well documented [17]. This method generated different types of adverse events, although the number in diagnostic events was rather low. It took a large time-investment to go through the medical records.

The reporting of adverse events by a pharmacist generated relatively few events. A study showed that community pharmacists correct 1% of the prescriptions [19], which obviously influences the number of remaining events. The low numbers may explained by the computerised systems, which including monitoring for potential contra-indications and interaction of drugs.

The audit of medical records of deceased patients was feasible, but the number of identified adverse events was low, and therefore its yield was restricted.

The limitations of this study should be recognized. The number and type of adverse events seemed to depend on the individual who registered. One could imagine asking other (para)medical caregivers (physiotherapists, midwifes, etc) to register as well. Also, in the communication with hospitals and referrals to specialists, things may go wrong. Specialists could therefore be a registration source of adverse events for GPs and this probably goes the other way around as well. Future studies should take this into account. Registration of events stays difficult as one kept in doubt whether an event is an adverse event or something that is generally accepted as inherent to GP care. This might have negatively influenced the number of registered events. Patients may also experience not every incident in their care as an adverse event. Whether this was the case may be influenced by the response of the doctor towards the event [20].

Patient safety has attracted "a level of public interest that the rest of the quality-improvement field in health care has failed to excite" [21]. This may be explained by a fascination for the accidental death, despite the prolonged uncertainty about the "true" estimates of mortality resulting from medical error [21]. In our study, none of the identified adverse events was associated with mortality.

Conclusion

The public interest for patient safety has led to a number of potentially valuable initiatives to identify and prevent adverse events. None of the methods to identify adverse events proved to be superior. We suggest that more research is needed on the various approaches in general practice before their wide scale implementation can be recommended.

References

  1. Green LA, Fryer GE, Yawn BP, Lanier D, Dovey SM: The ecology of medical care revisited. N Engl J Med. 2001, 344: 2021-2025. 10.1056/NEJM200106283442611.

    Article  CAS  PubMed  Google Scholar 

  2. Sandars J, Esmail A: The frequency and nature of medical error in primary care: understanding the diversity across studies. Fam Pract. 2003, 20: 231-236. 10.1093/fampra/cmg301.

    Article  PubMed  Google Scholar 

  3. Mikkelsen TH, Sokolowski I, Olesen F: General practitioners' attitudes toward reporting and learning from adverse events: results from a survey. Scand J Prim Health Care. 2006, 24: 27-32. 10.1080/02813430500508330.

    Article  PubMed  Google Scholar 

  4. WHO: Strengths and weaknesses of available methods for assessing the nature and scale of harm caused by the health system: a systematic literature review. Geneva. 2003

    Google Scholar 

  5. Britt H, Miller GC, Steven ID, Howarth GC, Nicholson PA, Bhasale AL, Norton KJ: Collecting data on adverse events: a method for monitoring incidents in general practice. Fam Pract. 1997, 14: 101-6. 10.1093/fampra/14.2.101.

    Article  CAS  PubMed  Google Scholar 

  6. Bhasale A: The wrong diagnosis: identifying causes of potentially adverse events in general practice using incident monitoring. Fam Pract. 1998, 15: 308-18. 10.1093/fampra/15.4.308.

    Article  CAS  PubMed  Google Scholar 

  7. Holden J, O'Donnell S, Brindley J, Miles L: Analysis of 1263 deaths in four general practices. Br J Gen Pract. 1998, 48: 1409-12.

    CAS  PubMed  PubMed Central  Google Scholar 

  8. World Health Organization, World Alliance for Patient Safety: The Conceptual Framework of an International Patient Safety Event Classsification. 2008, Copenhagen: WHO report, (Executive Summary)

    Google Scholar 

  9. Makeham MA, Dovey SM, County M, Kidd MR: An international taxonomy for errors in general practice: a pilot study. MJA. 2002, 177: 68-72.

    PubMed  Google Scholar 

  10. Leape LL, Brennan TA, Laird N, Lawthers AG, Localio AR, Barnes BA, Hebert L, Newhouse JP, Weiler PC, Hiatt H: The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med. 1991, 324: 377-384.

    Article  CAS  PubMed  Google Scholar 

  11. Blendon RJ, DesRoches CM, Brodie M, Benson JM, Rosen AB, Schneider E, Altman DE, Zapert K, Herrmann MJ, Steffenson AE: Views of practicing physicians and the public on medical errors. N Engl J Med. 2002, 347: 1933-40. 10.1056/NEJMsa022151.

    Article  PubMed  Google Scholar 

  12. Agoritsas T, Bovier PA, Perneger TV: Patient reports of undesirable events during hospitalization. J Gen Intern Med. 2005, 20: 922-28. 10.1111/j.1525-1497.2005.0225.x.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Wetzels R, Wolters R, Van Weel C, Wensing M: Harm caused by adverse events in primary care: a clinical observational study. J Clin Eval Pract.

  14. Olsen S, Neale G, Schwab K, Psaila B, Patel T, Chapman EJ, Vincent C: Hospital staff should use more than one method to detect adverse events and potential adverse events: incident reporting, pharmacist surveillance and local real-time record review may all have a place. Qual Saf Health Care. 2007, 16: 40-4. 10.1136/qshc.2005.017616.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Kuzel AJ, Woolf SH, Gilchrist VJ, Engel JD, LaVeist TA, Vincent C, Frankel RM: Patient reports of preventable problems and harms in primary health care. Fam Med. 2004, 2: 333-40. 10.1370/afm.220.

    Article  Google Scholar 

  16. Davis RE, Jacklin R, Sevdalis N, Vincent CA: Patient involvement in patient safety: what factors influence patient participation and engagement?. Health Expec. 2007, 10: 259-267. 10.1111/j.1369-7625.2007.00450.x.

    Article  Google Scholar 

  17. Buetow S, Elwyn G: Patient safety and patient error. Lancet. 2007, 369: 158-159. 10.1016/S0140-6736(07)60077-4.

    Article  PubMed  Google Scholar 

  18. Stange KC, Zyzanski SJ, Smith TF, Kelly R, Langa DM, Flocke SA, Jaén CR: How valid are medical records and patient questionnaires for physician profiling and health services research? A comparison with direct observation of patients visits. Med Care. 1998, 36: 851-67. 10.1097/00005650-199806000-00009.

    Article  CAS  PubMed  Google Scholar 

  19. Elder NC, Meulen von der MB, Cassedy A: The identification of medical errors by family physicians during outpatient visits. Ann Fam Med. 2004, 2: 125-129. 10.1370/afm.16.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Van Vorst RF, Araya-Guerra R, Felzien M, Fernald D, Elder N, Duclos C, Westfall JM: Rural community members' perceptions of harm from medical mistakes: a High Plains Research Network (HPRN) Study. J Am Board Fam Med. 2007, 20: 135-43. 10.3122/jabfm.2007.02.060147.

    Article  PubMed  Google Scholar 

  21. Buurma H, de Smet PA, Hoff van den OP, Egberts AC: Nature, frequency and determinants of prescription modifications in Dutch community pharmacies. Br J Clin Pharmacol. 2001, 52: 85-91. 10.1046/j.0306-5251.2001.01406.x.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Brennan TA, Gawande A, Thomas E, Studdert D: Accidental deaths, saved lives, and improved quality. N Engl J Med. 2005, 353: 1405-1409. 10.1056/NEJMsb051157.

    Article  CAS  PubMed  Google Scholar 

Pre-publication history

Download references

Acknowledgements

The participating general practitioners, patients, pharmacist are gratefully acknowledged. The Netherlands Organisation for Health Research and Development (ZonMW) provided the financial resources for this study.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Michel Wensing.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

RWe performed the study according to protocol and wrote the manuscript. RWe and RWo collected and analysed data. MW and CvW initiated and supervised the study, and developed the study protocol. All authors read and approved the final version of the manuscript.

Rights and permissions

Open Access This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Wetzels, R., Wolters, R., van Weel, C. et al. Mix of methods is needed to identify adverse events in general practice: A prospective observational study. BMC Fam Pract 9, 35 (2008). https://doi.org/10.1186/1471-2296-9-35

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1471-2296-9-35

Keywords