Research article | Open | Open Peer Review | Published:
Effect of accreditation on the quality of chronic disease management: a comparative observational study
BMC Family Practicevolume 15, Article number: 179 (2014)
Practice accreditation is widely used to assess and improve quality of healthcare providers. Little is known about its effectiveness, particularly in primary care. In this study we examined the effect of accreditation on quality of care regarding diabetes, chronic obstructive pulmonary disease (COPD) and cardiovascular disease (CVD).
A comparative observational study with two cohorts was performed. We included 138 Dutch family practices that participated in the national accreditation program for primary care. A first cohort of 69 practices was measured at start and completion of a 3-year accreditation program. A second cohort of 69 practices was included and measured simultaneously with the final measurement of the first cohort. In separate multilevel regression analyses, we compared both within-group changes in the first cohort and between-groups differences at follow-up (first cohort) and start (second cohort). Outcome measures consisted of 24 systematically developed indicators of quality of care in targeted chronic diseases.
In the within-group comparison, we found improvements on 6 indicators related to diabetes (feet examination, cholesterol measurement, lipid lowering medication prescription) and COPD (spirometry performance, stop smoking advice). In the between-groups comparison we found that first cohort practices performed better on 4 indicators related to diabetes (cholesterol outcome) and CVD (blood pressure outcome, smoke status registration, glucose measurement).
Improvements of the quality of primary care for patients with chronic diseases were found, but few could be attributed to the accreditation program. Further development of accreditation is needed to enhance its effectiveness on chronic disease management.
A range of strategies has been developed to assess and improve healthcare for patients with chronic diseases . A widely used strategy is accreditation of healthcare providers ,. Accreditation affects the institution or practice and is offered more or less voluntary, while certification focuses on a specific norm that should be reached by individuals or particular services ,. A key element of accreditation is audit and feedback. A Cochrane review suggested that an audit and feedback system has a small positive effect on quality of care overall , but the added value of accreditation was not considered. Greenfield et al.  suggested in their systematic review that accreditation can promote change, for example through the opportunity to reflect on organizational performance, and that it has an effect on professional development. However, they reported inconsistent findings regarding quantifiable effects of accreditation on measures of clinical processes and outcomes. Few rigorous evaluations of effectiveness of accreditation are available, particularly in primary care -. O'Beirne et al.  performed a review of peer-reviewed and grey literature regarding accreditation in primary care. They found indications that accreditation may improve both organization and outcomes of care, but conclude that more research is needed. Szecsenyi et al.  examined an accreditation program that focuses on practice management; they found improvements on several quality and safety measures regarding complaint management, analysis of critical incidents and quality development. Given the shortage of controlled evaluations, more research on the effects of well-defined accreditation programs is required.
In this study we assessed an accreditation program for primary care practices ,, set up by the Dutch college of General Practitioners in 2005. This program strongly focuses on the educational value of audit and feedback and participation is voluntary. However, part of the program does require meeting specific norms, for example on hygiene, first aid equipment and accessibility. The program addresses clinical care as well as practice management. The clinical care section consists mainly of three chronic care conditions: diabetes mellitus, chronic obstructive pulmonary disease (COPD) and cardiovascular disease (CVD).
In the same period, health insurers developed bundled payment schemes, with an initial focus on diabetes (approximate start in 2008), and in later years also on COPD (since 2009) and CVD (since 2011) ,. Bundled payment is suggested as a method to control costs while providing more patient centered, higher quality care . An international review of different types of bundled payment  suggests that there is weak but consistent evidence for a positive effect of bundled payment on healthcare costs, with no tangible impact on quality of care. Studies on the effect of bundled payment in the Netherlands show that it improved organization and management of care , and potentially also the outcomes of diabetes care . Nevertheless, a Cochrane review concluded that there is insufficient evidence to draw firm conclusions on the effectiveness of different these types of reimbursement schemes .
This study is focused on effects of the Dutch practice accreditation program in the context of a healthcare system, in which provider reimbursement of chronic illness care was improved simultaniously. In the Netherlands, approximately 80% of diabetes patients has the family physician (FP) as the principal caregiver. This percentage is slightly lower for COPD and CVD patients (50-65%). Generally, patients with the FP as main care provider tend to be stable, although not necessarily well controlled. The study aimed to determine whether participation in the accreditation program led to improvements in quality of care regarding diabetes, chronic obstructive pulmonary disease (COPD) and cardiovascular disease (CVD). We also aimed to assess whether the improved performance was different from the performance during pre-assessment of a second group of participants.
We performed a comparative observational study with two cohorts of primary care practices. A first cohort of practices was followed throughout the 3-year program with extended measurements at start and after three years. A second cohort was measured at their start of the accreditation program, during the same period of the post-assessment in the first cohort. Data collected at the start of the practice accreditation process portrayed practice performance prior to the accreditation process: data were collected retrospectively over a one year period prior to the start. We evaluated the effect of practice accreditation in a design with two separate comparisons. In the first comparison we assessed improvements over time within the first cohort. In the second comparison we assessed differences between the first and second cohort by comparing the follow-up measurement of the first cohort with the start measurement of the second cohort. Since both of these groups collected data in the same period, this comparison could offer additional information on effects of accreditation (3 years experience versus start of accreditation). The ethics committee of the Radboud university medical center provided a waiver for the study.
Practices included in the study were Dutch primary care practices that participated in the accreditation program of the Dutch College of General Practitioners between 2006 and 2011. We included a group of 69 practices in the first cohort that joined the accreditation program before 2009 and had collected follow-up data. Start measurements took place between 2006 and 2008; follow-up measurements of this cohort took place between 2009 and 2011. The average time period between data collection in the first cohort was four years and two months, with a minimal interval of 3 years and 2 months.
The second cohort consisted of primary care practices that had their start measurement between 2009 and 2011. We used a matched design to select an equally large sample of 69 practices (see Figure 1). Matching was based on availability of data, practice location (degree of urbanization), visitation date, practice type, and practice size.
Practices have been invited to participate voluntarily in the Dutch practice accreditation program since 2005. The preparatory phase of the program consisted of data collection regarding practice management and patient care. The measurement instruments used were previously validated questionnaires such as the `VIP', a visitation instrument for practice organization  and the `Europep'  for patient experiences with care. The clinical performance indicators were derived from the national guidelines valid at the time of measurement . These indicators were measured with the use of patient information that was extracted from electronic medical records; the family physician (FP) or nurse extracted the information either automatically or manually with a standardized extraction form. After data submission via an online system, the practice received a report that included information on their own performance and the performance of other practices as a benchmark. This information could be used to identify which areas needed improvement. The FPs wrote improvement plans with a plan-do-study-act cycle. The first audit was carried out after the approval of these plans to confirm adequate participation and to grant accreditation. After this audit a three-year accreditation cycle started. At the end of each year the practice staff evaluated whether the objectives of improvement programs were met and wrote new improvement plans for the following year. The prolongation of accreditation depended on this process; accreditation was not based on the actual quality of care itself, but rather on the quality of the improvement initiatives according to a structured program.
The primary care practice maintains comprehensive patient records, which includes information on all contacts and procedures in primary care, hospital, and several other care providers. For this study, we excluded all patients of which the FP was not the main care provider. All information used in this study was collected as part of the accreditation program. The prevalence of diabetes and COPD was defined as the number of patients with the condition divided by the total number of patients in the practice. Indicators of both process of care and intermediate outcomes were included regarding patients with diabetes, COPD and CVD (Table 1). The data that were collected before 2009 on CVD related to a broader range of patients; patients with high risk for CVD were also included. Since this made it impossible to compare data over time we only included data after 2009, that relate to patients with known CVD. During our research period several target levels as described in the guidelines shifted for diabetes (relating to indicators 2 and 3) and for CVD (indicator 2). We calculated the percentage of patients with a level below target level according to the target level that was valid at the time of measurement. Data on practice characteristics were collected through the use of online questionnaires which were also part of the practice accreditation program.
Descriptive statistics were used to describe the practice population and gain insight in the performance scores. For each of the three conditions we calculated a mean score for several process indicators. Some indicators could not be included in the mean scores due to low internal consistency with other indicators or use of subsamples: diabetes indicator 5, COPD indicator 3 and CVD indicators 4, 5 and 6 (Table 1).
In order to examine whether chronic care management improved after the three year cycle of the accreditation program, we compared the start and follow-up scores of the first cohort in a within-subject design. We performed separate multilevel regression analyses for each indicator. We accounted for repeated measures per practice and included predictors reflecting practice size, practice type and practice location.
Practices in the first cohort were compared to practices in the second cohort on follow-up and baseline scores respectively, in a separate regression analysis. The comparison was corrected for the match criteria practice size, practice type and practice location in each model. All analyses were performed with SPSS version 20 .
Table 2 shows the percentages and means of different practice characteristics in each group. There was a slight underrepresentation of practices that were part of a health center in the second cohort compared to the first cohort: 8 (12%) versus 11 (16%) practices. They were slightly more often located in an urban area (25 (36%) versus 23 (33%) practices). The mean number of patients in the practice increased in time; mean practice size was similar in the two cohorts.
The first two columns in Table 3 report on scores at start and follow-up of practices in the first cohort. The recorded prevalence of diabetes increased from 40 to 50 per 1000 patients. The recorded prevalence of COPD also increased, from 17 to 22 per 1000 patient. The prevalence of CVD/CVRM was not measured.
Table 3 also reports results from the multilevel regression analyses. Regarding diabetes we found that practices had improved on three performance indicators. Practice scores had higher percentages of patients with a recorded cholesterol measurement (p 0.04), feet examination (p 0.03) and prescription of lipid lowering medication (p 0.009). The percentage of patients with a blood pressure below target level was marginally lower at follow up (p 0.08). Practices in the first cohort had improved on three COPD process indicators. There was an increase in the percentage of patients with a recorded spirometry measurement (p 0.000), which was the sole contributor to the effect on the mean process score (p 0.01). We also found a large improvement in the percentage of patients who smoke that received an advice to stop smoking (p 0.002).
The figures for the between-groups comparison are reported in the columns on the right side of Table 3. No differences were found in the recorded prevalence of diabetes and COPD.
Diabetes patients in the second cohort less often had a total cholesterol value within target level compared to the patients in the first cohort (p 0.02). No other differences were found between the two cohorts regarding diabetes. We did not find any differences on COPD performance indicators. Practices in the first cohort provided higher quality of care than the second cohort on three CVD performance indicators. A higher percentage of patients had a blood pressure within target level (p 0.03). Furthermore, the number of patients with recorded smoke status (p 0.02) and glucose values (p 0.02) had increased.
Improvements were found regarding cholesterol measurement, feet examination and the prescription of lipid lowering medication (diabetes), spirometry performance and the provision of a stop smoking advice (COPD). As the follow-up measurement scores on these performance measures were similar to the baseline scores in the second cohort, it remains uncertain whether the effect can be attributed to the accreditation program. Compared to the second cohort of newly starting practices, the practices in the first cohort performed better regarding achievement of a target cholesterol level (diabetes) and blood pressure level (CVD) and the registration of smoke status and measurement of glucose levels (CVD). These differences could provide evidence for the added value of the accreditation program.
Explanation of findings
Our results are consistent with previous studies that show that there might be an effect of accreditation on quality of care processes and outcomes, although effects are not always consistent ,,. Furthermore, we mainly found improvements over time on those measures that had a lower score at baseline, which is consistent with findings from a systematic review on quality improvement strategies . Several scores, especially regarding diabetes, were already quite high, which might be a reflection of the general amount of attention towards diabetes care improvement, but it can also be related to the relatively large numbers of practices that provide vocational training in our study population, which is associated with better quality of care . In the Netherlands, approximately 30% of all practices have at least one family physician that provides training. In our population the percentage of practices with training was higher in both cohorts: 61% (42 of 69 practices) and 70% (48 of 69) in the first and second cohort respectively.
In the Dutch accreditation program, the practice is accredited if it is actively involved in quality improvement through use of improvement plans and can achieve improvements over time. An advantage of this approach is that the program is more attractive for health professionals; they take ownership of the improvement plans that are tailored to the individual practices. Like other accreditation programs, it uses audit and feedback as a central mechanism, embedded in a continuous quality improvement process that aims to enhance a culture that fosters quality and safety of healthcare ,. The fact that we did not find stronger evidence for an added value of the accreditation program may be partially explained by other developments in the primary care field during our research period. Starting in 2007, bundled payment was implemented broadly for diabetes and to a more limited extent for COPD; implementation of bundled payment for CVD just started at the end of our research period (2011). Although a large part of the practices may currently participate in bundled payment schemes , during our study period, most schemes were only just set up. Bundled payment offers audit and feedback and funds for support/education etcetera, but no other interventions to achieve quality improvement. The Dutch accreditation program is one example of an intervention that can complement further implementation through the development of improvement plans and continuous support. There are clear similarities between participation in bundled payment and the accreditation process as an audit and feedback system. Both require data collection and offer feedback; measures used on diabetes, COPD and CVD are almost identical. In fact, practices that participate in both initiatives often collect data only once and use the data for both purposes. This might explain why little added value of the accreditation program was detected for diabetes and COPD, two conditions for which many practices also participate in a bundled payment initiative. It could also account for the effect of accreditation on CVD, since implementation of bundled payment for this condition only just started during our research period.
Strenghts and limitations
Our study design implied a risk of confounding, because of the absence of random allocation. The second cohort was selected out of a possible 802 practices of all 4917 Dutch practices (16.3%) . At the end of our study period, more than half of all Dutch general practitioners (over 4600) worked in a practice that took part in the accreditation program and more than 40% of all practices were accredited. It is likely that a substantial part of these practices also participated in bundled payment for diabetes and COPD; the lack of exact information on this limited this study. In our models, we accounted for practice type, size and location. However, there may have been an underrepresentation of practices with a less than average interest in quality improvement, especially in the first cohort, since participation in the program was voluntary. Both the argument of confounding and of the representation of the study population reflect on the possibility that the effects could be more present in the whole population of general practices.
The increase in diabetes and COPD prevalence between 2006 and 2011 may indicate that practices have improved their registration. On the other hand it can also be a result of efforts to screen the population for undiagnosed patients. Process indicator scores might decrease when registration is better, since there is an association between performing a measurement and registering it. This can cause a bias, which may differ when registration improves. Outcome scores might increase after active screening of the population, due to the inclusion of patients with a mild form of diabetes or COPD.
General practices improved the quality of provided healthcare for patients with the targeted chronic diseases. Nevertheless, only a few improvements could be attributed to the accreditation program, which may be caused by the fact that other programs address quality of chronic illness care at the same time. Expectations of the effects of accreditation were high among participants and stakeholders, but the results of this evaluation do not support these fully. Continuous monitoring may have been beneficial for practices in order to maintain a high level of quality in some areas and improve further in others. After the study was completed, adaptations to the program were made, which reduced the burden of work and may improve the effectiveness. For instance, these could stimulate practices to not focus solely on the measured items .
AD performed data-cleaning and data-analyses and drafted the manuscript. JB contributed to the design of the study and revised the manuscript. RW contributed to the discussion and revised the manuscript. MB contributed to the discussion and provided logistical support in the provision of data. MW revised the manuscript and contributed to the discussion. All authors read and approved the final manuscript.
Chronic obstructive pulmonary disease
Dutch abbreviation for Dutch College of General Practitioners
Coleman K, Austin BT, Brach C, Wagner EH: Evidence on the Chronic Care Model in the new millennium. Health Aff. 2009, 28: 75-85. 10.1377/hlthaff.28.1.75.
Shaw C: The external assessment of health services. World Hosp Health Serv. 2004, 40: 24-27. 50, 51
Jaafaripooyan E, Agrizzi D, Akbari-Haghighi F: Healthcare accreditation systems: further perspectives on performance measures. Int J Qual Health Care. 2011, 23: 645-656. 10.1093/intqhc/mzr063.
Alkhenizan A, Shaw C: Impact of accreditation on the quality of healthcare services: a systematic review of the literature. Ann Saudi Med. 2011, 31: 407-416. 10.4103/0256-4947.83204.
Shaw C, Groene O, Mora N, Sunol R: Accreditation and ISO certification: do they explain differences in quality management in European hospitals?. Int J Qual Health Care. 2010, 22: 445-451. 10.1093/intqhc/mzq054.
Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O'Brien MA, Johansen M, Grimshaw J, Oxman AD: Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012, 6: CD000259-
Greenfield D, Braithwaite J: Health sector accreditation research: a systematic review. Int J Qual Health Care. 2008, 20: 172-183. 10.1093/intqhc/mzn005.
Greenfield D, Braithwaite J: Developing the evidence base for accreditation of healthcare organisations: a call for transparency and innovation. Qual Saf Health Care. 2009, 18: 162-163. 10.1136/qshc.2009.032359.
Hinchcliff R, Greenfield D, Moldovan M, Westbrook JI, Pawsey M, Mumford V, Braithwaite J: Narrative synthesis of health service accreditation literature. BMJ Qual Saf. 2012, 21: 979-991. 10.1136/bmjqs-2012-000852.
O'Beirne M, Zwicker K, Sterling PD, Lait J, Lee Robertson H, Oelke ND: The status of accreditation in primary care. Qual Prim Care. 2013, 21: 23-31.
Szecsenyi J, Campbell S, Broge B, Laux G, Willms S, Wensing M, Goetz K: Effectiveness of a quality-improvement program in improving management of primary care practices. CMAJ. 2011, 183: E1326-E1333. 10.1503/cmaj.110412.
NHG Practice accreditation: a quality label for ambition [in Dutch]. Bijblijven. 2010, 26: 7-12.
Dutch practice accreditation website [in Dutch]. , [http://www.praktijkaccreditering.nl/]
de Wildt JE, Leusink GL: Baseline Measurement Bundled Payment [in Dutch]. 2008, LVG, Utrecht, the Netherlands
Transparant Colaborative Care, Diabetes Mellitus, COPD and CVD, Report on Bundled Payment 2012 [in Dutch]. 2012, LHV/VHN/LVG, Utrecht
Porter ME, Pabo EA, Lee TH: Redesigning primary care: a strategic vision to improve value by organizing around patients' needs. Health Aff. 2013, 32: 516-525. 10.1377/hlthaff.2012.0961.
Hussey PS, Mulcahy AW, Schnyer C, Schneider EC: Closing the Quality Gap: Revisiting the State of Science. Vol. 1.: Bundled Payment: Effects on Health Care Spending and Quality. 2012, Agency for Healthcare Research and Quality (US), Rockville (MD)
de Bakker DH, Struijs JN, Baan CA, Raams J, de Wildt JE, Vrijhoef HJ, Schut FT: Early results from adoption of bundled payment for diabetes care in The Netherlands show improvement in care coordination. Health Aff. 2012, 31: 426-433. 10.1377/hlthaff.2011.0912.
Struijs JN, Baan CA: Integrating care through bundled payments - lessons from The Netherlands. N Engl J Med. 2011, 364: 990-991. 10.1056/NEJMp1011849.
Struijs JN, Til JT, Baan CA: Experimenting with a bundled payment system for diabetes care in the Netherlands: first tangible effects. 2010, National Institute for Public Health and the Environment, Bilthoven
Scott A, Sivey P, Ait Ouakrim D, Willenberg L, Naccarella L, Furler J, Young D: The effect of financial incentives on the quality of health care provided by primary care physicians.Cochrane Database Syst Rev 2011,:. doi:10.1002/14651858.CD008451.pub2.,
van den Hombergh P, Grol R, van den Hoogen HJ, van den Bosch WJ: Assessment of management in general practice: validation of a practice visit method. Br J Gen Pract. 1998, 48: 1743-1750.
Grol R, Wensing M, Mainz J, Jung HP, Ferreira P, Hearnshaw H, Hjortdahl P, Olesen F, Reis S, Ribacke M, Szecsenyi J: Patients in Europe evaluate general practice care: an international comparison. Br J Gen Pract. 2000, 50: 882-887.
Braspenning J, Dijkstra R, Tacken M, Bouma M, Witmer H: Visitation Instrument Accreditation (VIA) [in Dutch]. 2007, Kwaliteit van Zorg (WOK) UMC St Radboud, Nederlands Huisartsen Genootschap (NHG), NHG Praktijk Accreditering BV, Nijmegen/Utrecht
Corp IBM: IBM SPSS Statistics for Windows. Version 20.0. 2011, IBM Corp, Armonk
Tricco AC, Ivers NM, Grimshaw JM, Moher D, Turner L, Galipeau J, Halperin I, Vachon B, Ramsay T, Manns B, Tonelli M, Shojania K: Effectiveness of quality improvement strategies on the management of diabetes: a systematic review and meta-analysis. Lancet. 2012, 379: 2252-2261. 10.1016/S0140-6736(12)60480-2.
van den Hombergh P, Schalk-Soekar S, Kramer A, Bottema B, Campbell S, Braspenning J: Are family practice trainers and their host practices any better? Comparing practice trainers and non-trainers and their practices. BMC Fam Pract. 2013, 14: 23-10.1186/1471-2296-14-23.
Greenfield D, Pawsey M, Braithwaite J: What motivates professionals to engage in the accreditation of healthcare organizations?. Int J Qual Health Care. 2011, 23: 8-14. 10.1093/intqhc/mzq069.
Hassel DTP, Kenens RJ: Data from Registration of General Practitioners: Measurements 2012 [in Dutch]. 2013, NIVEL, Utrecht
Doran T, Kontopantelis E, Valderas JM, Campbell S, Roland M, Salisbury C, Reeves D: Effect of financial incentives on incentivised and non-incentivised clinical activities: longitudinal analysis of data from the UK Quality and Outcomes Framework. BMJ. 2011, 342: d3590-10.1136/bmj.d3590.
This work was financially supported by the Dutch College of General Practitioners (NHG) in the Netherlands. The Dutch College of General Practitioners developed the Dutch Accreditation Program; an independent organization (NPA: NHG practice accreditation, www.praktijkaccreditering.nl) is responsible for the implementation of the program and for the data collection. We acknowledge the NPA for their support in the provision of data for this study. No other funding was provided.
One of the authors (MB) is an employee of the company that funded this work and set up the Dutch Accreditation Program.