Efficacy of a strategy for implementing a guideline for the control of cardiovascular risk in a primary healthcare setting: the SIRVA2 study a controlled, blinded community intervention trial randomised by clusters
© Rodríguez-Salvanés et al; licensee BioMed Central Ltd. 2011
Received: 16 February 2011
Accepted: 19 April 2011
Published: 19 April 2011
This work describes the methodology used to assess a strategy for implementing clinical practice guidelines (CPG) for cardiovascular risk control in a health area of Madrid.
The results on clinical practice of introducing CPGs have been little studied in Spain. The strategy used to implement a CPG is known to influence its final use. Strategies based on the involvement of opinion leaders and that are easily executed appear to be among the most successful.
The main aim of the present work was to compare the effectiveness of two strategies for implementing a CPG designed to reduce cardiovascular risk in the primary healthcare setting, measured in terms of improvements in the recording of calculated cardiovascular risk or specific risk factors in patients' medical records, the control of cardiovascular risk factors, and the incidence of cardiovascular events.
This study involved a controlled, blinded community intervention in which the 21 health centres of the Number 2 Health Area of Madrid were randomly assigned by clusters to be involved in either a proposed CPG implementation strategy to reduce cardiovascular risk, or the normal dissemination strategy. The study subjects were patients ≥ 45 years of age whose health cards showed them to belong to the studied health area. The main variable examined was the proportion of patients whose medical histories included the calculation of their cardiovascular risk or that explicitly mentioned the presence of variables necessary for its calculation. The sample size was calculated for a comparison of proportions with alpha = 0.05 and beta = 0.20, and assuming that the intervention would lead to a 15% increase in the measured variables. Corrections were made for the design effect, assigning a sample size to each cluster proportional to the size of the population served by the corresponding health centre, and assuming losses of 20%. This demanded a final sample size of 620 patients. Data were analysed using summary measures for each cluster, both in making estimates and for hypothesis testing. Analysis of the variables was made on an intention-to-treat basis.
KeywordsPrimary healthcare Randomised clinical trial Cluster analysis Clinical practice guidelines Cardiovascular disease
Clinical practice guidelines (CPGs) can be defined as a series of systematically developed recommendations designed to help health professionals and patients take decisions regarding the most appropriate forms of treatment in specific clinical situations . They are used with the aim of preventing variability in clinical practice via the presentation of the best scientific evidence . This, however, should not be their only aim ; they should also be used to solve organisational problems, adjusting their recommendations to the preferences and values of patients, the availability of resources, and the concerns of managers . A great many CPGs have been produced in recent years. In 1992 the National Library of Medicine of the USA published an exhaustive monograph recounting the appearance of 533 between the years of 1985 and 1992 , yet this can only be an approximation to their true number. In Spain, the Guía Salud Project collected and reviewed 425 CPGs in February 2010 .
The evaluation of the use of CPGs in terms of health results is not very extended in Spain. With the exception of the Programa de Actividades Preventivas y de Promoción de la Salud (PAPPS; Program of Preventive Activities and Health Promotion) , no studies on the effectiveness of CPGs at the primary healthcare level appear to exist. A systematic review published in the Lancet in 1993  reported that, of 59 reviewed CPGs, none of which were used in the primary healthcare setting, 55 had improved the health of patients following their implantation. This improvement appeared to be related to the implementation strategy used. Studies that have assessed the impact of CPGs in the primary healthcare setting, however, show them not to have been of benefit to patients. A review focused on primary healthcare that investigated 91 studies found only five interventions to have been of any benefit to patients . In fact, only 13 actually met the inclusion criteria of the study. The authors of this review warn of the poor quality of the CPGs in question and their scant implementation.
According to a review of studies on the implementation of CPGs, only 40% of health professionals actually read them, and while 78% profess to use them the real figure could be as low as 5% .
It can be inferred that the use a CPG is related to a number of concatenated factors: it would appear important that health professionals relate to the CPG in question , that local adaptations have been made involving people close to the potential end users, and that guidelines are adjusted to the peculiarities of the setting in which they are to be used. These factors facilitate the implementation of CPGs, improve their use, and increase their final impact . However, one of the most important factors affecting the use of CPGs is the implementation strategy employed . No evidence is available, however, on what night be the best implementation strategy. The classic review by Fremantle  reports that the implementation of CPGs based on educational methods and using computer-based reminders can improve use by some 15%. Certainly, however, no studies are available on what might be the best way to implement a CPG at the primary healthcare level in Spain.
All bodies that propose recommendations for the production of CPGs indicate that implementation strategies should be included in the document's planning. Failure to do this is unlikely to lead to any changes in medical practice. The diffusion methods that have given the best results have been those designed to bring about a positive predisposition in the end user before the CPG is presented, e.g., via the release of part of the information so that end users might consider the need for change, and offering them alternatives in the use of the CPG, assuring them that it contains no strict rules to follow but recommendations that the professional should use adequately with each individual patient .
None of the above studies, however, affords conclusive results. In this light, the present work describes the aims and design of a project undertaken to determine the effectiveness of a CPG implementation strategy for the control of cardiovascular risk. A CPG for the detection and treatment of this problem was chosen given the important public health burden of heart disease, and because, despite the potential benefits of using CPGs in this area, their true use by primary care practitioners has room for improvement .
The main aim of this work was to compare the effectiveness of 1) a proposed strategy based on an educational method involving the participation of opinion leaders with 2) the usual method of dissemination, for implementing a locally adapted CPG for the control of cardiovascular risk among the primary healthcare teams of a health area in Madrid.
This study (ClinicalTrials.gov Identifier number: NCT01270022) was a controlled, blinded community intervention study, randomised by clusters (primary healthcare centres). This design was chosen since the intervention necessarily involved primary healthcare teams. Twenty one health centres of the Madrid A2 Primary Healthcare Area were randomly assigned to the normal dissemination or proposed implementation strategy arms of the study (10 to the former and 11 to the latter). These 21 health centres (clusters) provide care to 373,495 people. The two strategies were aimed at the healthcare teams (doctors and nurses) of each participating centre. The effect of the two strategies on 1) the recording of cardiovascular risk data in the clinical histories of the patients, 2) the control of cardiovascular risk factors, and 3) the incidence of cardiovascular events, was determined.
The CPG used in this study was a local adaptation of the different cardiovascular risk guidelines established by healthcare professionals working in both the primary and hospital settings. The dissemination of the CPG to the normal dissemination group was via the method traditionally used in the health area, i.e., sending it by mail to all healthcare professionals and the provision of an information workshop lasting 2 h. The strategy used with the implementation group involved an educational model based on the involvement of opinion leaders plus the method used with the normal dissemination group. The implementation strategy, which was aimed at the doctors and nurses employed at the health centres, involved 4 × 1 h education sessions. These were masked within the normal staff meeting held at each centre to avoid the undertaking of the present study becoming known. The sessions were led by trained personnel recognised as leaders of opinion by the healthcare professionals at the centres - a normal practice in these session. The first session was delivered by a doctor and a nurse, members of the team that coordinated the authoring of the CPG. During this session the methodology used to produce the CPG was explained, underlining the importance of stratifying the cardiovascular risk of patients attending primary healthcare clinics. In the second session, a nurse from the health area (one of the authors of the CPG) discussed with the health centre staff the diagnosis, classification and treatment of low risk patients, screening techniques for different cardiovascular risks, and methods for evaluating these risks. In the third session, a physician author of the CPG with prestige amongst his/her peers discussed the pharmacological recommendations associated with cardiovascular risk factors, and the diagnosis, treatment and classification of high risk patients. In the final session, a physician author (an internal medicine specialist) from the Hospital Universitario de la Princesa, discussed referral criteria and the coordination proposals included in the CPG.
The study was divided into three stages. 1) Preliminary work. In July 2004, a cross-sectional study was undertaken to determine the status of the cardiovascular information in the selected patients' records. This involved the collection of the data required to determine the different risk factors recorded, the degree of control of cardiovascular risk factors, and the collection of other information required for the present study. 2) The intervention. This intervention took place between February and April 2005. One year later (April 2006) a second cross-sectional study was undertaken to determine the effect of the two strategies on the calculation of cardiovascular risk, and the control of these risk factors. 3) Follow-up. This phase involved a third cross-sectional study to determine the effect of the intervention on health results (cardiovascular events and the control of cardiovascular risk factors) at five years.
To ensure blindness in the assessment of the effectiveness of the intervention (evaluation of the above variables in clinical records), the evaluators were not told to which arm of the study the patients belonged. The team members whose job it was to analyse the results did not know this information either.
The impact of the intervention was measured in terms of registry variables (cardiovascular risk data in the clinical histories of the patients), intermediate variables (the control of cardiovascular risk factors at one and five years) and health results (the incidence of cardiovascular events at five years). At one year, the effect of the two strategies was determined by calculating the difference in the proportions of overall cardiovascular risk calculations before and after the intervention, or the explicit recording of all the variables needed for such calculations to be made (blood pressure, fasting glycaemia, fasting cholesterol, use of tobacco, age and sex). Blood pressure was deemed well controlled when the systolic pressure was ≤ 140 mmHg and the systolic blood pressure ≤ 90 mmHg; this was reduced to 130/80 mmHg if the patient was diabetic. Good control of glycaemia was deemed present when fasting blood sugar was < 126 mg/dl. The control of dyslipidaemia was considered good when total cholesterol was < 200 mg/dl, or LDL-cholesterol was < 100 mg/dl in diabetics.
At five years, the differences in health variables between the arms of the study, i.e., percentage of cardiovascular events (acute myocardial infarction, angina, stroke and the need for revascularization), the cardiovascular mortality and overall mortality, were determined. Since each of the health centres had different characteristics, data on a series of potential confounding factors (characteristics of the health centre and their staff) were recorded.
The unit of randomisation taken into account was not each patient but each health centre. The number of centres included in the study was 21 - all those in the studied health area. The sample of patients finally selected at any centre was proportional to the size of the population ascribed to that centre. The required sample size was determined by assuming that the implantation of the CPG could be regarded as successful if a 15% increase in the recording of the study variables were seen in the proposed implementation strategy arm, i.e., increasing from 40% to 55%. To show this effect, the sample size needed to reject the null hypothesis could have been calculated in the normal way  for the comparison of two proportions. For an alpha of 0.05 and beta of 0.20, the necessary sample size would have been n = 173*2 = 346 patients. However, designs involving randomisation by clusters with a fixed number of clusters requires a larger sample in order to take into account the design effect (DE). According to Campbell , DE = 1 + (m - 1) · r, where m is the mean size of the cluster and r the intracluster correlation coefficient (ICC). Since 346 patients would be needed if sampling were not via clusters, and since the study involved 21 health centres, the mean sample size per cluster (m) (if clusters are used) becomes 16.5 (346 patients across 21 centres).
Given that reported by Adams et al. , an ICC of 0.032 was assumed for the majority of variables examined. The DE therefore became: DE = 1 + (m-1) · r = 1.5. The corrected sample size was therefore: n´= n · ED = 346 · 1.5 = 517. Assuming a loss rate of 20%, the final sample size would be 620 patients randomly assigned to the two arms of the study (10 clusters in the normal dissemination group, and 11 in the proposed implementation strategy group). This sample size was adjusted for the fact that, in the studied health area, some 47% of the population is over 45 years of age. The total number of medical histories to examine was therefore at least 1170 for both arms together. Figure 1 shows the number of patients finally studied and their distribution.
The use of a design involving randomisation by clusters conditions the statistical methods that can be used. Although some authors  propose performing individual level analysis when the number of clusters is >20, there is no consensus , and since the present number was 21 a more conservative approach (cluster level analysis) was followed. The overall impact of the intervention was determined by comparing the means of the variables measured in each cluster using the t-test. Although the assumptions of homoscedasticity and normal distribution may not be satisfied, this is not so important since the t-test is robust to such departures . Since the cluster sizes were not similar, the t-test was weighted for each cluster j by wj = nj1·nj2/nj1+nj2 (where 1 and 2 are moments before and after the intervention respectively) . The calculation of the confidence intervals also required weighting in the same way. Analysis was on an intention-to-treat basis. All p values reported were two-tailed. STATA v.8.0 software was used for all calculations.
Cardiovascular risk was chosen as an indicator of change for three reasons: its measurement is essential in the reduction of cardiovascular events, its measurement depends completely on good practice on the part of medical professionals, and it is not influenced by patient attitude or any medical professional/patient or patient/healthy system relationship. The present strategy is relatively simple and, if shown to provide reliable results, could be used in other settings in which the use of more complex designs (see for example 25) may not be viable.
A number of reasons dictated that randomisation by clusters be used rather than simple randomisation, despite the lower statistical power of this method . Firstly, since all the healthcare staff of each centre were involved, it avoids the contamination that might occur if simple randomisation were followed. Secondly, although the intervention was aimed at the staff, the assessment took the form of an audit of the patients' clinical records. Finally, since randomisation was by clusters, analysis by clusters was therefore necessary. In recent years, the method employed in this work has become standard in the assessment of the effects of health interventions (implementation research) .
The described trial was approved by the Comisión de Investigación del Área 2. Consent to be included was obtained from all the health centres involved. Medical histories were consulted only by the research team. At all times confidentiality was preserved; no patients or medical professionals were identified.
List of abbreviations used
Clinical Practice Guidelines
Intracluster Correlation Coefficient
The CPG was adapted by the SIRVA2 group* and a large group of medical professionals from the Área 2 health area of Madrid. We would like to acknowledge supporting obtained from Investigation Area of Agencia Lain Entralgo, Comunidad de Madrid for the traslation of original draft.
* We also want to thank to the SIRVA2 group members: A. ALBARRAN PINAR, E. BARTOLOMÉ BENITO, L. BARUTEL RUBIO, R. CARRARO CASIERI, JM. CASTILLO, M. CID ARTIGAS, A. CUBILLO SENRA, A. FERRÁNDEZ INFANTE, MJ. FERNÁNDEZ LUQUE, A. GALLEGO ARENAS, L. GARCIA OLMOS, I. GARCIA POLO, G. GIL JUBERÍAS, A. GONZALEZ GAMARRA, P. LOECHES BELINCHÓN, F. LÓPEZ CORRAL, C. LÓPEZ GONZALEZ, J. LÓPEZ GONZÁLEZ, MA. MARTINEZ ALVAREZ, H. MARTINEZ SANZ, MS. MAYAYO VICENTE, F. MAYORDOMO SANCHEZ DE LA MORENA, B. NOVELLA ARRIBAS, A. PAREJO LLENERO, S. PAYAN AGUDO, P. PÉREZ CAYUELA, C. REFOLIO ABELLA, F. RODRÍGUEZ SALVANÉS, S. RUIZ ALONSO, L. RUIZ DIAZ, M. RUIZ LÓPEZ, RM. SÁNCHEZ ALCALDE, LM. SÁNCHEZ-GÓMEZ, B. SIERRA GARCÍA and C. SUAREZ FERNÁNDEZ.
Funding for the trial was provided by the Plan Nacional de Investigación Científica, Desarrollo e Innovación Tecnológica (I+D+I). Instituto de Salud Carlos III - Fondo de Investigación Sanitaria. Expediente N° 031216.
- Committee to Advise the Public Health Service on Clinical Practice Guidelines, Institute of Medicine. Clinical Practice guidelines: directions for a new program. Edited by: En Field MJ, Lohr KN. 1990, Washington, DC; National Academy Press
- Chassin MR, Brook RH, Park RE, Keesey J, Fink A, Kosekoff J, et al: Variations in the use of medical and surgical services by the Medicare population. N Engl J Med. 1986, 314: 285-290. 10.1056/NEJM198601303140505.View ArticlePubMedGoogle Scholar
- Cook DJ, Greengold NL, Ellrodt AG, Weingarten SR: The Relation between Systematic Reviews and Practice Guidelines. Ann Intern Med. 1997, 127: 210-216.View ArticlePubMedGoogle Scholar
- Battista R, Hodge M, Vineis P: Medicine, practice and guidelines: the uneasy juncture of science and art. J Clin Epidemiol. 1995, 48: 875-880. 10.1016/0895-4356(94)00199-Z.View ArticlePubMedGoogle Scholar
- Scannell KM, Miller N, Glock M: Current Bibliographies in Medicine. Practice Guidelines. 1992, Bethesda, Maryland: National Library of Medicine, 92-14.Google Scholar
- Accessed January 2010, [http://portal.guiasalud.es/web/guest/catalogo-gpc]
- Subías PJ, Bouzá K, Casanovas E, García JA, Jiménez J, et al: Estudio de efectividad PAPPS (1998) y resultados preliminares de la evaluación PAPPS (1999). Aten Primaria. 1999, 24 (1 Suppl): 11-17.Google Scholar
- Grimshaw JM, Russel IT: Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet. 1993, 342: 1317-1322. 10.1016/0140-6736(93)92244-N.View ArticlePubMedGoogle Scholar
- Worral G, Chaulk P, Freaake D: The effects of clinical practice guidelines on patient outcomes in primary care: a systematic review. CMAJ. 1997, 156 (12): 1705-1712.Google Scholar
- Davis DA, Taylor-Vaisey A: Translating guidelines into practice. A systematic review of theoretic concepts, practical experience and research evidence in the adoption of clinical guidelines. CMAJ. 1997, 157 (4): 408-416.PubMedPubMed CentralGoogle Scholar
- Langley C, Faulkner A, Watkins C, Gray S, Harvey I: Use of guidelines in primary care practitioners, perspectives. Fam Pract. 1998, 15: 105-111. 10.1093/fampra/15.2.105.View ArticlePubMedGoogle Scholar
- Grimshaw J, Russell IT: Achieving health gain though clinical guidelines: Developing scientifically valid guidelines. Qual Health Care. 1993, 2: 243-248. 10.1136/qshc.2.4.243.View ArticlePubMedPubMed CentralGoogle Scholar
- Freemantle N, Harvey E, Wolf F, Oxman A, Grilli R, Bero L: The effectiveness of printed educational materials in changing the behavior of healthcare professionals. In the Cochrane Database of Systematic Reviews. 1997, OxfordGoogle Scholar
- Freemantle N: Implementation strategies. Fam Pract. 2000, 17: S7-S10. 10.1093/fampra/17.suppl_1.S7.View ArticlePubMedGoogle Scholar
- Cohen DL, Littenberg B, Wetzel C, Neuhauser DB: Improving physician compliance with preventive medicine guidelines. Med Care. 1992, 20: 1040-1045.View ArticleGoogle Scholar
- Doroodchi H, Abdolrasulnia M, Foster JA, Foster E, Turakhia MP, Skelding KA, Sagar K, Casebeer LL: Knowledge and attitudes of primary care physicians in the management of patients at risk for cardiovascular events. BMC Fam Pract. 2008, 9: 42-10.1186/1471-2296-9-42.View ArticlePubMedPubMed CentralGoogle Scholar
- Galan Maiques A, Brotons Cuixart C, Villar Álvarez F, Lobos Bejaranos JM, Torçal Laguna J, Orozco Beltran D, Navarro Pérez J, Ortega Sánchez-Pinilla R, Gil Guillen V, Vilaseca Canals J: Recomendaciones Preventivas cardiovasculares PAPPS 2009. Actualización PaPPS. [http://www.papps.org/upload/file/09%20PAPPS%20ACTUALIZACION%202009.pdf]
- Armitage P, Berry G: Statistical Methods in Medical Research. 1987, Blackwell Scientific Publications, 179-185.Google Scholar
- Cambpell MJ: Cluster randomized trials in general (family) practice research. Stat Methods in Med Res. 2000, 9: 81-94. 10.1191/096228000676246354.View ArticleGoogle Scholar
- Adams G, Gulliford M, Ukoumunne O, Eldridge S, Chinn S, Campbell M: Patterns in intra-cluster correlation from primary care research to inform study design and analysis. J Clin Epidemiol. 2004, 57: 785-794. 10.1016/j.jclinepi.2003.12.013.View ArticlePubMedGoogle Scholar
- Okoumunne O, Thompson S: Analysis of cluster randomized trials with repeated cross-sectional binary measurements. Stat Med. 2001, 20: 417-433. 10.1002/1097-0258(20010215)20:3<417::AID-SIM802>3.0.CO;2-G.View ArticleGoogle Scholar
- Hayes R, Moulton L: Cluster Randomised Trials. 2009, Chapman & Hall EdsView ArticleGoogle Scholar
- Donner A, Klar N: Desing and Analysis of Cluster Randomized Trials. 2000, Arnold: LondonGoogle Scholar
- Donner A, Donald A: Analysis of data arising from stratified design with the cluster as unit of randomization. Stat Med. 1987, 6: 43-52. 10.1002/sim.4780060106.View ArticlePubMedGoogle Scholar
- Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L, Shirran L, Wensing M, Dijkstra R, Donaldson C: Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004, 8: 6-View ArticleGoogle Scholar
- Bland JM, Kerry SM: Trials randomized in clusters. BMJ. 1997, 315: 600-View ArticlePubMedPubMed CentralGoogle Scholar
- Campbell M, Mollison J, Grimshaw J: Cluster trials in implementation research: estimation of intracluster correlation coefficients and sample size. Stat Med. 2001, 20: 391-399. 10.1002/1097-0258(20010215)20:3<391::AID-SIM800>3.0.CO;2-Z.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2296/12/21/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.