Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Improving preventive service delivery at adult complete health check-ups: the Preventive health Evidence-based Recommendation Form (PERFORM) cluster randomized controlled trial

  • Vinita Dubey1Email author,
  • Roy Mathew2,
  • Karl Iglar2,
  • Rahim Moineddin3 and
  • Richard Glazier3
BMC Family Practice20067:44

https://doi.org/10.1186/1471-2296-7-44

Received: 19 July 2005

Accepted: 12 July 2006

Published: 12 July 2006

Abstract

Background

To determine the effectiveness of a single checklist reminder form to improve the delivery of preventive health services at adult health check-ups in a family practice setting.

Methods

A prospective cluster randomized controlled trial was conducted at four urban family practice clinics among 38 primary care physicians affiliated with the University of Toronto. Preventive Care Checklist Forms© were created to be used by family physicians at adult health check-ups over a five-month period. The sex-specific forms incorporate evidence-based recommendations on preventive health services and documentation space for routine procedures such as physical examination. The forms were used in two intervention clinics and two control clinics. Rates and relative risks (RR) of the performance of 13 preventive health maneuvers at baseline and post-intervention and the percentage of up-to-date preventive health services delivered per patient were compared between the two groups.

Results

Randomly-selected charts were reviewed at baseline (n = 509) and post-intervention (n = 608). Baseline rates for provision of preventive health services ranged from 3% (fecal occult blood testing) to 93% (blood pressure measurement), similar to other settings. The percentage of up-to-date preventive health services delivered per patient at the end of the intervention was 48.9% in the control group and 71.7% in the intervention group. This is an overall 22.8% absolute increase (p = 0.0001), and 46.6% relative increase in the delivery of preventive health services per patient in the intervention group compared to controls. Eight of thirteen preventive health services showed a statistically significant change (p < 0.05) in favor of the intervention (adjusted RR (95% C.I.)): counseling on brushing/flossing teeth (9.2 (4.3–19.6)), folic acid counseling (7.5 (2.7–20.8)), fecal occult blood testing (6.7 (1.9–24.1)), smoking cessation counseling (3.9 (2.2–7.2)), tetanus immunization (3.0 (1.7–5.2)), history of alcohol intake (1.33 (1.2–1.5)), history of smoking habits (1.28 (1.2–1.4)) and blood pressure measurement (1.05 (1.00–1.10)).

Conclusion

This simple, low cost, clinically relevant intervention improves the delivery of preventive health services by prompting physicians of evidence-based recommendations in a checklist format that incorporates existing practice patterns. Periodic updates of the Preventive Care Checklist Forms© will allow a feasible and easy-to-use tool for primary care physicians to provide evidence-based preventive health services to adults at routine health check-ups. The forms can also be incorporated into an electronic health record. The Preventive Care Checklist Forms© are accessible in English and French at the College of Family Physicians of Canada web site.

Background

Integration of preventive care guidelines into clinical practice has been modest across all medical specialties and countries [113]. Reasons cited for why physicians do not follow clinical practice guidelines include a lack of awareness of guidelines, lack of familiarity to apply guidelines correctly, lack of agreement with the guidelines, lack of a believe that physician behaviours or outcomes can be changed and external barriers such as a lack of time or a reminder system [1416].

The Canadian Task Force on Preventive Health Care grades preventive health recommendations into five categories from A to E based on the strength of evidence for a specific preventive health maneuver, similar to the US Preventive Services Task Force and other such organizations [17, 18]. Grade A recommendations have good evidence to include in adult preventive service delivery, usually from randomized controlled trials, and B recommendations have fair evidence to include. Examples of grade A recommendations include colorectal cancer screening with a fecal occult blood testing, prevention of neural tube defects through folic acid supplementation, and providing smoking cessation counseling; and grade B recommendations include cervical cancer screening with a Papanicolaou smear and ensuring rubella immunity. The Task Force recommendations are most applicable to primary care physicians who conduct health examinations on asymptomatic individuals and have the greatest opportunity for prevention.

Primary care physicians self-report excellent delivery of preventive services but in reality rates are moderate to low [1923]. In one study of 4, 049 patient visits in 138 family physician offices in Ohio, for three groupings of preventive service delivery, patients were up to date on only 55% of screening, 24% of immunization and 9% of health habit counseling services [24]. Cancer screening has shown similar poor outcomes in primary care, despite the evidence that cancer screening can reduce morbidity and mortality when provided to populations of patients [17, 18, 2527]. Even at adult health check-ups, a time typically devoted to providing preventive health care, rates of preventive service delivery are low [24].

Barriers to the delivery of preventive health services include lack of time, physician forgetfulness, patient refusal, practice logistical difficulties, urgent concerns dominating visits, inadequate reimbursement and patient concerns regarding the intervention [2830]. It is estimated that 7.4 hours per working day are required just for the provision of preventive services by a primary care physician [31]. Tools to improve prevention must overcome such barriers [32, 33]. Otherwise, as Stange et al argue, it may be unrealistic to expect primary care physicians to deliver a comprehensive package of clinical preventive services among the many competing demands of providing primary care [24]. However, if primary care physicians are unequipped to deliver preventive health services, then how can patients and populations hope to receive these services? With an aging population the ability of a health care system to focus on prevention is even more essential to reduce preventive morbidity and mortality.

Continuing medical education and continuing professional development have been the traditional modalities for implementing changes in clinical practice, but their effectiveness has been disappointing. Knowledge translation is a new paradigm for putting knowledge into practice by incorporating tools to overcome barriers to change [3436]. Prompts, reminders and patient-mediated methods are examples of such tools.

There is no one proven maneuver to improve preventive care [5, 37, 38]. Multi-faceted interventions especially with organizational change are valuable, yet they are expensive, logistically complex to implement and are not consistently more effective than single interventions. Other successful interventions are not easily incorporated into routine practices as they require a nurse facilitator, complex computer systems, office-system changes and "teamwork and tenacity" [39, 40]. More research has been advocated to determine methods to disseminate prevention guidelines into practice.

Prompting physicians can lead to a significant improvement in health maintenance, such as checklists attached to the patient chart, tagged notes, computer generated encounter forms, prompting stickers and patient carried prompting cards [32]. Studies on physician reminders are dated, though they did consistently show improved rates of preventive activities between 6% and 24% compared to a control group [10, 38]. Computer generated reminder systems have been popular, but are expensive and require advanced computerized office systems, yet only 5 to 13% of primary care physicians have electronic health record systems.

Are simple reminder tools, such as a checklist form, still an effective way to assist providers to adopt preventive health care guidelines into daily practice? A few studies done in the 1980's consistently showed that checklists were highly adopted, continued to be used months after the commencement of an intervention, and were effective to increase rates of preventive service delivery [4145]. Recently, Litaker et al have urged that future interventions should address visit- and practice-specific factors that are more closely associated with preventive care [46]. In this study, we provide primary care physicians with a evidence-based, sex-specific Preventive Care Checklist Form©, to be used at adult health check-ups, to determine if this visit- and practice-specific tool will be effective to improve the delivery of preventive health services.

Methods

Setting

The trial was conducted at four academic family medicine clinics affiliated with St. Michael's Hospital and the University of Toronto. Over 40 family physicians and 20 residents see approximately 100 000 patient visits per year. The four clinics are part of the same hospital department that share clinical guidelines and practices, educational rounds, staffing levels, hours of operation, after-hours care, and teaching duties. Each clinic has diverse patient populations including traditional family practice and marginalized groups including those with low-income, disability, HIV/AIDS, and severe and persistent mental illness. Each clinic has the same access to health care and preventive services.

The intervention

Male and female evidence-based Preventive Care Checklist Forms© were developed using Canadian Task Force on Preventive Health Care recommendations and other sources where the Task Force had no up-to-date guidelines [see Additional File 1 and 2]. Grade A (good evidence to include) or B (fair evidence to include) recommendations were delineated by bold and italics text respectively. Non-evidence based but practice relevant components including functional inquiry and general physical examination were added. Male and female forms were photocopied on blue and pink paper respectively. An explanation sheet detailing the recommendations accompanied the form [see Additional File 3]. Pilot testing was conducted on 10 unaffiliated family physicians.

Stratified randomization by clinic size was used since two of the four clinics are large and the other two smaller and community-based. Clinics were randomized using a random number table. CONSORT guidelines for cluster randomized controlled trials were adhered to [47, 48].

All participating physicians gave informed consent for a trial on preventive medicine. No other details of the study were disclosed. In the control clinics, usual preventive care did not include any structured or organizational components. In the intervention clinics, physicians, nurses and clerical staff were informed at staff meetings, by email and in person that Preventive Care Checklist Forms© were available. They were not aware that the forms were part of a study or if the forms were to be evaluated. Sex-specific forms were attached by clerical staff to the chart of adults booked for complete health check-ups, which are usually 20–30 minute scheduled appointments. No other additions or restructuring of preventive services took place during the study period. Completed forms acted as documentation of the visit and were filed in the patient's chart. The forms were implemented in the intervention group during a 5-month period from November 2002 to March 2003. The study was approved by St. Michael's Hospital Research Ethics Board, Toronto, Canada.

Data collection

A simple stratified random sampling technique generated lists from the insurance billing database of adult patients who had a complete health check-up, of equal proportion per physician, from January to June 2002. Baseline characteristics of the patient and preventive health services offered to the patient were recorded on a standardized form by two chart abstractors. Chart abstractors were graduate students in epidemiology, trained using standard methods. Abstractors were blinded in the period before the intervention. The presence of the form precluded blinding in the post-intervention period. An identical chart review was carried out using a similar sampling technique for the post-intervention period from November 2002 to March 2003. A summary of physician practice profiles and physician characteristics was determined using the hospital and insurance billing databases averaged over the study period.

Thirteen pre-selected preventive maneuvers were recorded out of a possibility of 41 A and B evidence-based recommendations. The maneuvers selected were a mixture of preventive health services known to be well (blood pressure, clinical breast exam, Pap smear and smoking history), moderately (alcohol history, brushing/flossing teeth, hearing assessment, mammogram and smoking cessation) and poorly (folic acid supplementation, fecal occult blood testing, rubella immunity and tetanus immunization) implemented in our setting based on continuous quality improvement assessments, and rates found in other settings [20, 21, 24]. Aside from the study investigators, the clinics were not aware how or which preventive maneuvers, if any, were to be evaluated.

The whole chart, including requisitions, laboratory reports and consult notes, was reviewed to determine rates of the thirteen preventive maneuvers. A maneuver was considered complete if either the patient received or was offered yet refused it. Maneuvers not documented were considered not performed. Charts were excluded if the patient was younger than 21, an investigator completed the assessment or the encounter was disease/illness specific. To determine if a preventive health service was provided to a patient, age and/or sex specific criteria had to be met based on the criteria for each preventive health maneuver from the Canadian Task Force on Preventive Health Care [17]. The following exclusions were applied only to the eligibility of a patient to receive a particular preventive health service; the whole patient chart was not excluded. Women 21 to 45 years were excluded from folic acid supplementation and rubella immunity if they had a hysterectomy, bilateral oopherectomy, tubal ligation or confirmed menopause. Women 21 to 69 years were excluded from a Pap smear if they had a total abdominal hysterectomy, cervix removed or were never sexually active. Eligible women were excluded from a screening breast exam and mammogram if they had a previous history of breast cancer. Adults older than 50 years were excluded from fecal occult blood testing if they had a history of colon cancer or a sigmoidscopy/colonoscopy in the past 5 years. Seniors were excluded from a hearing assessment if they were known to be deaf.

Analysis

The sample size was calculated to detect a mean difference of 30%. For the sample size calculation, consistent with other studies, we assumed a conservative estimates of a 50% prevalence for each maneuver at baseline, an ICC of 0.02, a 5% type I error, and 85% power [49, 50]. Two-hundred twenty charts in each of the control and intervention group were needed to detect a 30% change. Using an alpha type I error of 0.05 and 90% power, 273 charts were required per group.

Data were input into SPSS version 11 and analyzed using SAS 8.2. An intention-to-treat analysis was used for form utilization. No forms were found in the control site.

The overall summary outcome was the percentage of up-to-date preventive health services delivered per patient between the two groups, computed using a t-test. The rates and relative risks of the performance of the 13 preventive health maneuvers were the trial endpoints. In a univariate analysis, if a potential confounder was significant (p < 0.10) for a single maneuver it was retained in the final model for all maneuvers. Adjusted odds ratios were calculated in a separate model for each maneuver using Poisson regression with log link, controlling for cluster randomization, pre-intervention rate of the maneuver for that group, the number of years in practice of the physician, average proportion of patients seen per half-day, average number of visits per patient, Charlson comorbidity scale [51], and mental health diagnosis. The Charlson comorbidty scale and mental health diagnosis together were used as a proxy of patient complexity and comorbidity in this ambulatory care setting. Logistic regression was not used since preventive outcomes were not rare, and odds ratios would over-estimate the true relative risks. With Poisson regression, the adjusted relative risks could be estimated [52]. All outcomes were controlled for cluster randomization.

Results

A total of 38 of 42 physicians were involved in the trial (90.4%), 20 in the intervention arm, 18 in the control. Three physicians were excluded because they were involved in the study and one did not have an active practice. The adult health check-up was billed in 7.7% of all ambulatory encounters, and 33.6% of patients had at least one health check-up billed in 2002.

At baseline, a total of 509 patient charts were included in the study (Figure 1). Table 1 lists baseline characteristics of patients, physicians and physician practice profiles. Compared with the control arm, the intervention arm had a higher proportion of female patients, less comorbidity, fewer patients with a mental health diagnosis and physicians were somewhat older, in practice longer and had more health check-ups per patient visit. Baseline data on education, occupation, income, language and ethnicity were not included because of poor chart documentation.
Figure 1

Randomization process and progress through the study.

Table 1

Characteristics of patients, physicians, and physician practice profiles at baseline assessment

Patient Characteristics:

Intervention n (%)

Control n (%)

Sample Size

248

261

Mean Age (SD)

47.0 (16.5)

41.9 (14.1)

Female

160(64.5)

148(56.7)

Married/Common-law

91(40.1)

74(33.5)

Charlson Comorbidity Scale*

  

   0–2

240(96.7)

233(89.3)

   3–8

8(3.3)

28(10.7)

Mental Health Diagnosis

25(10.2)

52(20.1)

Number of visits at Clinic

  

   ≤3 visits

62(25.1)

53(20.9)

   4–9 visits

40(16.2)

53(20.9)

   ≥10 visits

145(58.7)

148(58.2)

Resident Involved

16(6.6)

28(10.7)

Physician Characteristics:

  

Sample Size

20

18

Mean Age (SD)

45.9 (10.1)

37.8 (7.3)

Female Sex

11(55.0)

11(61.1)

Mean Number of Years in Practice (SD)

16.6 (9.1)

7.5 (6.2)

Physician Practice Profiles:

  

Mean Age of Patients (SD)

46.5 (4.4)

43.9 (3.0)

Females in Practice

(63)

(48.9)

Complete Health Check-ups per Patient Visits

(10.0)

(5.8)

Mean Half-Days per Week (SD)

5.5 (2.8)

5.6 (1.3)

Mean Patient Seen per Clinic Day (SD)

14.0 (7.0)

13.7 (3.6)

Mean Patient Visits per Patient (SD)

3.0 (0.5)

3.0 (0.7)

*Charlson ME, Pompei P, Ales KL, McKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: Development and validation. J Chron Dis 1987;40:373–383.

Inter-observer reliability was high with a kappa of 0.80 (range 0.70 to 0.97). For coders A and B, the intra-observer reliability was κ = 0.94 (range 0.80 to 1.0) and κ = 0.87 (range 0.64 to 0.97) respectively.

During the post-intervention period, a total of 608 patient charts were analyzed. More patients were included compared to baseline because of fewer chart exclusions. The rates of preventive health maneuvers at baseline and post-intervention in both groups are listed in Table 2. At baseline, there were no significant differences between the intervention and control group with the exception of brushing/flossing teeth (p < 0.05). However, after controlling for physician practice profiles (mean age of patients, number of health check-ups per patient visits, mean half-days per week, mean patients seen per clinic day and mean patient visits per patient) differences in all 13 maneuvers became non-significant at baseline. Rates varied from 3% for fecal occult blood testing to 93% for blood pressure measurement.
Table 2

Rates of preventive maneuvers performed at baseline and post-Intervention in both Groups

 

Intervention Group

Control Group

 

Baseline

Post-Intervention

Change

P-value

Baseline

Post-Intervention

Change

P-value

Preventive Maneuver

n (%)

n (%)

%

 

n (%)

n (%)

%

 

Brushing/Flossing

174 (29.8)

310 (47.9)

18.1

<0.001

239 (8.4)

298 (6.7)

-1.7

0.51

Blood Pressure

246 (93.5)

310 (96.6)

3.1

0.11

259 (92.7)

298 (93.0)

0.3

0.89

History of Alcohol

248 (66.5)

310 (90.9)

24.4

<0.001

261 (74.7)

298 (74.8)

0.1

0.97

History of Smoking

248 (73.4)

310 (92.4)

19

<0.001

261 (79.7)

298 (79.2)

-0.5

0.91

Smoking Cessation

43 (39.5)

45 (70.0)

30.5

0.005

78 (29.5)

71 (25.4)

-4.1

0.58

Tetanus Immunization

242 (12.8)

308 (40.9)

28.1

<0.001

259 (19.7)

297 (9.4)

-10.3

<0.001

Folic Acid Counseling

96 (8.2)

119 (34.7)

26.5

<0.001

111 (10.9)

125 (3.2)

-7.7

0.03

Rubella Immunity

96 (15.5)

119 (34.7)

19.2

0.002

111 (10.0)

125 (9.6)

-0.4

0.93

Breast Exam

157 (82.1)

173 (97.4)

15.3

<0.001

149 (80.7)

148 (77.7)

-3

0.57

Mammography

55 (41.8)

50 (76.6)

34.8

<0.001

33 (57.5)

16 (50.0)

-7.5

0.76

Pap Smear

148 (73.6)

164 (84.7)

11.1

0.02

145 (86.2)

145 (88.3)

2.1

0.72

Fecal occult Blood

83 (13.4)

102 (50.6)

37.2

<0.001

55 (3.6)

67 (7.5)

3.9

0.45

Hearing Assessment

44 (31.8)

44 (41.2)

9.4

0.50

22 (22.7)

33 (12.1)

-10.6

0.45

The rate of form utilization was 84% in the intervention group. In the unadjusted intention-to-treat analysis during the post-intervention period, 11 of 13 maneuvers were statistically significant, all in support of the intervention (Table 2). After adjustment for confounders, 8 of 13 remained significant and rubella immunity, clinical breast exam and hearing assessment became non-significant, while blood pressure measurement reached significance (Table 3).
Table 3

Unadjusted* and adjusted relative risk of performance of preventive health maneuvers post-intervention

Preventive Maneuver

Sample Size

Unadjusted* Relative Risk (95% CI)

Adjusted† Relative Risk (95% CI)

Brushing/Flossing

605

6.96

(3.93–10.07)

9.19

(4.32–19.57)

Blood Pressure

605

1.03

(0.98–1.09)

1.05

(1.00–1.10)

History of Alcohol

605

1.23

(1.05–1.45)

1.33

(1.17–1.51)

History of Smoking

605

1.20

(1.05–1.37)

1.28

(1.16–1.42)

Smoking Cessation

116

2.55

(1.60–4.06)

3.93

(2.16–7.15)

Tetanus Immunization

605

4.35

(2.37–7.98)

3.00

(1.72–5.22)

Folic Acid Counseling

244

10.20

(4.36–23.90)

7.47

(2.69–20.75)

Rubella Immunity

244

3.43

(1.32–8.89)

3.14

(0.78–12.62)

Breast Exam

321

1.25

(1.09–1.43)

1.06

(0.97–1.16)

Mammography

66

1.49

(0.90–2.48)

1.41

(0.76–2.61)

Pap Smear

309

0.96

(0.86–1.07)

0.92‡

(0.83–1.01)

Fecal occult Blood

169

4.96

(1.81–13.55)

6.69

(1.85–24.17)

Hearing Assessment

77

9.50

(1.46–61.87)

5.13

(0.70–37.32)

*Controlled only for Cluster Randomization.

†Poisson regression (continuous variable unless otherwise stated) controlling at the level of the clinic: cluster randomization and the pre-intervention rate for that group; at the level of the physician: the number of years in practice; at the level of the practice demographics: mean proportion of patients seen per half-day and average number of visits per patient; and at the level of the patient: mean age, comorbidity (categories of 0, 1, 2), and mental health diagnosis (categories of Y, N).

‡For model stability, unable to adjust for mean patient age.

The overall summary statistic was a comparison of the percentage of up-to-date preventive health services delivered per patient in each group (Table 4). Before the intervention, each patient received on average 51.8% of maneuvers in the control group and 51.4% in the intervention group (p = 0.81). After the intervention, the results were significantly different (p = 0.0001). In the control group each patient was up-to-date on only 48.9% of eligible maneuvers, while the intervention group was up-to-date on 71.7% of eligible maneuvers. This is an absolute difference of 22.8% between the two groups, corresponding with a relative increase in preventive care of 46.6%.
Table 4

Summary outcome: percentage of up-to-date preventive health services delivered per patient at baseline and post-intervention

Before Intervention

Sample Size

Mean %

95% CI

Control Group

261

51.8

(49.8–53.9)

Intervention Group

248

51.4

(48.6–54.2)

After Intervention

   

Control Group

298

48.9

(47.0–50.8)

Intervention Group

310

71.7

(65.1–78.3)

Discussion

This study evaluates a single double-sided checklist reminder form to be used by physicians at adult health check-ups to improve a broad range of preventive health services. The results of this study strongly support the effectiveness of the Preventive Care Checklist Forms© to improve the delivery of preventive health care. This simple intervention resulted in a clinically important, 22.8% absolute change in preventive care, of larger magnitude than many multi-faceted complex prevention trials, where an absolute increase of 10% is usually considered an excellent outcome.

This study makes a significant contribution beyond prior work in this area for several reasons. Through this study, the Preventive Care Checklist Forms© have been validated and endorsed by the College of Family Physicians of Canada [see Additional Files 1, 2, 3]. They are available to be used by other clinics in English and French [53]. The forms can also be incorporated into an electronic medical record since they are available in an electronic format. Primary care practitioners and health service managers can easily implement this form into clinical practice and be confident that it will improve the delivery of preventive health services.

This trial was different from other reminder systems because it integrated existing practice styles. The single check-list form contains all recommended preventive services for healthy adults as suggested by the Canadian Task Force on Preventive Health Care and other sources, similar to the US Preventive Services Task Force. Many previous reminder forms focused on particular preventive maneuvers such as cancer screening or immunizations. They did not incorporate a broad range of preventive health services, nor existing non-evidence based practice components, such as physical examination and documentation space. Age- and sex-specific prompts were also valuable. No continuing medical education or training session was required to use the forms, though the forms were announced to physicians and medical staff at staff meetings, via email and memos. Forms were placed on the charts of patients scheduled for a check-up by clerical staff. Physicians adopted their own approach to utilize the form. The Preventive Care Checklist Form© is a comprehensive tool not only to deliver preventive health services to healthy adults, but also to act as a documentation tool for the encounter in general. Our hypothesis for why the forms were a success is because the forms provide a rapid way to administer preventive health services that is harmonious to practice, addressing many physicians' concerns that there is "no time for prevention" [29]. In Canada, the Rourke baby record is a widely used checklist for evidence-based pediatric care, so physicians were already familiar with using evidence-based checklist forms [5457].

No recent studies have demonstrated that checklist reminder sheets alone are still effective for family physicians, especially in today's technological age. Only 5–13% of primary care physicians in the United States have electronic health record systems [58]. Barriers to implement electronic patient care records include costs, varying and untested vendor quality and life expectancy and products that are not tailored to specific practice needs. Small practices may be the last to adopt electronic health records, especially without supports available including financial incentives [59, 60]. Seventy-eight percent of physicians in the United States practice in groups of 8 or fewer [58]. Small practices are also less likely to incorporate multi-system interventions to assist in the delivery of preventive health services. The Preventive Care Checklist Form© will be particularly attractive to these small practices. Still, those who have an electronic health record system can incorporate this form into a record of the adult health check-up, as physicians and health record companies in our area have already done so. The forms are available free of charge in a PDF format on-line or can be accessed as a word-processing document [53]. Perhaps the simpler interventions that incorporate existing practice patterns are more effective than complex technological interventions that require training and/or continuing medical education.

Individualization of the forms will also enhance their ability to be implemented into practice. Some physicians/practices may want additional space or to add other recommendations based on their client population. In our study, physicians adopted their own style to use the form. Some physicians completed the form in a single visit. Others completed the history and physical examination in one visit, ordered necessary testing, and brought the patient back in a second visit to go over the results of investigations and provide health habit counseling. Others had a nurse visit with the patient before the physician encounter where the nurse filled in many components of the form. While the form was standardized, its use by physicians was varied. Overall, 77% of physicians who used the form in this intervention said they would continue to use the form in their practice [61]. The commonest reasons for not adopting the forms in routine care were because the layout was thought not to flow easily and to be cluttered, too much repetition, time consuming, or a dislike of using standardized forms [61]. Individualization in the utilization of the form is another reason for its continued success. Individualization has been proven effective in other settings as well and is a successful method to change provider behavior in an acceptable fashion [62, 63]. For instance, large staffed health care systems may consider the incorporation of the forms as part of a tool for a pre-visit check by nurses, or further developed and extended based on the goals of the organization.

Much work has been carried out to determine why the implementation of clinical practice guidelines and preventive health guidelines has been so poorly adopted. Numerous interventions and meta-analyses have been conducted to determine which interventions are most successful. Unfortunately, no one intervention or group of interventions has consistently shown to be effective. Even physicians' inclination to provide preventive care is not sufficient as an isolated factor to guarantee that patients will receive preventive health services [45]. Active interventions are more effective than passive interventions and educational outreach and reminders continue to be considered "promising approaches." [8]. When most physicians think of clinical guidelines, on the one hand they think of cookbook medicine which takes away the individualized care plans for patients; on the other hand well developed and evidence-based guidelines can improve patient care [10]. The Preventive Care Checklist Form© is a tool that incorporates the evidence in a way that allows physician flexibility for individual care. Studies have also shown that as physicians age, they are not keeping up with evidence-based approaches to patient care [64]. Having an evidence-based reminder system for physicians can ensure that preventive health services remain up-to-date. Clinicians and clinic mangers also differ in their reasons for using evidence. Clinicians invoke clinical intuition as a guide to most routine clinical decisions, and managers articulate both motivation and interest in using medical research to guide decision-making, most commonly promoted by cost [65]. The Preventive Care Checklist Form© addresses both of these stances.

Some have argued that it may simply be unrealistic to expect primary care clinicians to deliver a comprehensive package of clinical preventive services among the many competing demands of providing primary care, especially since it is suggested that 7.4 hours a day are required to provide only preventive services in primary practice [24, 31]. Others have suggested that top ranked preventive services based on burden of disease and cost-effectiveness, such as tobacco counseling or colorectal cancer screening, be implemented with full force over other preventive services [66]. The success of the Preventive Care Checklist Form© to improve the delivery of a wide range of preventive health services refutes these arguments. Others argue that health habit counseling is associated with diminished patient satisfaction. However, in one study, providing tobacco screening and smoking cessation counseling improved patient satisfaction [67].

There are some limitations to consider. Intervention and control patients and physicians were somewhat different in characteristics that may be associated with preventive service delivery. While the groups varied somewhat in physician and patient composition, rates of preventive service delivery were very similar in both groups at baseline. After the intervention, preventive service delivery only changed substantially in the intervention group. As well, controlling for these differences and other potential confounders did not change the main results of the study. Other patient characteristics such as education, occupation, income, language and ethnicity were not available in our setting. Randomization of individual physicians was not appropriate due to the likelihood of contamination between physicians seeing each other's patients at the same clinic. There was no evidence for contamination of the use and/or knowledge of the forms in the control group. Actual use of the forms was limited to the intervention group only and any contamination would bias the results toward the null since it would encourage improved preventive service delivery in the control group. The rates of preventive service delivery did not change in the control group compared to baseline. Apart from the forms we were not aware of any changes in preventive practices or patient population during the five-month study period. This study was conducted at four academic clinics in an urban setting in Canada which may not be generalizable to other settings. Still, the forms are applicable to primary care in general and the baseline rates of preventive maneuvers at our centre are similar to other sites in diverse, non-academic settings [21, 24]. Our main outcome measure did not account for preventive measures that were done and not documented or those that were documented and not done, which likely under-represented rates of performance. However, this assumption is consistent with medico-legal standards. Due to age, sex and other eligibility criteria, some preventive health services had small sample sizes, but still many of the results were positive despite decreased power in these sub-strata. We assumed the ICC for primary care outcomes was 0.02 [50]. Given the many statistically significant outcomes, a type II error is minimal. We only looked at preventive maneuvers carried out pre-booked adult health check-ups. A patient who comes in for a complete check-up may not be comparable to a patient who only comes in only for illness visits. Still, acute care needs can be addressed at a complete health check-up, but the focus is on a comprehensive approach. Testing of the form in a broader range of visit types in subsequent work would be helpful since it may be difficult to assure a wellness visit for chronically ill patients.

Some of the post-intervention rates of preventive maneuvers were limited by ceiling effects, such as blood pressure, clinical breast exam, history of smoking and history of alcohol, which all reached rates of over 90%. Of the maneuvers that showed a significant change, there were no consistent patterns favoring health habit counseling, physical examination or procedure/investigation maneuvers, nor were outcomes related to the strength of evidence for the recommendation. Maneuvers that reached statistical significance are not due to documentation alone, since certain services, such as fecal occult blood testing and rubella immunity, have laboratory reports documenting they were performed, rather than just a written check by the physician. These findings suggest that improved delivery of preventive services in the intervention arm is not accounted for by improved documentation alone or preference for maneuvers with more evidence. Still, improved documentation alone will reduce the over delivery of certain services which will in turn free up time for additional services to be delivered [40].

The Task Force and others have recommended the abandonment of the annual health check-up and suggest that preventive health maneuvers be completed at periodic health examinations and others have questioned its necessity [68, 69]. However studies have shown that both physicians and patients prefer and are more likely to carry out preventive health maneuvers at routine well visits [6871]. A complete health check-up is a chance to build trust with the patient, to be thorough, and to use it as an organizational strategy to offer preventive health care. Patients are also more likely to recall advice during a well care visit such as a health check-up than during an illness visit [72].

Conclusion

The Preventive Care Checklist Form© is an evidence-based, validated, simple, low cost, clinically relevant tool [see Additional Files 1, 2, 3]. It facilitates uptake into practice in an attractive way by making the delivery of preventive health care easier rather than more difficult. It prompts physicians about issues that are difficult to remember, especially since many maneuvers are age and sex-specific. It incorporates requirements from insurance payers and practice patterns that are not evidence-based, but a part of routine care. In our setting, no documentation other than the form is required for the visit. The colourful form is easy to find in a chart and facilitates documentation of ongoing preventive care. It is easy to incorporate into practice and is inexpensive. It does not require computer systems, nurse facilitators or patient materials, yet it can be readily incorporated into an electronic medical record system. Periodic updates of the form will keep preventive primary care current with evolving evidence, and ease the delivery of preventive services.

Declarations

Acknowledgements

This research was supported by physicians of Ontario through a grant from The P.S.I. Foundation. The physicians and staff at St. Michael's Hospital family practice units for partaking in the study and incorporating the Preventive Care Checklist Form© into regular patient care; Dr. Prabo Dwight for help with form design and invaluable feedback; Eleanor Boyle for guidance on study design; Francesca Reyes and Babak Aliarzadeh for collecting and inputting the data. The forms were inspired by the Rourke Evidence-based Baby Record.

Authors’ Affiliations

(1)
Dept of Public Health Sciences, University of Toronto
(2)
Dept of Family and Community Medicine, St. Michael's Hospital
(3)
Inner City Health Research Unit, University of Toronto and St. Michael's Hospital

References

  1. Anderson LM, May DS: Has the use of cervical, breast and colorectal cancer screening increased in the United States?. Am J Public Health. 1995, 85: 840-842.View ArticlePubMedPubMed CentralGoogle Scholar
  2. Backer EL, Geske JA, McIlvain HE, Dodendorf DM, Minier WC: Improving female preventive health care delivery through practice change: An every woman matters study. J Am Board Fam Pract. 2005, 18: 401-8.View ArticlePubMedGoogle Scholar
  3. Basinski A: Evaluation of clinical practice guidelines. CMAJ. 1995, 153: 1575-81.PubMedPubMed CentralGoogle Scholar
  4. Davis DA, Taylor-Vaisey A: Translating guidelines into practice. A systematic review of theoretic concepts, practical experience and research evidence in the adoption of clinical practice guidelines. CMAJ. 1997, 157: 408-16.PubMedPubMed CentralGoogle Scholar
  5. Ellis P, Robinson P, Ciliska D, Armour T, Brouwers M, O'Brien MA, Sussman J, Raina P: A systematic review of studies evaluating diffusion and dissemination of selected cancer control interventions. Health Psychology. 2005, 24: 488-500. 10.1037/0278-6133.24.5.488.View ArticlePubMedGoogle Scholar
  6. Ellrodt AG, Conner L, Riedinger M, Weingarten S: Measuring and Improving Physician Compliance with Clinical Practice Guidelines: A controlled intervention trial. Ann Intern Med. 1995, 122 (4): 277-282.View ArticlePubMedGoogle Scholar
  7. Feldstein A, Elmer PJ, Orwoll E, Herson M, Hillier T: Bone mineral density measurement and treatment for osteoporosis in older individuals with fractures: A gap in evidence-based practice guideline implementation. Arch Intern Med. 2003, 163: 2165-2172. 10.1001/archinte.163.18.2165.View ArticlePubMedGoogle Scholar
  8. Grimshaw J, Russell I: Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet. 1993, 342: 1317-22. 10.1016/0140-6736(93)92244-N.View ArticlePubMedGoogle Scholar
  9. Grol R: Successes and Failures in the Implementation of Evidence-Based Guidelines for Clinical Practice. Medical Care. 2001, 39 (8,Suppl 2): II46-54. 10.1097/00005650-200108002-00003.PubMedGoogle Scholar
  10. Schulte B, O'Hea EL, Darling P: Putting Clinical Guidelines into Practice. Fam Practice Management. 2001, 8: 45-46.Google Scholar
  11. Wei Ek, Ryan CT, Dietrich AJ, Colditz GA: Improving colorectal cancer screening by targeting office systems in primary care practices: Disseminating research results into clinical practice. Arch Intern Med. 2005, 165: 661-666. 10.1001/archinte.165.6.661.View ArticlePubMedGoogle Scholar
  12. Weingarten S: Using Practice Guideline Compendiums to Provide Better Preventive Care. Ann of Intern Med. 1999, 130 (5): 454-458.View ArticleGoogle Scholar
  13. Weingarten S, Stone E, Hayward R, Tunis S, Pelter M, Huang H, Kristopaitis R: The Adoption of Preventive Care Practice Guidelines by Primary Care Physicians: Do Actions Match Intentions?. J Gen Intern Med. 1995, 10: 138-144.View ArticlePubMedGoogle Scholar
  14. Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud P-AC, Rubin HR: Why don't' physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999, 282: 1458-1465. 10.1001/jama.282.15.1458.View ArticlePubMedGoogle Scholar
  15. Chaudhry R, Kottke TE, Naessens JM, Johnson TJ, Nyman MA, Cornelius LA, Petersen JD: Busy physicians and preventive services for adults. May Clin Proc. 2000, 75: 156-162.View ArticleGoogle Scholar
  16. Hudon E, Beaulieu M-D, Roberge D: Integration of the recommendations of the Canadian Task Force on Preventive Health Care. Family Practice. 2004, 21: 11-17. 10.1093/fampra/cmh104.View ArticlePubMedGoogle Scholar
  17. Canadian Task Force on the Periodic Health Examination: The Canadian guide to clinical preventive health care. 1994, Ottawa: Health CanadaGoogle Scholar
  18. U.S. Preventive Services Task Force: Guide to Clinical Preventive Services. 2004, Agency for Healthcare Research and Quality, 3Google Scholar
  19. Abelson J, Lomas J: Do health service organizations and community health centres have higher disease prevention and health promotion levels than fee-for-service practices?. CMAJ. 1990, 142 (6): 575-581.PubMedPubMed CentralGoogle Scholar
  20. Carney PA, Dietrich AJ, Freeman DH, Mott LA: The Periodic Health Examination Provided to Asymptomatic Older Women: An Assessment Using Standardized Patients. Ann Intern Med. 1993, 119 (2): 129-135.View ArticlePubMedGoogle Scholar
  21. Hutchison B, Woodward CA, Norman GR, Abelson J, Brown JA: Provision of preventive care to unannounced standardized patients. CMAJ. 1998, 158 (2): 185-193.PubMedPubMed CentralGoogle Scholar
  22. Morris P, Morris E: Family practice residents' compliance with preventive medicine recommendations. Am J Prev Med. 1988, 4 (3): 161-5.PubMedGoogle Scholar
  23. Smith H, Herbert C: Preventive Practice among Primary Care Physicians in British Columbia: relation to recommendations of Canadian Task for on the Periodic Health Examination. CMAJ. 1993, 149 (12): 1795-1800.PubMedPubMed CentralGoogle Scholar
  24. Stange KC, Flocke SA, Goodwin MA, Kelly RB, Zyzanski SJ: Direct observation of rates of preventive service delivery in community family practice. Prev Med. 2000, 31: 167-176. 10.1006/pmed.2000.0700.View ArticlePubMedGoogle Scholar
  25. Lafata JE, Williams LK, Ben-Menachem T, Moon C, Divine G: Colorectal carcinoma screening procedure use among primary care patients. Cancer. 2005, 104: 1356-1361. 10.1002/cncr.21333.View ArticlePubMedGoogle Scholar
  26. Walsh JM, Posner SF, Perez-Stable EJ: Colon cancer screening in the ambulatory setting. Prev Med. 2002, 31: 167-176.Google Scholar
  27. Zapka JG, Paleo E, Vickers-Lahti M, Luckmann R: Healthcare system factors and colorectal cancer screening. Am J Prev Med. 2002, 23: 28-35. 10.1016/S0749-3797(02)00444-0.View ArticlePubMedGoogle Scholar
  28. McPhee S, Detmer W: Office-Based Interventions to Improve Delivery of Cancer Prevention Services by Primary Care Physicians. Cancer. 1993, 72 (3): 1100-12. 10.1002/1097-0142(19930801)72:3+<1100::AID-CNCR2820721327>3.0.CO;2-N.View ArticlePubMedGoogle Scholar
  29. Kottke T, Brekke M, Solberg L: Making "Time" for Preventive Services. Mayo Clin Proc. 1993, 68: 785-791.View ArticlePubMedGoogle Scholar
  30. Szilagyi PG, Shone LP, Barth R, Kouides RW, Long C, Humiston SG, Jennings J, Bennett NM: Physician practices and attitudes regarding adult immunizations. Prev Med. 2005, 40: 152-161. 10.1016/j.ypmed.2004.05.010.View ArticlePubMedGoogle Scholar
  31. Yarnall KSH, Pollak KI, Ostbye T, Krause KM, Michener JL: Primary Care: Is There Enough Time for Prevention?. Am J Public Health. 2003, 93 (4): 635-641.View ArticlePubMedPubMed CentralGoogle Scholar
  32. Balas EA, Weingarten S, Garb CT, Blumenthal D, boren SA, Brown GD: Improving preventive care by prompting physicians. Arch Intern Med. 2000, 160: 301-308. 10.1001/archinte.160.3.301.View ArticlePubMedGoogle Scholar
  33. Stone EG, Morton SC, Hulscher ME, Maglione MA, Roth EA, Grimshaw JM, Mittman BS, Rubenstein LV, Rubenstein LZ, Shekelle PG: Interventions that Increase Use of Adult Immunization and Cancer Screening Services: A Meta-Analysis. Ann Intern Med. 2002, 136 (9): 641-651.View ArticlePubMedGoogle Scholar
  34. Davis J, Patrick P, Bobula J: Improving prevention in primary care: Physicians, patients, and process. J of Fam Practice. 1992, 35 (4): 385-387.Google Scholar
  35. Davis D, Evans M, Jadad A, Perrier L, Rath D, Ryan D, Sibbald G, Straus S, Rappolt S, Wowk M, Zarenstein M: The case for knowledge translation: shortening the journey from evidence to effect. BMJ. 2003, 327: 33-35. 10.1136/bmj.327.7405.33.View ArticlePubMedPubMed CentralGoogle Scholar
  36. Margolis PA, Lannon CM, Stuart JM, Fried BJ, Keyes-Elstein L, Moore DE: Practice based education to improve delivery systems for prevention in primary care: randomized trial. BMJ. 2004, 328: 388-394. 10.1136/bmj.38009.706319.47.View ArticlePubMedPubMed CentralGoogle Scholar
  37. Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, Grilli R, Harvey E, Oxman A, O'Brien MA: Changing provider behavior: An overview of systematic reviews of interventions. Medical Care. 2001, 39 (S2): II-2-II-45.Google Scholar
  38. Hulscher ME, Wensing M, Grol RP, van der Weijden T, van Weel C: Interventions to improve the delivery of preventive services in primary care. Am J of Public Health. 1999, 89 (5): 737-746.View ArticleGoogle Scholar
  39. Carpiano RM, Flocke SA, Frank SH, Stange KC: Tools, teamwork, and tenacity: an examination of family practice office system influences on preventive service delivery. Prev Med. 2003, 36: 131-140. 10.1016/S0091-7435(02)00024-5.View ArticlePubMedGoogle Scholar
  40. Goodwin MA, Zyzanski SJ, Zronek S, Ruhe M, Weyer SM, Konrad N, Esola D, Stange KC: A clinical trial of tailored office systems for preventive service delivery: the Study to Enhance Prevention by Understanding Practice. Am J Prev Med. 2001, 21: 20-8. 10.1016/S0749-3797(01)00310-5.View ArticlePubMedGoogle Scholar
  41. Cheney C, Ramsdell J: Effect of medical records' checklists on implementation of periodic health measures. Am J of Medicine. 1987, 83: 129-36. 10.1016/0002-9343(87)90507-9.View ArticleGoogle Scholar
  42. Cohen DI, Littenberg B, Wetzel C, Neuhauser D: Improving physician compliance with preventive medicine guidelines. Medical Care. 1982, 20 (10): 1040-5.View ArticlePubMedGoogle Scholar
  43. Cowan J, Heckerling P, Parker J: Effect of a fact sheet reminder on performance of the periodic health examination: A randomized control trial. Am J of Preventive Med. 1992, 8 (2): 104-9.Google Scholar
  44. Davidson RA, Fletcher SW, Retchin S, Duh S: A nurse-initiated reminder system for the periodic health examination: Implementation and Evaluation. Arch Intern Med. 1984, 144: 2167-70. 10.1001/archinte.144.11.2167.View ArticlePubMedGoogle Scholar
  45. Szilagyi PG, Bordley C, Vann JC, Chelminski A, Kraus RM, Margolis PA, Rodewald LE: Effect of Patient Reminder/Recall Interventions on Immunization Rates. JAMA. 2000, 284 (14): 1820-1827. 10.1001/jama.284.14.1820.View ArticlePubMedGoogle Scholar
  46. Litaker D, Flocke SA, Frolkis JP, Stange KC: Physicians' attitudes and preventive care delivery: insights from the DOPC study. Prev Med. 2005, 40: 556-563. 10.1016/j.ypmed.2004.07.015.View ArticlePubMedGoogle Scholar
  47. Moher D, Schultz KF, Altman DG: The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials. BMC Medical Research Methodology. 2001, 1: 2-10.1186/1471-2288-1-2.View ArticlePubMedPubMed CentralGoogle Scholar
  48. Elbourne DR, Campbell MK: Extending the CONSORT statement to cluster randomized trials: for discussion. Statist Med. 2001, 20: 489-496. 10.1002/1097-0258(20010215)20:3<489::AID-SIM806>3.0.CO;2-S.View ArticleGoogle Scholar
  49. Baskerville N, Hogg W, Lemelin J: The Effect of Cluster Randomization on Sample Size in Prevention Research. J of Fam Pract. 2001, 50 (3): W241-W246.Google Scholar
  50. Campbell MK, Mollison J, Steen N, Grimshaw JM, Eccles M: Analysis of cluster randomized trials in primary care: a practical approach. Fam Pract. 2000, 17: 192-6. 10.1093/fampra/17.2.192.View ArticlePubMedGoogle Scholar
  51. Charlson ME, Pompei P, Ales KL, McKenzie CR: A new method of classifying prognostic comorbidity in longitudinal studies: Development and validation. J Chron Dis. 1987, 40: 373-383. 10.1016/0021-9681(87)90171-8.View ArticlePubMedGoogle Scholar
  52. McNutt LA, Wu C, Xue X, Hafner JP: Estimating the Relative Risk in Cohort Studies and Clinical Trials of Common Outcomes. Am J of Epidemiol. 2003, 157 (10): 940-943. 10.1093/aje/kwg074.View ArticleGoogle Scholar
  53. The College of Family Physicians of Canada. [http://www.cfpc.ca]
  54. Gundrum S: Well-baby Care in the New Millennium: Evidence-based Care and the New Rourke Baby Record. Can Fam Physician. 1998, 44: 470-471.PubMedPubMed CentralGoogle Scholar
  55. Panagiotou L, Rourke L, Rourke J, Wakefield J, Winfield D: Evidence-based Well-baby Care. Part 1: Overview of the next generation for the Rourke Baby Record. Can Fam Physician. 1998, 44: 558-67.PubMedPubMed CentralGoogle Scholar
  56. Panagiotou L, Rourke L, Rourke J, Wakefield J, Winfield D: Evidence-based Well-baby Care. Part 2: Education and advice section of the next generation of the Rourke Baby Record. Can Fam Physician. 1998, 44: 558-67.PubMedPubMed CentralGoogle Scholar
  57. Rourke L: Developing the Rourke Baby Record. Paediatr Child Health. 1998, 3 (5): 315-320.PubMedPubMed CentralGoogle Scholar
  58. Baron RJ, Faben EL, Schiffman M, Wolf E: Electronic health records: Just around the corner? Or over the cliff?. Ann Intern Med. 2005, 143: 222-226.View ArticlePubMedGoogle Scholar
  59. Basch P: Electronic health records and the national health information network: Affordable, adoptable, and ready for prime time?. Ann Intern Med. 2005, 143: 227-228.View ArticlePubMedGoogle Scholar
  60. Editors: Have paper records passed their expiry date?. Can Med Assoc J. 2005, 173: 725-10.1503/cmaj.051229.Google Scholar
  61. Dubey V, Glazier R: Preventive Care Checklist Form©: Evidence-based tool to improve preventive health care during complete health assessment of adults. Can Fam Physician. 2006, 52: 48-55.PubMedPubMed CentralGoogle Scholar
  62. Stange KC: "One size doesn't fit all": Multimethod research yields new insights into interventions to improve preventive service delivery in family practice. J Fam Pract. 1996, 43: 358-60.PubMedGoogle Scholar
  63. Stange KC, Goodwin MA, Zyzanski SJ, Dietrich AJ: Sustainability of a practice-individualized preventive service delivery intervention. Am J Prev Med. 2003, 25: 296-300. 10.1016/S0749-3797(03)00219-8.View ArticlePubMedGoogle Scholar
  64. Norman GR, Eva KW: Does clinical experience make up for failure to keep up to date?. Evidence-based Medicine. 2005, 10: 66-67. 10.1136/ebm.10.3.66-a.View ArticleGoogle Scholar
  65. Lorenz KA, Ryan GW, Morton SC, Chan KS, Wang S, Shekelle PG: A qualitative examination of primary care providers' and physician managers' uses and views of research evidence. International J for Quality in Health Care. 2005, 17: 409-414. 10.1093/intqhc/mzi054.View ArticleGoogle Scholar
  66. Coffield AB, Maciosek MV, McGinnis JM, Harris JR, Caldwell MB, Teutsch SM, Atkins D, Richland JH, Haddix A: Priorities among recommended clinical preventive services. Am J Prev Med. 2001, 21: 1-9. 10.1016/S0749-3797(01)00308-7.View ArticlePubMedGoogle Scholar
  67. Barzilai DA, Goodwin MA, Zyzanski SJ, Stange KC: Does Health Habit Counseling Affect Patient Satisfaction?. Prev Med. 2001, 33: 595-599. 10.1006/pmed.2001.0931.View ArticlePubMedGoogle Scholar
  68. Gordon PR, Senf J, Campos-Outcalt D: Is the Annual Complete Physical Examination Necessary?. Arch Intern Med. 1999, 159: 909-10. 10.1001/archinte.159.9.909.View ArticlePubMedGoogle Scholar
  69. Laine C: The Annual Physical Examination: Needless Ritual or Necessary Routine?. Ann Intern Med. 2002, 136 (9): 701-3.View ArticlePubMedGoogle Scholar
  70. Beaulieu MD, Hudon E, Roberge D, Pineault R, Forte D, Legare J: Practice guidelines for clinical prevention: Do patients, physicians and experts share common ground?. CMAJ. 1999, 161 (5): 519-523.PubMedPubMed CentralGoogle Scholar
  71. Cooper GS, Goodwin MA, Stange KC: The delivery of preventive services for patient symptoms. Am J Prev Med. 2001, 21: 177-181. 10.1016/S0749-3797(01)00348-8.View ArticlePubMedGoogle Scholar
  72. Flocke SA, Stange KC: Direct observation and patient recall of health behavior advice. Prev Med. 2004, 38: 343-349. 10.1016/j.ypmed.2003.11.004.View ArticlePubMedGoogle Scholar
  73. Pre-publication history

    1. The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2296/7/44/prepub

Copyright

© Dubey et al; licensee BioMed Central Ltd. 2006

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Advertisement