Open Access

An international cross-sectional survey on the Quality and Costs of Primary Care (QUALICO-PC): recruitment and data collection of places delivering primary care across Canada

  • Sabrina T Wong1, 2, 3, 4Email author,
  • Leena W Chau1, 2,
  • William Hogg4, 5, 6,
  • Gary F Teare7, 9, 10,
  • Baukje Miedema11, 12,
  • Mylaine Breton13,
  • Kris Aubrey-Bassler14, 15,
  • Alan Katz16, 17,
  • Fred Burge11,
  • Antoine Boivin18,
  • Tim Cooke19,
  • Danièle Francoeur20 and
  • Walter P Wodchis7, 8, 9
BMC Family Practice201516:20

DOI: 10.1186/s12875-015-0236-7

Received: 27 October 2014

Accepted: 29 January 2015

Published: 18 February 2015

Abstract

Background

Performance reporting in primary health care in Canada is challenging because of the dearth of concise and synthesized information. The paucity of information occurs, in part, because the majority of primary health care in Canada is delivered through a multitude of privately owned small businesses with no mechanism or incentives to provide information about their performance. The purpose of this paper is to report the methods used to recruit family physicians and their patients across 10 provinces to provide self-reported information about primary care and how this information could be used in recruitment and data collection for future large scale pan-Canadian and other cross-country studies.

Methods

Canada participated in an international large scale study-the QUALICO-PC (Quality and Costs of Primary Care) study. A set of four surveys, designed to collect in-depth information regarding primary care activities was collected from: practices, providers, and patients (experiences and values). Invitations (telephone, electronic or mailed) were sent to family physicians. Eligible participants were sent a package of surveys. Provincial teams kept records on the number of: invitation emails/letters sent, physicians who registered, practices that were sent surveys, and practices returning completed surveys. Response and cooperation rates were calculated.

Results

Invitations to participate were sent to approximately 23,000 family physicians across Canada. A total of 792 physicians and 8,332 patients from 772 primary care practices completed the surveys, including 1,160 participants completing a Patient Values survey and 7,172 participants completing a Patient Experience survey. Overall, the response rate was very low ranging from 2% (British Columbia) to 21% (Nova Scotia). However, the participation rate was high, ranging from 72% (Ontario) to 100% (New Brunswick/Prince Edward Island and Newfoundland & Labrador).

Conclusions

The difficulties obtaining acceptable response rates by family physicians for survey participation is a universal challenge. This response rate for the QUALICO-PC arm in Canada was similar to rates found in other countries such as Australia and New Zealand. Even though most family physicians operate as self-employed small businesses, they could be supported to routinely submit data through a collective effort and provincial mandate. The groundwork in setting up pan-Canadian collaboration in primary care has been established through this study.

Keywords

Primary health care Physician Response rate French English Survey Practice

Background

In Canada, performance reporting in primary health care (PHC) is challenging because of the dearth of concise and synthesized information [1]. The paucity of information occurs, in part, because the majority of PHC is publicly funded through a single payer (i.e. provincial and federal governments) but delivered through a multitude of privately owned small businesses [2], also known as PHC practices. The majority of PHC practices are family physician owned and operated; they employ staff, pay overhead, and provide health care services to patients. The majority of family physicians’ income is derived from billing the government fees for services, which are negotiated between each provincial government and organized medicine in their respective jurisdictions, with some being paid through a blend of payments [3]. Many family physicians are part of a group or interprofessional practice, such as group practices or community health centres, while others provide care in a solo practice [3].

Outside Canada, reporting on PHC performance also remains challenging [4-7]. Indeed, the activity of reporting in PHC is not a traditional role providers are accustomed to. Despite PHC being publicly funded in developed countries, providers have not considered it their role to report on their performance or the performance of their practice with the goal of improving the PHC sector or larger health care system.

Yet, there are growing demands for performance reporting from many stakeholders including patients [8,9]. Regional case studies of performance reporting [10,11] and evidence from the hospital sector [12] indicate it can influence quality improvement agendas and improve performance. Past work shows that public reporting may improve performance [6,10,13-15], as it has the potential to “improve the quality of care, increase accountability, facilitate public participation in health care,”([14], p.62, 15) impact societal and professional values and direct attention to issues not currently on the policy agenda [16-18]. It may also facilitate collaboration among stakeholders as they set a common agenda [19]. While performance reporting in the hospital sector grows, performance reporting in PHC lags behind.

The most commonly referenced performance information about PHC internationally is from the Commonwealth Fund patient and clinician surveys in industrialized nations [20-26]. The surveys are based on samples of 1000 patients or clinicians per country and show (for the dimensions addressed by the surveys) that PHC performance in Canada is poor compared to other Organization for Economic Co-operation and Development (OECD) countries. In an attempt to improve the quality of information used to report on PHC performance, the international QUALICO-PC (Quality and costs of primary care) study was developed with the overarching goals of: 1) examining the relationship between the strength of the primary care system and the performance of the overall healthcare system ([27] and 2), satisfy the demand for benchmarking performance information and to inform primary care reform through cross national comparisons [27,28]. The purpose of this paper is twofold: 1) to report the methods used to recruit family physicians and their patients across the 10 provinces of Canada to participate in the QUALICO-PC study; and 2) to interpret patterns of recruitment to participate in this study. This work is important in reflecting on what could be done in recruitment and data collection for future large scale pan-Canadian and other cross-country studies.

Methods

Design

The QUALICO-PC study started as a research program funded by the European Union (EU) including 26 member states and five non-EU European countries, Iceland, Macedonia, Norway, Switzerland, and Turkey [27,28]. Outside Europe, Canada, Israel, Australia and New Zealand also participated, funding their own participation. A total of 34 countries participated. QUALICO-PC used a cross-sectional study design to collect self-reported data from family physicians, their practices and 10 patients who were seen by them.

Survey content

A set of four surveys, designed to collect in-depth information regarding primary care activities was collected [27]. The surveys, which are described in detail elsewhere [27]. included concepts important to the delivery and organization of primary care through individual patient, physician, and practice surveys. The practice survey (PRA) collected information on organizational features such as design and delivery of primary care (e.g., financing, regulation, resources), whereas the family physician survey (FPS) collected information on the type of tasks and services (e.g., first contact care, prevention, continuity of care, and integrated service provision) delivered. The patient experiences survey (PES) contained questions aimed at collecting information on their experiences including: coordination, continuity, quality of care, and equity in treatment of primary care. The patient values survey (PVS) asked questions on the importance of access, quality of care (e.g. interpersonal communication) and services delivered in primary care. The surveys and procedures for collecting the data were originally developed and validated by the European team [29]. Minor adjustments were made by the Canadian research team to align with the different health care systems of the provinces, yet remained as close as possible to the validated European surveys.

Eligibility of participants

All 10 provinces in Canada participated in QUALICO-PC; Canada’s provinces vary in geographic size and population, with Québec, Ontario, and British Columbia being the largest provinces in terms of land mass and population. Prince Edward Island is Canada’s smallest province in terms of land mass and population. Two of Canada’s smaller provinces, New Brunswick and Prince Edward Island, combined their recruitment and data collection efforts. Physicians who were working with a family/general practice (e.g., not specializing in a narrow set of conditions or treatments) were eligible to participate. We maximized recruitment of the variety of practices where family physicians work by having only one physician per practice eligible to participate. A "practice" was defined as one or more physicians that share one of revenue, staff or patients. Only patients of participating family physicians were eligible to take part in the study. Patients had to be 18 years of age or older, speak and read English or French, and not have cognitive impairment to participate.

Sources and methods of participant selection

Each provincial team followed the same data collection method but recruitment methods varied slightly from province to province. For example, in Alberta, a notice of the study was posted on both the Alberta Medical Association and Alberta College of Family Physicians websites, and all related survey material was posted on the Health Quality Council of Alberta site for physician reference.

All family physicians were recruited in collaboration with organizations that had lists of practicing physicians such as the provincial chapters of the Canadian College of Family Physicians. All provinces, except Québec and Manitoba, used a census approach where all family physicians on these organizations’ lists were recruited to be study participants. Some provinces (Manitoba, New Brunswick and Prince Edward Island) worked with organizations who could recruit from their membership, while other provinces (Québec) received permission to use a list of primary care physicians registered to the Department of Health or the Fédération des médecins omnipraticiens du Québec. The research lead and provincial chapters of the Canadian College of Family Physicians or other authorities (e.g., Departments of Family Medicine) gave their support in jointly inviting family physicians to participate in the Canadian QUALICO-PC study through mailed or emailed letters. Interested physicians registered either online or by fax. The Québec recruitment differed in that physicians were called and could register over the telephone if they did not respond by mail or email. Additionally, Québec specifically tried to recruit a stratified random sample of physicians by geographic area and, in some cases, allowed participation by more than one family physician from each practice to accommodate geographic areas with a lower number of practices. In Manitoba recruitment was weighted so that 67% of the family physicians were in Winnipeg; the rest were randomized from non-Winnipeg regional health authorities to try and ensure equitable representation.

Procedures

The Canadian Primary Health Care Research and Innovation Network http://www.cphcrin-rcrissp.ca/ coordinated efforts across the 10 provinces [30]. They facilitated the provincial research teams in meeting regularly throughout the recruitment phase. This pan-Canadian working group was responsible for planning and coordinating data collection, data sharing, and analyses of the Canadian data [31]. An underlying principle during the recruitment phase of this working group was to minimize potential bias in recruitment by creating a supportive learning environment through sharing ethics applications and recruitment materials and mutual problem solving. As much as possible data collection followed a standard protocol and was led by one of the research team members from that province.

Family physicians and patients were required to provide informed consent. In some provinces, personal health numbers (PHNs) and full date of birth were collected from consenting patients for linkage to administrative databases in order to determine relationships between physician, practice, patient experiences/values, utilization of medical services and health outcomes. Family physicians were compensated $200 CDN as a token of recognition for the disruption to their work.

Data were collected in 2013 and early 2014; Once data collection started in each province, it lasted up to 4 months except in Québec where data collection lasted for 9 months. Physicians who registered were contacted by a research assistant by telephone or email to confirm eligibility and interest in participation. Each participating physician was couriered a package containing Scantron [32] format surveys, instructions, pens, and return courier packages. In addition to filling out the provider (n = 1) and practice (n = 1) surveys, physicians were asked to choose a day during which the patients to be seen represented their regular patient panel. Patient surveys (n = 9 patient experience; n = 1 patient values) were administered to consecutive consenting patients on this day. If the surveys were not completed in a single day, recruitment continued for up to 3 days. Practices couriered back their completed surveys. Practices that registered (online, telephone, fax, or email) were contacted up to a maximum of 21 times to encourage completion and return of their surveys. It is important to note in cases where there were a higher number of contacts, this was sometimes due to the research team and family physician office leaving messages for each other, the office asking the research team to call back, or that the family physician was on holidays.

De-identified survey data were scanned into a data file by the provincial research team or couriered to Canmark Technologies Ltd. [33], a third party purveyor located in Toronto, Ontario. The provincial data files were subsequently merged into a national dataset for cross-jurisdictional comparisons. Any hard copies of the surveys originally couriered to Canmark were couriered back to the provincial sites to be stored in a secure location. Separately, and only within each provincial research team, the personal health numbers were entered along with the study ID number into a secure encrypted file. All procedures were approved by behavioural research ethics boards (BREBs) located at the institution where each provincial lead investigator was affiliated (BREB number for lead author’s institution: H12-03300). See Table 1 for the names of the research ethics boards.
Table 1

Research Ethics Boards

Province

Institution

Ethics board

British Columbia

University of British Columbia

Behavioural Research Ethics Board

Alberta

Health Quality Council of Alberta

Community Research Ethics Board of Alberta

Saskatchewan

Health Quality Council of Saskatchewan

University of Sakatchewan Behavioural Research Ethics Board

Manitoba

University of Manitoba

Health Research Ethics Board

Ontario

University of Toronto

Health Sciences Research Ethics Board

 

University of Ottawa

Health Sciences and Science Research Ethics Board

Québec

University of Sherbrooke

Comité institutionnel d'éthique de la recherche avec les êtres humains

New Brunswick/PEI

Horizon Health Network

Research Ethics Board

Nova Scotia

Dalhousie University

Health Sciences Research Ethics Board

New Foundland & Labrador

Memorial University

Interdisciplinary Committee on Ethics in Human Research

In Canada, we followed the protocol devised by our European counterparts. The target sample for European countries was generally 220 family physicians from 220 different practices per country with a few smaller countries aiming for samples of 75 family physicians from 75 different practices. In order to compare across provinces, our target sample was 220 family physicians from different practices in three of the most populous provinces (Alberta, Ontario and Québec) and 75 family physicians from different practices in all other provinces (with New Brunswick and Prince Edward Island combined as a single sampling unit).

Data management and analysis

Using an excel spreadsheet, each provincial team was asked to keep records on the number of: physician recruitment emails/letters sent, physicians who registered, practices who were sent surveys, and practices who returned the completed set of surveys. Data were also recorded on the number of contact attempts made to practices to encourage their participation. Finally, field notes were recorded by each provincial team to gain an understanding of recruitment challenges and reasons why practices were unable to participate. All recruitment data were managed by one research assistant in British Columbia.

Response rates were calculated as the number of physicians who signed up to participate in the study divided by the total number of invitations sent out. Participation rates were calculated as the number of physicians who were sent a package of surveys divided by the number of physicians who signed up to participate in the study. Cooperation rates were calculated as the number of physicians who returned completed surveys divided by the total number of physicians who received surveys. Field note data, including reasons why practices were unable to participate were aggregated into themes. Credibility and trustworthiness [34] of the data were undertaken by having the provincial teams discuss the resulting themes and also through international [35] and national [36] dialogue.

Results

Invitations to participate were sent to approximately 23,000 family physicians across Canada. The majority of invitations were sent to family physicians in Ontario, followed by British Columbia, and Alberta. The frequency of invitations (including initial) sent ranged from 2–4 across the provinces. The frequency of follow-up phone calls to encourage registered physicians and their practices to complete data collection and return their packages ranged from 1–21. The mean number ranged from 3.3 in Newfoundland and Labrador to 6.53 in Québec.

Across Canada a total of 792 physicians and 8,332 patients from 772 primary care practices in Canada completed the surveys, including 1,160 participants who completed a Patient Values survey and 7,172 participants who completed a Patient Experience survey (Table 2). Patient participants who also consented to link their survey data to administrative data ranged from 57% in New Brunswick/Prince Edward Island to 86% in British Columbia.
Table 2

QUALICO-PC completed surveys by province

Completed surveys

British Columbia

Alberta

Saskat chewan

Manitoba

Ontario

Québec

New Brunswick/PEI

Nova Scotia

New Foundland & Labrador

Total completed surveys

Physician surveys, n

59

116

20

41

184

218

54

59

41

792

Practice surveys, n

58

117

20

24

183

218

53

58

41

772

# Patient experience surveys, n

537

1240

185

353

1698

1798

497

544

320

7,172

# Patient values Surveys, n

90

207

33

48

282

289

69

92

50

1,160

Total surveys by province

744

1680

258

466

2347

2523

673

753

452

 
Four sites across five provinces, Ontario, Québec, New Brunswick/Prince Edward Island, and Nova Scotia (see Figure 1, response rate) met their targeted sample size of practices through the recruitment procedures. Overall, the response rate was very low ranging from 2% (British Columbia) to 21% (Nova Scotia). Even with a longer data collection period, Québec achieved only an 18% response rate. However, the participation rate was high, ranging from 72% (Ontario) to 100% (New Brunswick/Prince Edward Island and Newfoundland & Labrador). The cooperation rate (see Figure 1) shows that although no province successfully collected data from the targeted number of practices, once practices agreed to participate and received the data collection package, the majority of them were able to return completed surveys. The cooperation rate ranged from 57% (Newfoundland and Labrador) to 84% (British Columbia and Québec).
Figure 1

Flow diagram: Pan-Canadian QUALICO-PC Recruitment. Response rates were calculated as the number of physicians who signed up to be part of this study divided by the total number of invitations sent out. Participation rates were calculated as the final number of eligible physicians who were sent a package of surveys divided by the number of physicians who signed up to participate in the study. Cooperation rates were calculated as the number of physicians who returned completed surveys divided by the total number of physicians who received surveys. Abbreviations: British Columbia (BC), Alberta (AB), Saskatchewan (SK), Manitoba (MB), Ontario (ON), Québec (QC), New Brunswick/Prince Edward Island (NB/PEI), Nova Scotia (NS), Newfoundland and Labrador (NL).

Across all provinces except New Brunswick/Prince Edward Island and Newfoundland, there was some attrition that affected the participation rate (n = 162). In confirming eligibility, 65 (40%) of the family physicians were found to be located in the same practice as one of their colleagues who had already been admitted to the study. In some cases, one of the team members was unable to reach the physician who had signed up online (n = 32). In other cases, once data collection procedures were explained by our team, physicians declined to participate because many of their clients did not speak or read English or French well (n = 26), they were too busy (n = 12), or had other reasons (n = 26), such as being away or on maternity leave during the time data were to be collected.

There was also some attrition that affected the cooperation rate (n = 254) for similar reasons. Nineteen (7%) physicians did not meet our eligibility requirements or our team was unable to contact the physician (46%). Other common reasons for attrition in the cooperation rate was that some physicians (21%) reported that administering the survey was not feasible because appropriate approval in working with First Nation patients had not been sought by the practice, or stated that the surveys would take too long to complete.

Half the 792 family physicians were female and just under half (45.3%) were 65 years or older (Table 3). The majority (71.6%) were born in Canada with an overwhelming percentage (90.3%) indicating they were self-employed and working in group practices. Providers worked on average 41 hours per week, though there was wide variability in their reported work hours. Participating practices were from a variety of geographic locations, with just under one-third reporting they delivered services in an inner city. Just under 40% of physicians perceived that the number of elderly patients from their panel was above national average. Over half reported being open after 6 pm with fewer practices having connections to either lab (29.8%) or x-ray (19.3%) facilities.
Table 3

Respondent characteristics

Provider’s characteristics

n = 792

Sex, Female: n (%)

393 (49.6)

Age: n (%)

 

Under 44

19 (2.4)

45-64

408 (52.2)

65+

354 (45.3)

Born in Canada: n (%)

 

Yes

563 (71.6)

Self-employed or salaried: n (%)

 

Self-employed

708* (90.3)

Average hours worked per week: Mean (SD)

40.7 (12.7)

Practice characteristics

 

Solo or group practice: n (%)

 

Group practice

708 (90.3)

Geographic profile: n (%)

 

Inner city

236 (30.1)

Suburbs

136 (17.3)

Small town

153 (19.5)

Mixed urban–rural

107 (13.6)

Rural

152 (19.4)

Patients above national average: n (%)

 

Elderly

305 (38.6)

Disadvantaged

226 (28.6)

Ethnic minority

157 (19.9)

Extended hours: n (%)

 

Open after 6 pm (at least once a week)

432 (54.9)

Open weekends (at least once a month)

303 (38.9)

Access within practice/centre to: n (%)

 

Lab facilities

236 (29.8)

X-ray facilities

152 (19.3)

Patient characteristics

8,332 (7,172 patient experiences + 1,160 patient values)

Sex, Female: n (%)

N = 5,447 (66.7)

Age: n (%)

 

18-30

957 (12.1)

31-44

1,678 (21.2)

45-64

3,320 (41.9)

65+

1,971 (24.9)

Education: n (%)

 

No qualifications, pre-primary, primary, or lower secondary

761 (9.4)

Upper secondary education (grades 10–12)

2,587 (31.9)

Post-secondary education (includes college, undergraduate)

4,765 (58.7)

Presence of chronic condition(s): n (%)

 

Yes

4,547 (55.5)

General health status: n (%)

 

Very good

1,859 (22.6)

Good

4,249 (51.6)

Fair

1,797 (21.8)

Poor

332 (4.0)

Note. All characteristics are self-reported. All categorical data do not add up to 100% due to missing data. *14 of these 708 respondents indicated they are both salaried and self-employed.

The majority of patient respondents were female (66.4%). Many patients were between the ages of 45–64 years (41.9%). Four out of every 10 patient respondents reported not having a post-secondary education. The majority of patients reported their health as very good or good (73.7%) and had a chronic condition (55.6%).

Discussion

Initial response rates in our study varied from province to province but were generally low across Canada. This response rate for the QUALICO-PC arm in Canada was similar to rates found in other countries such as Australia [37] (ranged from below 1% to 14.5%) and New Zealand (12.2%). Indeed, the difficulties obtaining acceptable response rates from family physicians for survey participation is a universal challenge. However, our results suggest that once family physicians agree to participate in a survey, their participation and cooperation rates are high.

There are some lessons learned from in the recruitment of family physicians for multijurisdictional studies. In Canada, where funding was obtained from multiple sources and varied by province, the low payment to participating physicians made it possible for all 10 provinces to participate. Yet, the low response rate is reflective of the minimal amount of resources available for recruitment in this pan-Canadian study. This amount may seem like a high amount for participation in a study; ethics boards would suggest even higher amounts may unnecessarily influence potential participants to ‘voluntarily enter the study’ [38]. From a small business viewpoint, this amount of money could be seen as less than a token of appreciation since it would not cover even half the costs of disruption to their businesses. Offering a larger payment in appreciation of the disruption to their practices might have improved the initial response rate but likely not enough to meet the targeted sample size at the provincial level. Past work has shown that a monetary incentive significantly improves physicians’ response rates [39] yet, little remains known about how much monetary incentive is needed to achieve adequate response rates (e.g., ≥50%) [40] amidst the known trend of decreasing response rates in this population [41,42]. There are likely other factors that also influence family physician participation in studies such as the lack of time in building relationships with the front office staff. A fine balance between resources available and payment for participation needs to be carefully assessed.

A low response rate from family physicians is, in part, due to structural challenges. Without a central source for such data, researchers and others (including provinces) must individually seek out family physicians and their patients. Other sectors of the health care system are required to regularly report on specific information to a provincial and often a pan-Canadian repository that is made available to them for the purpose of quality improvement and others for the purpose of planning health services and research. Examples of these data include the Discharge Abstract Database from acute care and the Resident Assessment Instrument from long term and home care. Not only is does each organization required to report data have specific resources to obtain the necessary data but at the pan-Canadian level, the Canadian Institute for Health Information [43] manages the quality of the data. Currently primary care has no such required reporting requirements or the infrastructure to produce any information using such data.

Another structural challenge in collecting data from primary care is that no province keeps an up-to-date list of practicing family physicians. This is somewhat surprising given the amount of public money spent on physician services by government. What we did find is that the degree to which provinces have accurate family physician lists varies. Some of the smaller provinces update their lists every few years whereas in the larger provinces, accuracy of the lists exists at a health authority or other organizational (e.g. division of family practice in British Columbia) level.

As with any study, this work has limitations. Our recruitment strategy was to contact individual physicians to participate in our study because no list exists to recruit at a practice level. Using the lists of family physicians we could obtain, it is likely that our denominator used for calculating the response rate is larger than who was actually eligible to participate (only one family physician per practice who did not have a specialized practice). Although we had limited resources to carry out recruitment, more highly resourced and similar surveys (e.g. Commonwealth Fund International Health Policy Survey [13], Commonwealth Fund International Health Policy Survey of Sicker Adults [44]) also have poor response rates (albeit higher than ours). Cross country surveys aimed at family physicians may be able to increase its response rates by putting more effort (and resources) into creating situations which could increase the face validity of the study [45]. Finally, given the low response rate within provinces, it is not possible to compare primary care practices at a provincial level. However, useful analyses looking for associations between the quality of primary care and practice and physician characteristics can still be undertaken, similar to what has been done by the Commonwealth fund [13,44,46].

We also note that the recruitment methods varied across provinces. Regardless of trying to recruit using a randomized sample or a census approach, the response rate remains low across all provinces. It is possible that a more involved and longer recruitment approach such as what was used in Québec could increase participation in future practice-based primary care studies. In order to implement this across Canada, more resources such as an up-to-date physician list and full time staff to conduct the study would be needed.

Despite these limitations and overall low response rate, the data collected through the Canadian arm of QUALICO-PC represents the largest dataset on the quality and organization of primary care data in Canada. Through our collective recruitment efforts we have the largest number of cooperating practices, physicians, and patients compared to any other country participating in QUALICO-PC. These data can tell us about important patient experiences and values among those who have access to primary care and how practice and provider characteristics might impact patient outcomes such as activation, the degree to which patients become engaged to manage their own care [47].

Conclusion

Generally there is a need to have better information in the area of primary care. Challenges that need to be addressed in how to obtain better information include increasing response rates from primary care providers and using an appropriate sampling strategy that acknowledges the shift to team-based care and increasingly different models of care (e.g. nurse practitioner practices). Moreover, a more coordinated effort to gather this information using short, valid surveys is needed in Canada but also internationally. Even though most family physicians operate as small businesses, they could be supported to routinely submit data through a collective effort and provincial mandate. The groundwork in setting up pan-Canadian collaboration in primary care has been established through this study. Future work to attain external validity is now needed. Establishing an information structure that routinely collects primary care data will be key to developing evidence-informed policy and delivery of health services.

Abbreviations

QUALICO-PC: 

Quality and Costs of Primary Care

PHC: 

Primary Health Care

OECD: 

Organization for Economic Co-operation and Development

EU: 

European Union

PRA: 

Practice Survey

FPS: 

Family Physician Survey

PES: 

Patient Experiences Survey

PVS: 

Patient Values Survey

PHN: 

Personal Health Number

Declarations

Acknowledgements

We would like to thank the physicians, office staff, and patients who participated in this study. We would also like to acknowledge the support of the research assistants and data analysts Kevin Walker and Sandra Peterson. Financial support was provided by the Canadian Institutes of Health Information and the Canadian Foundation for Healthcare Improvement (CFHI). Provincial research grants were received from the Nova Scotia Health Services Research Foundation, la Commissaire à la santé et au bien-être et la Fédération des médecins omnipraticiens du Québec, and the Health System Performance Research Network funded by the Ontario Ministry of Health and Long Term Care. The views expressed in this manuscript are those of the authors and do not necessarily represent the views of the funders.

Authors’ Affiliations

(1)
School of Nursing, University of British Columbia (UBC)
(2)
Centre for Health Services and Policy Research, University of British Columbia (UBC)
(3)
Department of Family Medicine, University of British Columbia (UBC)
(4)
The Canadian Primary Health Care Research and Innovation Network (CPHCRIN)
(5)
Department of Family Medicine, University of Ottawa
(6)
C.T. Lamont Primary Health Care Research Centre, Élisabeth Bruyère Research Institute
(7)
Institute of Health Policy Management and Evaluation, University of Toronto
(8)
Toronto Rehabilitation Institute
(9)
Institute for Clinical Evaluative Sciences
(10)
Saskatchewan Health Quality Council
(11)
Faculty of Medicine, Dalhousie University
(12)
Family Medicine Teaching Unit, University of New Brunswick
(13)
Université de Sherbrooke, Charles LeMoyne Research Center Hospital
(14)
Faculty of Medicine, Memorial University of Newfoundland
(15)
Health Sciences Centre
(16)
Departments of Community Health Sciences and Family Medicine, University of Manitoba
(17)
Manitoba Centre for Health Policy
(18)
Department of Family Medicine, University of Montreal
(19)
Health Quality Council of Alberta
(20)
Institut national de santé publique du Québec (INSPQ)

References

  1. Veillard J, Huynh T, Ardal S, Kadandale S, Klazinga NS, Brown AD. Making health system performance measurement useful to policy makers: Aligning strategies, measurement and local health system accountability in ontario. Healthc Policy. Canada; 2010;5(3):49.
  2. Hutchison B, Levesque J-F, Strumpf E, Coyle N. Primary health care in Canada: systems in motion. Milbank Q. 2011;89(2):32.View ArticleGoogle Scholar
  3. College of Family Physicians of Canada, Canadian Medical Association, Royal College of Physicians and Surgeons of Canada. 2007 National Physician Survey [Internet]. 2007 [cited 2014 Aug 19]. Available from: http://nationalphysiciansurvey.ca/surveys/2007-survey/
  4. Centers for Disease Control and Prevention. Ambulatory health care data. Survey instruments: NAMCS survey instruments [Internet]. Atlanta, GA: Centers for Disease Control and Prevention; 2011. Available from: http://www.cdc.gov/nchs/ahcd/ahcd_survey_instruments.htm.
  5. Fung CH, Lim Y-W, Mattke S, Damberg C, Shekelle PG. Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med. 2008;148(2):111–23. United States: American College of Physicians.View ArticlePubMedGoogle Scholar
  6. Watson DE. For discussion: a roadmap for population-based information systems to enhance primary healthcare in Canada. Healthc Policy. 2009;5(Spec no):105. Canada.PubMedPubMed CentralGoogle Scholar
  7. Canadian Institute for Health Information. Experiences With Primary Health Care in Canada [Internet]. Ottawa, Ont: Canadian Institute for Health Information; 2009. p. 1–23. Available from: https://secure.cihi.ca/free_products/cse_phc_aib_en.pdf.Google Scholar
  8. Berta W, Barnsley J, Brown A, Murray M. In the eyes of the beholder: population perspectives on performance priorities for primary care in Canada. Healthc Policy. 2008;4(2):86–100.PubMedPubMed CentralGoogle Scholar
  9. Shortell SM, Casalino LP. Health care reform requires accountable care systems. J Am Med Assoc. 2008;300(1):95–7. United States: AMER MEDICAL ASSOC.View ArticleGoogle Scholar
  10. Smith MA, Wright A, Queram C, Lamb GC. Public reporting helped drive quality improvement in outpatient diabetes care among Wisconsin physician groups. Health Aff. United States: PROJECT HOPE; 2012;31(3):570–7.
  11. Young GJ. Multistakeholder regional collaboratives have been key drivers of public reporting, but now face challenges. Health Aff. United States: PROJECT HOPE; 2012;31(3):578–84.
  12. Tu J V, Donovan LR, Lee DS, Wang JT, Austin PC, Alter DA, et al. Effectiveness of Public Report Cards for Improving the Quality of Cardiac Care: The EFFECT Study: A Randomized Trial. J Am Med Assoc. United States: AMER MEDICAL ASSOC; 2009;302(21):2330–7.
  13. The Commonwealth Fund. 2011 Commonwealth Fund International Health Policy Survey [Internet]. New York, NY: The Commonwealth Fund; 2011. Available from: http://www.commonwealthfund.org/Surveys/2011/Nov/2011-International-Survey.aspx.Google Scholar
  14. Faber M, Bosch M, Wollersheim H, Leatherman S, Grol R. Public reporting in health care: how do consumers use quality-of-care information? A systematic review. Med Care. United States: Lippincott Williams & Wilkins; 2009;47(1):1–8.
  15. Hibbard JH, Greene J, Sofaer S, Firminger K, Hirsh J. An experiment shows that a well-designed report on costs and quality can help consumers choose high-value health care. Health Aff. United States: PROJECT HOPE; 2012;31(3):560–8.
  16. Powell AE, Davies HT, Thomson RG. Using routine comparative data to assess the quality of health care: understanding and avoiding common pitfalls. Qual Saf Health Care. 2003;12(2):122–8.View ArticlePubMedPubMed CentralGoogle Scholar
  17. Oxman AD, Lavis JN, Lewin S, Fretheim A. SUPPORT tools for evidence-informed health policymaking (STP) 1: what is evidence-informed policymaking? Heal Res Policy Syst. 2009;7 Suppl 1:S1.View ArticleGoogle Scholar
  18. Oxman AD, Lewin S, Lavis JN, Fretheim A. SUPPORT tools for evidence-informed health policymaking (STP) 15: engaging the public in evidence-informed policymaking. Heal Res Policy Syst. 2009;7 Suppl 1:S15.View ArticleGoogle Scholar
  19. Van Walraven C, Dhalla IA, Bell C, Etchells E, Stiell IG, Zarnke K, et al. Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community. Can Med Assoc J. 2010;182(6):551–7.View ArticleGoogle Scholar
  20. Blendon RJ, Schoen C, DesRoches C, Osborn R, Zapert K. Common concerns amid diverse systems: health care experiences in five countries. Health Aff. 2003;22(3):106–21.View ArticleGoogle Scholar
  21. Blendon RJ, Schoen C, Donelan K, Osborn R, DesRoches CM, Scoles K, et al. Physicians’ views on quality of care: a five-country comparison. Health Aff. 2001;20(3):233–43.View ArticleGoogle Scholar
  22. Schoen C, Davis K, DesRoches C, Donelan K, Blendon R, Strumpf E. Equity in health care across five nations: summary findings from an international health policy survey. Washington, DC: Commonwealth Fund; 2000. p. 1–7.Google Scholar
  23. Schoen C, Osborn R, Huynh PT, Doty M, Zapert K, Peugh J, et al. Taking the pulse of health care systems: Experiences of patients with health problems in six countries. Health Aff. 2005;Suppl Web(W5):509–25.
  24. Schoen C, Osborn R, Trang Huynh P, Doty M, Davis K, Zapert K, et al. Primary care and health system performance: Adults’ experiences in five countries. Health Aff. 2004;Suppl Web(W4):487–503
  25. Schoen C, Osborn R, Squires D, Doty M, Rasmussen P, Pierson R, et al. A survey of primary care doctors in ten countries shows progress in use of health information technology, less in other areas. Health Aff. 2012;31(12):2805–16.View ArticleGoogle Scholar
  26. Davis K, Stremikis K, Schoen C, Squires D. Mirror, Mirror on the Wall, 2014 Update: How the U.S. Health Care System Compares Internationally [Internet]. Available from: http://www.commonwealthfund.org/publications/fund-reports/2014/jun/mirror-mirror
  27. Schäfer WLA, Boerma WGW, Kringos DS, De Maeseneer J, Gress S, Heinemann S, et al. QUALICOPC, a multi-country study evaluating quality, costs and equity in primary care. BMC Fam Pract. 2011;12(1):115.View ArticlePubMedPubMed CentralGoogle Scholar
  28. Laberge M, Pang J, Walker K, Wong S, Hogg W, Wodchis P. QUALICOPC (Quality and Costs of Primary Care) Canada – A focus on the aspects of primary care most highly rated by current patients of primary care practices. 2014.Google Scholar
  29. Schäfer WLA, Boerma WGW, Kringos DS, De Ryck E, Greß S, Heinemann S, et al. Measures of quality, costs and equity in primary health care instruments developed to analyse and compare primary care in 35 countries. Qual Prim Care. 2013;21(2):67–79.PubMedGoogle Scholar
  30. Hogg W, Dyke E. Improving measurement of primary care system performance. Can Fam Physician. Mississauga: Coll Family Physicians Canada; 2011;57(7):758–60.
  31. Hogg W, Wodchis W, Katz A, Wong ST, Cullen R, Yelland G. Bringing Canada together: An effective organizational structure for multi-jurisdictional health services research projects. Can Family Phys. In press.
  32. Scantron. Scantron [Internet]. 2012 [cited 2014 Sep 23]. Available from: http://www.scantron.com/.
  33. Canmark Technologies Ltd. Canmark Technologies Ltd. [Internet]. 2014 [cited 2014 Aug 19]. Available from: http://www.canmarktech.com/.
  34. Thorne S. Interpretive Description. Walnut Creek, CA: Left Coast Press; 2008.Google Scholar
  35. Wong ST, Wodchis W, Hogg W, Katz A, Cooke T, Lahtinen M, et al. QUALICO-PC in Canada-Recruitment, Responses, and Initial Cross Country Comparison With Australia and New Zealand. New Zealand: Health Services and Policy Research Conference; 2013. Available from: http://www.hsraanz.org/Home.aspx.Google Scholar
  36. Wong ST, Katz A, Wodchis W, Teare G, Burge F, Ashton T, et al. QUALICO-PC in Canada, Australia, and New Zealand: Recruitment, Responses, and Initial Cross Country Comparison With Australia and New Zealand. Toronto, ON: Canadian Association for Health Services and Policy Research; 2014.Google Scholar
  37. Parkinson A, Jorm L, Douglas KA, Gee A, Sargent GM, Lujic S, et al. Recruiting general practitioners for surveys: reflections on the difficulties and some lessons learned. Aust J Prim Health. http://dx.doi.org/10.1071/PY13129.
  38. UBC Behavioural Research Ethics Office. Behavioural Guidance Notes - Reimbursements & Payments [Internet]. 2014 [cited 2014 Sep 23]. Available from: http://research.ubc.ca/ore/behavioural-guidance-notes-reimbursements-payments.
  39. VanGeest JB, Johnson TP, Welch VL. Methodologies for improving response rates in surveys of physicians: a systematic review. Eval Health Prof [Internet]. 2007 Dec [cited 2014 Dec 24];30(4):303–21. Available from: http://www.ncbi.nlm.nih.gov/pubmed/17986667.
  40. James KM, Ziegenfuss JY, Tilburt JC, Harris AM, Beebe TJ. Getting physicians to respond: the impact of incentive type and timing on physician survey response rates. Health Serv Res [Internet]. 2011 Feb [cited 2014 Dec 24];46(1 Pt 1):232–42. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3034272&tool=pmcentrez&rendertype=abstract.
  41. Changes in telephone survey nonresponse over the past quarter century - WebSM [Internet]. [cited 2014 Dec 24]. Available from: http://www.websm.org/db/12/15984/Web Survey Bibliography/Changes_in_telephone_survey_nonresponse_over_the_past_quarter_century/.
  42. Berk ML, Schur CL, Feldman J. Twenty-five years of health surveys: does more data mean better data? Health Aff (Millwood) [Internet]. [cited 2014 Dec 24];26(6):1599–611. Available from: http://www.ncbi.nlm.nih.gov/pubmed/17978380.
  43. Canadian Institute for Health Information. Canadian Institute for Health Information [Internet]. 2014 [cited 2014 Aug 19]. Available from: http://www.cihi.ca/CIHI-ext-portal/internet/EN/Home/home/cihi000001.
  44. The Commonwealth Fund. 2008 Commonwealth Fund International Health Policy Survey of Sicker Adults [Internet]. New York, NY: The Commonwealth Fund; 2009. Available from: http://www.commonwealthfund.org/Surveys/2008/2008-Commonwealth-Fund-International-Health-Policy-Survey-of-Sicker-Adults.aspx.Google Scholar
  45. Van der Heide A, Deliens L, Faisst K, Nilstun T, Norup M, Paci E, et al. End-of-life decision-making in six european countries: descriptive study. Lancet. 2003;362(9381):345–50.View ArticlePubMedGoogle Scholar
  46. The Commonwealth Fund. Surveys [Internet]. New York, NY: The Commonwealth Fund; Available from: http://www.commonwealthfund.org/Surveys.aspx.
  47. Hibbard JH, Greene J. What the evidence shows about patient activation: better health outcomes and care experiences; fewer data on costs. Health Aff. 2013;32(2):207–14.View ArticleGoogle Scholar

Copyright

© Wong et al.; licensee BioMed Central. 2015

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.