- Research article
- Open Access
Patient and family views of team functioning in primary healthcare teams with nurse practitioners: a survey of patient-reported experience and outcomes
BMC Family Practice volume 22, Article number: 76 (2021)
Nurse practitioners (NPs) have been added to primary healthcare teams to improve access to care. Team processes, including communication and decision-making, explicate how patients and families view team functioning. Yet, important gaps exist in our understanding of patient-reported experience and outcomes at the level of the healthcare team. We aimed to examine the influence of individual, team, and organizational characteristics, and role clarity on outcomes of care mediated by team processes in primary healthcare teams that include NPs.
A cross-sectional survey across six sites representing practices with NPs in Québec, Canada, was conducted between March 2018 and April 2019 as part of a multiple-case study. Patients and families (n = 485; response rate: 53%) completed a validated questionnaire, which included a patient-reported experience measure (PREM) and a patient-reported outcome measure (PROM) of team functioning (Cronbach alpha: 0.771 (PROM) to 0.877 (PREM)). We performed logistic regression and mediation analyses to examine relationships between the individual, team, and organizational characteristics, role clarity, and outcomes of care mediated by team processes.
Patients and families expressed positive perceptions of team functioning (mean 4.97/6 [SD 0.68]) and outcomes of care (5.08/6 [0.74]). Also, high team processes (adjusted odds ratio [AOR] 14.92 [95% CI 8.11 to 27.44]) was a significant predictor of high outcomes of care. Role clarity (indirect effect coefficient ab = 6.48 [95% CI 3.79 to 9.56]), living in an urban area (-1.32 [-2.59 to -0.13]), patient as respondent (-1.43 [-2.80 to -0.14]), and income (1.73 [0.14 to 3.45]) were significant predictors of outcomes of care mediated by team processes.
This study provides key insights on how primary healthcare teams with NPs contribute to team functioning, using a validated instrument consistent with a conceptual framework. Results highlight that high role clarity, living in a non urban area, family as respondent, and adequate income were significant predictors of high outcomes of care mediated by high team processes. Additional research is needed to compare teams with and without NPs in different settings, to further explicate the relationships identified in our study.
There is worldwide interest in measuring patient experience and patient-reported outcomes . Currently, patient-reported experience measures (PREMs) report on what occurs and how events unfold during a healthcare visit from the patient’s perspective , whereas patient-reported outcome measures (PROMs) examine the impact of health conditions and the effectiveness of care from the patient’s perspective. Systems measuring PROMs such as PROMIS© have grown phenomenally, with over 300 validated instruments currently in their database . However, in their systematic review of PREMs, Bull et al.  found that these instruments remained focussed on episodic care in hospitals and interactions at the patient-provider level specific to a condition or a setting. No measures were identified at the healthcare team level. In another systematic review of 321 PROMs in primary care, Murphy et al.  found limited consensus on appropriate PROMs to use to capture the overall outcome of a primary care visit. Thus, to better understand patient experience and outcomes in primary healthcare, it is essential to capture team-level processes.
Team processes explicate patient and family perceptions about team effectiveness (PTE) and how teams function, and include decision-making, clear communication, care coordination, cohesion, problem-solving, and a focus on the needs of patients and families . Furthermore, processes explain how situations unfold over time in response to events in the surrounding context . Researchers have likened team processes to a black box because it is difficult to capture the jointly created rules, complex interactions, and relationships that occur between individuals . In 2014, Strasser et al.  noted that measures of team functioning sufficiently sensitive to identify changes in patient outcomes were only just emerging . Such measurement of team functioning from the patient and family perspective has been a critical challenge because of the human and financial resources required and the lack of validated instruments available [4, 7].
Evaluating how teams function is important because poor team functioning can lead to patient harm . In a recent report for the Organisation for Economic Co-operation and Development, Auraanen et al.  found that lapses in primary care are common and represent approximately half of the global burden of patient harm. These authors argued that integrated information systems, patient involvement in measurement, and increased teamwork were essential conditions to improve safety in primary care . Although performance of primary healthcare teams and patient experience have been examined using system-level indicators, including access to care, wait times, and rates of cancer screening , as a measure of performance, individual assessments must be aggregated to the team, organizational or system level to then be able to track results locally, nationally, and internationally .
Given the growing pressures to improve team performance, different professionals have been added to primary healthcare teams to increase access to safe patient care . In primary healthcare, Hofhuis et al.  found that team performance was influenced by how frequently team members interacted with each other. Nurses can make important contributions to team member interaction and access to care as they make up the largest portion of the global healthcare workforce . Making the best use of nurse roles is critical to support efforts to reach the World Health Organization’s  Sustainable Development Goals centered on universal access to healthcare. Nurse practitioners (NPs), recognized as advanced practice nurses, have developed globally to respond to the inadequate supply of primary care providers . However, NP role implementation has been unequal internationally . Over half of the studies included in a scoping review of the international literature on barriers and facilitators of NP role implementation were conducted in North America . In the United States, where NPs were implemented in the 1960s , their numbers have increased exponentially and more than doubled between 2010 and 2017, reaching a total number of 190,000 . Other countries and jurisdictions within countries have implemented NPs in primary care at a much slower rate .
NPs are registered nurses trained at the graduate level who have acquired in-depth clinical knowledge, skills, and decision-making autonomy for expanded practice . Evidence from systematic reviews of effectiveness and cost-effectiveness have highlighted that NPs in primary care improve access to and the quality of care for patients and families, particularly for vulnerable patients or those living in underserviced areas . Furthermore, NPs reduce healthcare costs when they work to their optimal scope of practice . In long-term care, primary healthcare NPs reduce polypharmacy rates, the number of unnecessary medications, admissions to acute care, transfers to the emergency department, and costs [19, 20]. Nevertheless, NP consultation times may be longer than physician consultation times, particularly when scope of practice regulations are too restrictive [17, 18]. Even so, researchers  have argued that NPs’ holistic decision-making and supportive communication may contribute to longer consultation times and improve patient experience of care. In addition to holistic decision-making and supportive communication, teamwork and patient engagement are essential in the delivery of safe and effective care .
Although patients and families consistently report that they are satisfied with care provided by primary healthcare teams with NPs , there are important gaps in our understanding of how these teams function because research so far has focussed on individual-level interactions with providers rather than on micro-level processes in healthcare teams [4, 23]. There is a lack of research measuring team processes rather than outcomes of teamwork , and little is known about the relationship between individual, team and organizational characteristics, and team processes on outcomes . Furthermore, to understand patient experience and how teams with NPs function in the context of patient-centered care , it is essential to include patients and families. This paper describes the views of patients and families of how their healthcare teams with NPs function. More specifically, in primary healthcare teams with NPs, we aim to examine the influence of individual, team and organizational characteristics, and role clarity on outcomes of care (PROM) mediated by team processes (PREM). Because NP roles in primary care were emerging in Québec at the time of the study, we aimed to describe how teams with NPs influence team functioning.
We adapted the conceptual framework describing perceptions of team effectiveness (PTE) in healthcare teams with NPs by Kilpatrick et al. (2013)  (See Fig. 1). The framework was based on the structures-processes-outcomes model. The structural dimension includes factors at the patient, team and organizational levels. Team processes include trust and PTE (e.g., communication, care coordination) with a focus on the needs of patients and families (PREM). Role clarity, defined as understanding one’s role and the role of others in the team, has been identified as a critical factor to improve team functioning [27, 28]. The lack of role clarity can jeopardize teamwork. Outcomes of care are defined as the results of care provided by the healthcare team (PROM). The adapted framework illustrates direct relationships between the patient, team and organizational characteristics, role clarity, and outcomes of care and indirect effects mediated by team processes.
We conducted a cross-sectional survey across six sites as part of a mixed methods multiple-case study in Québec, Canada. Québec is the second largest province in Canada in terms of population. NP roles in primary care were implemented in 2007  in Québec. NPs in primary care practice in offices, home care, and long-term care . We purposefully selected sites to represent practices with NPs in Québec across a range of characteristics , including rural and urban settings, type of practice, and team size.
Data collection and recruitment
Data were collected as part of a larger study examining how teams with NPs optimize roles of members of the healthcare team, including patients and families (2016 to 2019). The survey was administered between March 2018 and April 2019, following research ethics board approval. The minimum sample size of 400 was required to obtain significant results from a logistic regression with 6 significant independent variables (IVs) in the final model .
Each site included between one to three NPs, and each NP worked with one healthcare team in the organization. Respondents in four of the six sites identified their location as rural. Patients needed to be cared for by a healthcare team with an NP. Family included persons significant to the patient who were involved in their care. Participants were randomly selected from the NPs’ caseload using a computer-generated random sequence. We recruited 200 patients and families where the NP’s caseload included at least 200 patients. The three NPs working in home care and long-term care had a caseload of less than 200 patients, and all the patients in their caseload were recruited. We used Dillman’s method with four mailing cycles . Participants could complete the questionnaire on paper or on-line. A self-addressed stamped return envelope was included. A total of 982 questionnaires were mailed out, 75 were returned undelivered, and 485 questionnaires were completed (response rate: 53%). Questionnaires completed on-line represented 28% (n = 134) of responses. A small voucher incentive ($5-gift card at a national coffee house) was included with the first mailing as a thank you gesture.
The Patient-PTE Questionnaire is a 43-item self-report instrument that measures how teams function. It is available in English and in French and takes approximately 10 min to complete . Responses range from 1 (strongly disagree) to 6 (strongly agree) on a scale with no neutral point. Higher scores indicate improved PTE and team functioning. One question (Q. 14) specifically examines beliefs about team effectiveness (BE). One question (Q. 8) describes the NP role and asks respondents if their team includes an NP. One open-ended question (Q. 43) gathers additional comments. Psychometric testing (n = 355) of the instrument included construct validity using known groups with additional testing in teams with physiotherapists in extended roles . Differences were identified by clinical specialty, patient education, length of follow-up, and reason of the health visit. Reliability was assessed using Cronbach α with values ranging from 0.76 to 0.94. Rho coefficients (rs) for processes included in the Team Processes subscale ranged from 0.55 to 0.79 (p < 0.001) indicating that these processes were highly correlated with BE. Responsiveness was assessed and differences were noted in low and high functioning teams (p < 0.001). Individual results can be aggregated to the team, organization, or system level. The questionnaire measures teamwork by including key dimensions of team functioning consistent with our conceptual framework based on the structures-processes-outcomes model and by situating patients and families as members of the healthcare team. Using the current sample, reliability was assessed for the 15-item Team Processes (PREM) and Outcomes of Care (PROM) subscales using Cronbach α. Values ranged from 0.771 (PROM) to 0.877 (PREM).
Operational description of variables
All the variables needed for the analysis were included in the questionnaire and consistent with our conceptual framework. Given that age is not specific to team care in primary care, we included age as a descriptive variable rather than a variable in the predictive models.
The Outcomes of Care subscale represents a PROM of team functioning. Six items measure timely care, promptly dealing with potential or actual complications, medication, access to information, answers to questions about care, and knowledge and skill of the healthcare team.
Individual variables included patient and family characteristics with type of respondent (patient, family), age (years), sex (male/female), perceived health status (low: poor, fair; high: good, very good, excellent), marital status (living with a partner: yes/no), education (completed high school: yes/no), employment status (not employed/employed), financial status (perceived income adequate/ inadequate), length of follow-up (less than 24 months/ greater than 24 months), routine health visit (yes/no), country of birth (Canada: yes/Other: no).
Team characteristics included healthcare team size with small (less than 5 team members), medium (5 to 10 team members), large (more than 10 team members). Team size was dichotomized as small/ medium-large.
Organizational characteristics were measured using two items. Healthcare setting included 15 possible response options in the questionnaire. Survey responses were grouped into primary care and home care/long-term care. Location including urban, rural, remote and rurban, was dichotomized as urban/non urban.
Role clarity was measured using two items (i.e., how well roles are defined in teams, how work is divided among team members).
Perception of Team Effectiveness includes 15 items: decision-making (information is shared, ideas are valued), communication (plan of care is communicated, health record is up-to-date, flow of information), care coordination (next steps in care plan, care adjusted to change in patient’s condition, care well organized), cohesion (working together), problem-solving (differences of opinion are respected), patient and family focus (patient/family has a role in the team, contribution is valued, working with family), trust (trust in team), and belief about team effectiveness (team is effective).
Logistic regression and mediation analyses were used to examine the influence of the IVs (i.e., individual, team, and organizational characteristics) and role clarity on outcomes of care mediated by team processes. We then examined the influence of team processes on outcomes of care, the dependent variable (DV) [31, 32]. Analyses were completed with IBM Statistical Package for Social Sciences version 27 (2020)  and STATA version 13 (2013) . The 5% threshold was used to determine significance. Negatively worded items were reverse-coded prior to analysis. As described above, dichotomous variables were created for all the independent variables in the model. Numeric responses from 1 to 4 were recoded as low, and responses 5 and 6 were recoded as high. Descriptive statistics (number and proportion) were generated. Bi-variate analyses were conducted to examine significant relationships between the independent variables, team processes, and outcomes of care. Multicolinearity between the individual characteristics was examined . Perceived health status and education were highly correlated (p < 0.001 with other individual characteristics and were not included in the models. Unadjusted odds ratios (UORs) were estimated. Finally, we performed three multivariate binary logistic regression and mediation analyses to identify in primary healthcare teams with NPs the (i) IVs (i.e., individual, team and organizational characteristics, and role clarity) that influence team processes (Model 1); (ii) IVs that influence outcomes of care (Model 2); (iii) influence of the IVs and team processes on outcomes of care (Model 3). Coefficients for the indirect effects were calculated using unstandardized variables to identify the potential mediator effect of team processes . The indirect effect (mediation) of team processes was considered when the effect of X on Y was weaker in Model 3 than in Model 2. This was considered complete indirect effect if X no longer had an effect on Y when the effect of the mediator was controlled (Model 3). Sensitivity analyses were performed to examine the indirect effect using Sobel test, Aroian test, and Goodman test [32, 35, 36]. As proposed by Rijnhart et al. (2019) , we used the following equations:
Where: X represents the characteristics and role clarity (IV); Y represents the outcome (PROM); M represents the mediator (PREM); \(i\) represent the intercepts; aX represents the slope of each independent variable; bM represents the slope of the mediator; cX represents the slope of outcome; and c’X represents the slope of outcome (when the IV and PREM are also a predictor of the PROM); ab represents indirect effect.
The 95% confidence interval (95% CI) of the indirect effect ab was determined by the set of values between the 2.5th percentile and the 97.5th percentile of the distribution of ab obtained by the hierarchical Bayesian method (simulation). Using the computational program developed by Falk and Biesanz (2016)  the confidence interval was calculated.
The variables that were included in the final model were kept because of their identified conceptual significance. We did not impute data. All the patients and families with at least one missing datum were removed during the multivariate analysis. The Stepwise procedure was used for the selection of variables retained in the models (using Likelihood Ratio) . The Hosmer–Lemeshow statistic allowed us to determine the quality of the models .
Patient and public involvement
Our research team's expert patient (NF) was involved in all phases of the study: development and validation of the Patient-PTE questionnaire, study conception and design, acquisition and analysis of study data, and dissemination.
Four hundred and eighty-five respondents completed the survey. Of these, 391 participants (81.0%) had complete datasets that were used for the logistic regression analysis. Sensitivity analysis allowed us to determine that there were no significant differences between the respondents who were included and those who were excluded from the analysis. The descriptive characteristics of our sample and mean values for team processes and outcomes of care are presented in Tables 1, 2, 3. More specifically, respondents were primarily patients (66.7%). Most respondents were followed by teams in primary care (87.6%) with less than five members (57.8%). Individual characteristics included female sex (73.1%), aged 15 to 64 (75.2%), being married/living with a partner (71.9%), completed high school education (88.2%), good to excellent perceived health status (90.1%), being employed (60.9%), adequate perceived income (79.6%), and located in an urban area (55.9%).
Overall, the mean value for team process was 4.97/6 (SD 0.68), with a range of 4.27/6 (SD 1.17) for patient-family focus to 5.47/6 (SD 0.82) for BE. The mean for role clarity was 5.28/6 (SD 0.79) and outcomes of care was 5.08/6 (SD 0.74).
Model 1 examined the independent variables that predict team processes. Role clarity (UOR 10.79 [95% CI 4.76 to 24.47]; adjusted odds ratio [AOR] 11.07 [95% CI 4.45 to 27.55]), living in an urban area (UOR 0.66 [0.46 to 0.96]; AOR 0.62 [0.40 to 0.95]), patient as respondent (UOR 0.71 [0.48 to 1.05]; AOR 0.59 [0.37 to 0.95]) and level of perceived income (UOR 1.80 [1.14 to 2.83]; AOR 1.90 [1.06 to 3.42]) predicted team processes.
Model 2 examined the independent variables that predict outcomes of care. High role clarity (UOR 5.36 [95% CI 2.89 to 9.95]; AOR 7.24 [95% CI 3.52 to 14.89]), sex (UOR 1.36 [0.88 to 2.08]; AOR 1.68 [1.03 to 2.75]) and routine health visit (UOR 3.63 [1.75 to 7.52]; AOR 3.75 [1.73 to 8.14]) were significant predictors of high outcomes of care.
Model 3 examined the influence of team processes on outcomes of care. High team processes (UOR 11.81 [95% CI 7.34 to 19.01]; AOR 14.92 [95% CI 8.11 to 27.44]) was a significant predictor of high outcomes of care. Also, high role clarity (UOR 5.36 [2.89 to 9.95]; AOR 3.02 [1.36 to 6.73]), living in an urban area (UOR 1.32 [0.90 to 1.95]; AOR 2.27 [1.31 to 3.95]), being employed (UOR 1.53 [1.02 to 2.29]; AOR 1.92 [1.10 to 3.36]), routine health visit (UOR 3.63 [1.75 to 7.52]; AOR 4.90 [1.96 to 12.22]) and patient as respondent (UOR 1.13 [0.74 to 1.71]; AOR 1.81 [1.02 to 3.22]) remained significant predictors of high outcomes of care.
Moreover, the indirect effect of role clarity on outcomes of care (ab = 6.48) mediated by team processes was statistically significant (95% CI [3.79 to 9.56]; p < 0.001). Similarly, living in an urban area (ab = -1.32; 95% CI [-2.59 to -0.13]; p < 0.05), patient as respondent (ab = -1.43; 95% CI [-2.80 to -0.14]; p < 0.05), and income (ab = 1.73; 95% CI [0.14 to 3.45]; p < 0.05) were significant predictors of outcomes of care mediated by team processes. The results of the mediator effect of team processes are presented in Fig. 2.
We surveyed patients and families followed by primary healthcare teams with NPs across six sites in the Canadian province of Québec to examine their views of how these teams functioned. Most respondents had completed high school education. They perceived their health status to be good to excellent and their income to be adequate. Further, our research adds that in primary healthcare teams with NPs, team processes have a mediator influence on outcomes of care. Direct effects on outcomes of care were identified for individual characteristics (i.e., sex, employment status, routine health visit), whereas indirect effects were identified for individual characteristics (i.e., respondent, perceived adequacy of income) and organizational characteristics (i.e., location) and role clarity. Our study is the first study to measure PREMs and PROMs at the level of the healthcare team. Our results provide key insights on how primary healthcare teams with NPs contribute to outcomes of care, and highlight the importance of role clarity and the mediator effect of team processes on outcomes in teams with NPs while taking into account individual, team, and organizational characteristics.
Our study makes important empirical and conceptual contributions to the PREM and PROM literature by measuring the influence of role clarity on outcomes of care and the mediator role of team processes. Consistent with previous research [39, 40], characteristics related to the type of respondent, location, being employed, and a routine health visit were identified as significant predictors of perceived outcomes in teams with NPs. Globally, role clarity is a critical factor for success when implementing NP roles in teams [41,42,43,44,45]. The need for role clarity among team members has long been recognized as an imperative to optimize team functioning and outcomes from the provider perspective . With its focus on roles and responsibilities, role clarity’s influence from the patient and family perspective had not yet been explicated in-depth . Given the challenges identified by healthcare providers to ensure role clarity and the strong influence role clarity plays from the patient and family perspective, it is essential to monitor role clarity from the patient and family perspective and take steps to clarify roles among members of the healthcare team. The implementation of evidence-informed strategies to support role clarity and to develop non-technical skills, such as communication, is crucial when issues are identified. Interventions, including debriefing, structured communications, and speaking-up, show promise to improve team functioning . Further, the co-development of an implementation plan with key stakeholders including patients, families, care providers, and decision-makers can support role clarification in teams .
Our findings have broad implications because of the wide-ranging use of teams in healthcare and the increased engagement of patients with multiple team members. As such, our study supports emerging evidence to link team functioning  and patient engagement  in primary care to performance. Moreover, our study provides insights into the team processes that contribute to effective team functioning and identifies team processes that can be improved to enhance team functioning and patient experience, and ultimately support greater patient engagement . Determining what constitutes better team functioning from the patient and family perspectives is essential to improve care and give patients and families a voice in the organization of their healthcare . More specifically, working closely with families to involve them as team members is an important strategy to enhance team processes and outcomes of care.
Furthermore, our study provides evidence of the mediator effect of team processes on outcomes of care, which can be seen to reduce the influence of other variables when included in the final model. Of note, patients and families expressed positive perceptions of team functioning and outcomes when healthcare teams included NPs. Such team processes, including decision-making, communication, care coordination, cohesion, problem-solving, a focus on the needs of patients and families, and trust are key components that patients and families appreciate and that can ultimately ensure a higher level of patient and family engagement. Outcomes, including timely care, dealing promptly with complications, and team knowledge and skill, were positively evaluated by patients and families. Although previous research has shown that healthcare teams that include NPs improve patient satisfaction with care [18, 40], our results explicate the micro-level processes in teams with NPs that contribute to patient experience and shed light on the mediator role played by team processes on outcomes at the team level.
Implications for research and practice
Our study focussed on measuring perceived experience and outcomes of primary healthcare teams with NPs from the patient and family perspective. Based on our findings, we contend that paying close attention to role clarity and the quality of team processes, including communication, problem-solving, and a focus on patient and family needs represents a key strategy to improve patient primary care outcomes because these variables represent key modifiable factors. Similarly, providers in primary care and acute care teams report that teams function well when team members pay close attention to role clarity and team processes [19, 41].
In our study, team size, marital status, length of follow-up, and country of birth were not significant predictors of outcomes of care. Researchers have identified mixed findings when considering these characteristics [40, 49,50,51]. Further research is needed to take into account the mediator effect of team processes on outcomes of care while controlling for these factors. Additional condition-specific research is needed to determine the relationship between team processes, perceived outcomes, and condition-specific care outcomes for patients and families in different settings, including acute care, from the patient, as well as the provider perspective. The effects of these variables on patient and family engagement in their care, and other healthcare domains (i.e., teaching, administration, research) should also be investigated. Because we had identified a conceptual contribution, all the variables were retained in the statistical models, including non-significant effects identified in our sample. Subsequent research will need to further test the relationships in the conceptual framework and use a comparative study design to control more closely for confounding variables. Finally, complementary to our metrics approach to teamwork, a qualitative approach could be adopted to focus on tacit knowledge held by team members that could provide nuanced insights into inter-professional contingencies that also determine team functioning.
Even though our study included a small number of purposefully selected practices in one Canadian province, Québec, we anticipate that our findings are generalizable to teams in primary care across Québec and in other jurisdictions. At the outset, no group comparisons were planned and we did not control for NP characteristics, given that NP implementation in Québec was very recent at the time of data collection. Our response rate of 53% was slightly higher than response rates of 44 to 46% reported by Pieper et al. . To further reduce the risk of bias in the assessment of team processes and outcomes, additional research is needed with teams that are selected at random, teams with and without NPs, teams with NPs who have different characteristics, with patients and families in home care and long-term care, and those of lower socio-economic status and lower educational levels. As previously stated, we decided a priori to not impute data in the analysis phase. Although our sample included 485 questionnaires with 16.7% missing data, it was sufficiently large, with almost 400 completed questionnaires. Furthermore, no question was left unanswered systematically, and the characteristics of patients and families who were excluded from the analysis because of missing data did not differ from those with which the analyses were carried out. Finally, although the use of categorical response options in the questionnaire limits our ability to apply more sophisticated statistical techniques aimed at reducing the percentage of missing data in the analysis , it eases the response burden for participants.
We completed a cross-sectional survey of patients and families in Québec, Canada, under the care of primary healthcare teams with NPs to examine patient and family perceptions of team functioning and outcomes of care. We found that team processes exerted a mediator influence on outcomes of care, and role clarity was the only variable to influence team processes and outcomes of care. Although different patient and organizational characteristics influenced either team processes or outcomes of care, additional research is needed in teams without NPs, selected at random, and in different settings, to further explicate and test the relationships between patient, team and organizational characteristics, role clarity, team processes, and outcomes of care that we identified in our study. Such knowledge will support the creation of high functioning healthcare teams from the perspective of patients, families, healthcare providers, and decision-makers.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author Kelley Kilpatrick on reasonable request.
Adjusted odds ratio
Beliefs about team effectiveness
Patient-reported experience measure
Patient-reported outcome measure
Perceptions of team effectiveness
Unadjusted odds ratio
Bull C, Byrnes J, Hettiarachchi R, Downes M. A systematic review of the validity and reliability of patient-reported experience measures. Health Serv Res. 2019. https://doi.org/10.1111/1475-6773.13187.
Health Measures. Intro to PROMIS- Adult Assessment. PROMIS 2020; http://www.healthmeasures.net/explore-measurement-systems/promis/intro-to-promis. Accessed 18 Dec 2020.
Murphy M, Hollinghurst S, Salisbury C. Identification, description and appraisal of generic PROMs for primary care: A systematic review. BMC Fam Pract. 2018. https://doi.org/10.1186/s12875-018-0722-9.
Kilpatrick K, Tchouaket E, Paquette L, Guillemette C, Jabbour M, Landry V, et al. Measuring patient and family perceptions of processes in healthcare teams: Questionnaire development and psychometric evaluation. BMC Health Serv Res. 2019. https://doi.org/10.1186/s12913-018-3808-0.
Ramaswamy R, Reed J, Livesley N, Boguslavsky V, Garcia-Elorrio E, Sax S, et al. Unpacking the black box of improvement. Int J Qual Health Care. 2018. https://doi.org/10.1093/intqhc/mzy009.
Strasser DC, Burridge AB, Falconer JA, Uomoto JM, Herrin J. Toward spanning the quality chasm: an examination of team functioning measures. Arch Phys Med Rehabil. 2014. https://doi.org/10.1016/j.apmr.2014.06.013.
Makhni EC, Bozic KJ. Team approach: Clinical outcome collection, done practically. JBJS Rev. 2018. https://doi.org/10.2106/JBJS.RVW.17.00174.
Rosen M, Diaz D, Dietz A, Benishek L, Thompson D, Provonost P, et al. Teamwork in healthcare: Key discoveries enabling safer, high-quality care. Am Psychol. 2018. https://doi.org/10.1037/amp0000298.
Auraaen A, Slawomirski L, and Klazinga N. The economics of patient safety in primary and ambulatory care- Flying blind. OECD Health Working Papers(No. 106). 2018; https://www.oecd-ilibrary.org/docserver/baf425ad-en.pdf?expires=1581521805&id=id&accname=guest&checksum=29DD216FCC15CC2C2B370A4CFC4433E4. Accessed 18 Dec 2020.
Haj Ali W, Hutchison B. Establishing a primary care performance measurement framework for Ontario Mise en place d’un cadre pour la mesure du rendement des soins primaires en Ontario. Healthc Policy. 2017;12(3):66–79.
Hofhuis J, Mensen M, Ten Den L, Van Den Berg A, Koopman-Draijer M, Van Tilburg M, et al. Does functional diversity increase effectiveness of community care teams? The moderating role of shared vision, interaction frequency, and team reflexivity. J Appl Soc Psychol. 2018. https://doi.org/10.1111/jasp.12533.
World Health Organization. Optimizing the contribution of the nursing and midwifery workforce to achieve universal health coverage and the Sustainable Development Goals through education, research and practice-Human Resources for Health Observer Series No. 22. 2017; https://www.who.int/hrh/resources/health-observer22/en/. Accessed 18 Dec 2020.
Maier CB, Aiken LH. Task shifting from physicians to nurses in primary care in 39 countries: A cross-country comparative study. Eur J Public Health. 2016. https://doi.org/10.1093/eurpub/ckw098.
Torrens C, Campbell P, Hoskins G, Strachan H, Wells M, Cunningham M, et al. Barriers and facilitators to the implementation of the advanced nurse practitioner role in primary care settings: A scoping review. Int J Nurs Stud. 2020. https://doi.org/10.1016/j.ijnurstu.2019.103443.
Auerbach DI, Buerhaus PI, Staiger DO. Implications of the rapid growth of the nurse practitioner workforce in the US. Health Aff. 2020. https://doi.org/10.1377/hlthaff.2019.00686.
Ordre des infirmières et infirmiers du Québec and Collège des médecins du Québec. Pratique clinique de l'infirmière praticienne spécialisée en soins de première ligne: Lignes directrices-Juillet 2019. 2019; https://www.oiiq.org/documents/20147/237836/2505-lignes-directrices-IPSPL-web.pdf/. Accessed 18 Dec 2020.
Poghosyan L, Liu J, Norful AA. Nurse practitioners as primary care providers with their own patient panels and organizational structures: A cross-sectional study. Int J Nurs Stud. 2017. https://doi.org/10.1016/j.ijnurstu.2017.05.004.
Martin-Misener R, Harbman P, Donald F, Reid K, Kilpatrick K, Carter N, et al. Cost-effectiveness of nurse practitioners in primary and specialised ambulatory care: systematic review. BMJ Open. 2015. https://doi.org/10.1136/bmjopen-2014-007167.
Kilpatrick K, Tchouaket E, Jabbour M, Hains S. A mixed methods quality improvement study to implement nurse practitioner roles and improve care for residents in long-term care facilities. BMC Nurs. 2020. https://doi.org/10.1186/s12912-019-0395-2.
Tchouaket E, Kilpatrick K, Jabbour M. Effectiveness of introducing nurse practitioners in six long-term care facilities in Québec, Canada: A cost-savings analysis. Nurs Outlook. 2020. https://doi.org/10.1016/j.outlook.2020.06.002.
Bentley M, Stirling C, Robinson A, Minstrell M. The nurse practitioner–client therapeutic encounter: An integrative review of interaction in aged and primary care settings. J Adv Nurs. 2016. https://doi.org/10.1111/jan.12929.
Sangaleti C, Schveitzer MC, Peduzzi M, Zoboli E, Soares CB. Experiences and shared meaning of teamwork and interprofessional collaboration among health care professionals in primary health care settings: a systematic review. JBI Database System Rev Implement Rep. 2017. https://doi.org/10.11124/JBISRIR-2016-003016.
Pomey M, Lebel P, Clavel N, Morin É, Morin M, Neault C, et al. Development of patient-inclusive teams: Toward a structured methodology- L’accueil du patient dans l’équipe clinique: vers une méthodologie structurée. Healthc Q. 2018. https://doi.org/10.12927/hcq.2018.25640.
Marriage B, Kinnear J. Assessing team performance - Markers and methods. Trends Anaesth Crit Care. 2016. https://doi.org/10.1016/j.tacc.2016.05.002.
Norful AA, Swords K, Marichal M, Cho H, Poghosyan L. Nurse practitioner-physician comanagement of primary care patients: The promise of a new delivery care model to improve quality of care. Health Care Manage Rev. 2019. https://doi.org/10.1097/HMR.0000000000000161.
Kilpatrick K, Lavoie-Tremblay M, Lamothe L, Ritchie JA, Doran D. Conceptual framework of acute care nurse practitioner role enactment, boundary work, and perceptions of team effectiveness. J Adv Nurs. 2013. https://doi.org/10.1111/j.1365-2648.2012.06046.x.
Ly O, Sibbald SL, Verma JY, Rocker GM. Exploring role clarity in interorganizational spread and scale-up initiatives: the “INSPIRED” COPD collaborative. BMC Health Serv Res. 2018. https://doi.org/10.1186/s12913-018-3474-2.
Suter E, Mallinson S, Misfeldt R, Boakye O, Nasmith L, Wong ST. Advancing team-based primary health care: a comparative analysis of policies in western Canada. BMC Health Serv Res. 2017. https://doi.org/10.1186/s12913-017-2439-1.
Bujang MA, Sa’at N, Sidik T, Joo LC. Sample size guidelines for logistic regression from observational studies with large population: Emphasis on the accuracy between statistics and parameters based on real life clinical data. Malays J Med Sci. 2018. https://doi.org/10.21315/mjms2018.25.4.12.
Dillman DA, Smyth JD, and Christian LM. Internet, phone, mail, and mixed-mode surveys: The tailored design method. 4th ed. New Jersey: Wiley: Hoboken; 2014.
Field A. Discovering statistics using IBM SPSS Statistics. 5th ed. SAGE edge Publications Ltd; 2018.
Rijnhart JJM, Twisk JWR, Eekhout I, Heymans MW. Comparison of logistic-regression based methods for simple mediation analysis with a dichotomous outcome variable. BMC Med Res Methodol. 2019. https://doi.org/10.1186/s12874-018-0654-z.
IBM SPSS Statistics Version 27. 2020. https://www.ibm.com/products/spss-statistics/details. Accessed 18 Dec 2020.
StataCorp LP. Stata User’s Guide Release 13. Statistical Software. 2013. College Station: A Stata Press Publication. https://www.stata.com/manuals13/u.pdf. Accessed 21 Jan 2021
Preacher K and Leonardelli G. Calculation for the Sobel test: An interactive calculation tool for mediation tests. 2010–2020. http://quantpsy.org/sobel/sobel.htm. Accessed 18 Dec 2020.
Mackinnon DP, Cheong J, and Pirlott AG. Statistical mediation analysis. 2008; In Cooper H, Camic PM, Long DL, Panter AT, Rindskopf D, Sher KJ (Eds.), APA handbooks in psychology®. APA handbook of research methods in psychology, Vol. 2. Research designs: Quantitative, qualitative, neuropsychological, and biological (p. 313–31). American Psychological Association https://doi.org/10.1037/13620-018
Falk CF, Biesanz JC. Two cross-platform programs for inferences and interval estimation about indirect effects in mediational models. SAGE Open. 2016. https://doi.org/10.1177/2158244015625445.
Heinze G, Wallisch C, Dunkler D. Variable selection - A review and recommendations for the practicing statistician. Biom J. 2018. https://doi.org/10.1002/bimj.201700067.
Ammi M, Ambrose S, Hogg B, Wong S. The influence of registered nurses and nurse practitioners on patient experience with primary care: Results from the Canadian QUALICO-PC study. Health Policy. 2017. https://doi.org/10.1016/j.healthpol.2017.09.019.
Regragui S, Lacasse A, Gallagher F. La collaboration interprofessionnelle entre les médecins et les infirmières praticiennes spécialisées en soins de première ligne : Une étude qualitative. REFIRI. 2020. https://doi.org/10.1016/j.refiri.2020.100209.
Brault I, Kilpatrick K, D’amour D, Contandriopoulos D, Chouinard V, Dubois CA, et al. Role clarification processes for better integration of nurse practitioners into primary healthcare teams: A multiple-case study. Nurs Res Pract. 2014. https://doi.org/10.1155/2014/170514.
Dlamini CP, Khumalo T, Nkwanyana N, Mathunjwa-Dlamini TR, Macera L, Nsibandze BS, et al. Developing and implementing the family nurse practitioner role in Eswatini: implications for education, practice, and policy. Ann Glob Health. 2020. https://doi.org/10.5334/aogh.2813.
Donald F, Bryant-Lukosius D, Martin-Misener R, Kaasalainen S, Kilpatrick K, Carter N, et al. Clinical nurse specialists and nurse practitioners: title confusion and lack of role clarity. Nurs Leadersh. 2010. https://doi.org/10.12927/cjnl.2010.22276.
Bryant-Lukosius D and Martin-Misener R. ICN Policy brief advanced practice nursing: An essential component of country level human resources for health. 2015; https://www.who.int/workforcealliance/knowledge/resources/ICN_PolicyBrief6AdvancedPracticeNursing.pdf. Accessed 18 Dec 2020.
Wintle M, Newsome P, Livingston P. Implementation of the nurse practitioner role within a Victorian healthcare network: An organisational perspective. Aust J Adv Nurs. 2011;29(1):48–54.
Turner DA, Rehder KJ, Nagler A, Aucoin J, Edwards P, Kuhn C. What’s in a name? Role clarity goes well beyond a simple title. J Patient Exp. 2019. https://doi.org/10.1177/2374373518803615.
Kilpatrick K, Paquette L, Jabbour M, Tchouaket E, Fernandez N, Al Hakim G, et al. Systematic review of the characteristics of brief team interventions to clarify roles and improve functioning in healthcare teams. PLoS ONE. 2020. https://doi.org/10.1371/journal.pone.0234416.
Wu FM, Rubenstein LV, Yoon J. Team functioning as a predictor of patient outcomes in early medical home implementation. Health Care Manage Rev. 2018. https://doi.org/10.1097/HMR.0000000000000196.
Dahrouge S, Muldoon C, Ward N, Hogg W, Russell G, Taylor-Sussex R. Roles of nurse practitioners and family physicians in community health centres. Can Fam Physician. 2014;60:1020–70.
Russell GM, Dahrouge S, Hogg W, Geneau R, Muldoon L, Tuna M. Managing chronic disease in Ontario primary care: the impact of organizational factors. Ann Fam Med. 2009. https://doi.org/10.1370/afm.982.
Muldoon L, Rayner J, Dahrouge S. Patient poverty and workload in primary care: study of prescription drug benefit recipients in community health centres. Can Fam Physician. 2013;59(4):384–90.
Pieper D, Kotte N, Ober P. The effect of a voucher incentive on a survey response rate in the clinical setting: a quasi-randomized controlled trial. BMC Med Res Methodol. 2018. https://doi.org/10.1186/s12874-018-0544-4.
De Leeuw ED. Reducing missing data in surveys: An overview of methods. Qual Quant. 2001. https://doi.org/10.1023/A:1010395805406.
The authors would like to acknowledge the patients and families who responded to the questionnaire.
This study was funded by the Canadian Institutes of Health Research (CIHR) (Project Grant Number PJT-363528; Principal investigator: KK). KK and ET are supported by the Fonds de recherche du Québec-Santé (Research Scholar Junior 2 [KK] and Junior 1 [ET] salary awards). We recognize the support from the McGill Nursing Collaborative for Education and Innovation in Patient and Family Centered Care (Newton Foundation/McGill Faculty of Medicine and Health Sciences) and Graphos/McGill Writing Centre (Professor Donetta Hines) through the Writing for Publication initiative, and Ms R. Atallah for the manuscript preparation.
This work was funded by the Canadian Institutes of Health Research (CIHR Project Grant Number PJT-363528; Principal investigator: KK). The funder played no role in: study design, data collection, analysis and interpretation; development of the manuscript nor the recommendations. The corresponding author had full access to all study data and final responsibility for the decision to submit for publication.
Ethics approval and consent to participate
Approval for the study was obtained from the Centre intégré universitaire de santé et de services sociaux de l’Est-de-l’Île-de-Montréal-Hôpital Maisonneuve-Rosemont (CIUSSS-EMTL-HMR) Research Ethics Committee (REC) on March 8th, 2017 (ref: MP-12–2017-841, CIUSSS-EMTL-203). Since this REC considered the study at low-risk for participants, participants were advised that survey completion was voluntary, and they could withdraw from the study at any time. Completing and submitting the study questionnaire was considered as consent to participate by the CIUSSS-EMTL-HMR REC. Moreover, study information was mailed to participants, including the link to the online questionnaire. In instances where patients were minor children (14-18 years of age), the parents/guardians were contacted to complete the questionnaire. Hence, no recruitment of minor children occurred. However, it was possible to retain minor children’s responses (n=2) in the dataset because minor children aged 14 years or older can consent and participate in low-risk studies in Québec, where the study was conducted (https://www.msss.gouv.qc.ca/professionnels/ethique/ethique-en-sante-et-services-sociaux/recherche-portant-sur-des-personnes-mineures-ou-majeures-inaptes/).
Consent for publication
All authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Kilpatrick, K., Tchouaket, E., Fernandez, N. et al. Patient and family views of team functioning in primary healthcare teams with nurse practitioners: a survey of patient-reported experience and outcomes. BMC Fam Pract 22, 76 (2021). https://doi.org/10.1186/s12875-021-01406-y
- Nurse practitioner
- Patient-reported experience measure
- Patient-reported outcome measure
- Perceptions of team effectiveness
- Team functioning