- Research article
- Open Access
- Open Peer Review
Establishing chronic condition concordance and discordance with diabetes: a Delphi study
BMC Family Practice volume 16, Article number: 42 (2015)
The vast majority of patients with diabetes have multiple chronic conditions, increasing complexity of care; however, clinical practice guidelines, interventions, and public reporting metrics do not adequately address the interaction of these multiple conditions. To advance the understanding of diabetes clinical care in the context of multiple chronic conditions, we must understand how care overlaps, or doesn’t, between diabetes and its co-occurring conditions. This study aimed to determine which chronic conditions are concordant (share care goals with diabetes) and discordant (do not share care goals) with diabetes care, according to primary care provider expert opinion.
Using the Delphi technique, we administered an iterative, two-round survey to 16 practicing primary care providers in an academic practice in the Midwestern USA. The expert panel determined which specific diabetes care goals were also care goals for other chronic conditions (concordant) and which were not (discordant). Our diabetes care goals were those commonly used in quality reporting, and the conditions were 62 ambulatory-relevant condition categories.
Sixteen experts participated and all completed both rounds. Consensus was reached on the first round for 94% of the items. After the second round, 12 conditions were concordant with diabetes care and 50 were discordant. Of the concordant conditions, 6 overlapped in care for 4 of 5 diabetes care goals and 6 overlapped for 3 of 5 diabetes care goals. Thirty-one discordant conditions did not overlap with any of the diabetes care goals, and 19 overlapped with only 1 or 2 goals.
This study significantly adds to the number of conditions for which we have information on concordance and discordance for diabetes care. The results can be used for future studies to assess the impact of concordant and discordant conditions on diabetes care, and may prove useful in developing multimorbidity guidelines and interventions.
Most adults with diabetes have at least one other chronic condition, and almost half have 5 or more other conditions [1-3]. Multimorbiditiy increases with age, but the majority of persons with multimorbidity are middle aged . Patients with more chronic conditions has been shown in some studies to have better or similar care as those with fewer conditions, partially through increased interactions with the health care system [3,5]. Other studies, however, have suggested that the presence of more conditions are associated with increased mortality . The differing impact of multiple chronic conditions on diabetes care could be due to differences in specific comorbidities’ interactions with diabetes care [1-3].
Currently, we are limited in our knowledge of which comorbidities may improve or inhibit optimal diabetes care [7,8]. We need to understand the interaction between chronic conditions in order to provide adequate care for patients with diabetes and multimorbidity . Current diabetes care guidelines directly address the co-management of diabetes and only a limited number of comorbidities, and remain silent or provide broad, nonspecific guidance for other co-occurring conditions [9,10]. When caring for multimorbid patients, the application of single-condition guidelines may lead to the provision of contradictory and potentially harmful care . Additionally, public reporting metrics are derived from clinical care guidelines, and the presence of public reporting metrics can shape clinical care, so it is particularly valuable to understand the influence of multiple chronic conditions on diabetes care in order to have meaningful quality reports [11,12]. Public reporting could be more meaningful if personalized to individual patients’ comorbidities . Further, effective interventions in multiple chronic conditions are limited, with few evidence-based interventions that target patients with a specific, rather than general, combination of conditions, such as diabetes plus specific comorbidities . Understanding the interaction between the care for diabetes and specific comorbidities might help in the development of tailored interventions.
A potentially valuable approach to integrating diabetes care with the care of other chronic conditions is to consider comorbidities as concordant or discordant with diabetes care . In this general framework, conditions that share the same clinical management as diabetes are concordant with diabetes, and the presence of these conditions could cue providers to deliver the same or similar care as required for diabetes management, resulting in better diabetes care [7,15]. Discordant conditions do not share care with diabetes, and therefore would not cue providers to deliver recommended diabetes care, and may distract from diabetes care. Discordant conditions are those with treatments that either directly oppose diabetes care, such as requiring steroids that will increase blood sugar, or that simply do not share care with diabetes, such as using antacids for esophageal reflux. When a discordant condition is present, time limitations, competing demands, and other challenges may cause patients with diabetes to receive lower quality care for diabetes and the discordant condition as compared to having a single condition alone [8,15]. This framework could allow providers to target patients with multiple chronic conditions who are most at-risk for suboptimal diabetes care based on having fewer concordant conditions or more discordant conditions.
Previous investigations of the Piette and Kerr framework have been limited by the lack of a comprehensive list of diabetes concordant and discordant chronic conditions. Without a comprehensive set of concordant and discordant conditions, we cannot fully examine this framework for potential clinical use. Past studies have used a limited number of conditions categorized as concordant or discordant based on an overall impression of similar, or not, management, formed by context experts, researchers, and clinical practice [7,16]. Additionally, these categorizations have considered comorbidities as entirely concordant or discordant with diabetes [7,15,17]. However, diabetes care is complex and encompasses multiple testing and treatment goals . Each comorbidity may be concordant or discordant with diabetes for one care goal and not for another. For instance, glaucoma may be concordant with the annual eye exam goal and discordant with the HbA1c testing goal. If it is found that concordant conditions are related to improved care in diabetes, and discordant conditions are related to worsened care, guidelines and interventions could highlight where synergistic care occurs and list conditions that might make the patients who have them benefit from extra attention to diabetes care. Also, public reporting could be stratified by concordant and discordant conditions to reflect differences in care and provide more personalized reports.
Our study aimed to provide more information for the future research and clinical use of the concordant-discordant framework by (1) increasing the number of conditions that can be characterized as concordant or discordant with diabetes, and (2) examining the number of care goals that are shared between concordant and discordant conditions. We aimed to provide researchers with a much needed tool to examine how comorbid chronic conditions might impact diabetes care.
We used Delphi methodology , a consensus-building technique that has been well-studied and is the basis for the RAND Appropriateness Method . This technique is most effective when there is a lack of or inadequate information about an issue , such as exists in the literature defining chronic conditions as concordant or discordant with diabetes. Compared to committees and meetings, which can be dominated by a single individual, this technique considers all respondents’ opinions through anonymous reporting and feedback . In our Delphi survey, we asked primary care providers (PCPs) to state whether each of 6 diabetes care goals were also care goals for a comprehensive set of outpatient-relevant chronic conditions. Conditions that shared the majority of care goals with diabetes were defined as concordant, and those that did not were defined as discordant. This study was approved by the University of Wisconsin Health Sciences Minimal Risk Institutional Review Board.
Diabetes care goals
We chose 6 diabetes care goals that represent aspects of diabetes management across the pathophysiologic spectrum of diabetes that are necessary for most adult patients with diabetes (type 1 and type 2) and are related to short- and long-term health outcomes in diabetes . The 6 diabetes care goals were: glycemic management, LDL cholesterol management, blood pressure management, kidney function monitoring, annual eye exam, and tobacco cessation counseling. These care goals can be measured in a standardized fashion across populations and health systems and are used for state and national diabetes performance metrics [18,20]. Despite some controversy in their use, there is general agreement that achieving these goals leads to better diabetes outcomes, and strong evidence shows that major complications are reduced if these goals are achieved [18,20].
We built a list of 62 outpatient-relevant chronic condition categories (“conditions”) from a set of chronic condition categories previously used in multimorbidity research [21,22] and based on the AHRQ clinical classification system (CCS) of chronic medical conditions . We further modified this set of condition categories to enhance the representation of cardiovascular, metabolic and mental health conditions by separating out conditions in these categories. Our 62 chronic condition categories encompass 1,412 ICD-9 codes. We counted a patient with multiple chronic conditions (multiple ICD-9 codes) within any single CCS category as having one chronic condition [21,22]. For example, a patient with the codes “malignant hypertension” (401.0) and “hypertensive heart disease” (402.0) is considered to have one condition, hypertension. Our modified chronic condition classification system is available online at no cost at www.hipxchange.org.
Experts for our Delphi survey were recruited by an email sent to local PCPs who care for adult patients with chronic conditions. These PCPs (general internal medicine and family medicine physicians, physician assistants, and nurse practitioners) all practice at clinics affiliated with a large Midwestern academic medical center. We chose practicing PCPs as opposed to other specialists because they offer expertise in the management and care coordination of diabetes, along with a spectrum of multiple other chronic conditions.
The criterion for expert panel participation (practicing PCPs) was developed by all co-authors. Experts were chosen from the local academic health center for their academic clinical reputation and for convenience. The authors who conceptualized this study and designed the survey (E.M., R.G. and M.S.) did not serve as experts for the Delphi survey to avoid biasing results.
We initially contacted 161 academic PCPs by email for interest in participating in the Delphi survey. Sixteen PCPs agreed to participate and all 16 completed both rounds of the survey. The Delphi technique allows for selection of experts and does not require a representative sample of the population . It also does not require a certain sample size, although 10–20 is typically considered sufficient . Twelve PCPs were family medicine physicians, 2 were internal medicine physicians, 1 was an internal medicine physician assistant, and 1 was an internal medicine nurse practitioner. Most had more than 10 years of practice experience.
Delphi survey procedure
We used a web-based survey to reduce undue peer influence on responses  (Additional file 1). The respondents remained anonymous to one another. The survey asked the PCPs to give their opinion on the management of chronic conditions in primary care. We asked providers if each of 6 listed care goals was “indicated” or “not indicated” in the management of each of the 62 listed chronic conditions. Management of a condition was explained as any testing or treatment that the provider would do beyond care for a normal, healthy individual. Providers were instructed to consider each patient as having only the listed chronic condition for each response. Diabetes was not mentioned anywhere on the survey to reduce the chance that respondents would think the hypothetical patient had both the listed chronic condition and diabetes and therefore respond that care was “indicated” for the chronic condition when they meant that it was indicated for the condition in the presence of diabetes. The survey had 372 condition-goal pairs (62 conditions for each of 6 care goals).
We determined diabetes concordance and discordance both on a goal-specific and summary condition-level for each chronic condition.
Goal-specific concordance and discordance
Goal-specific concordance of a chronic condition was defined as provider consensus opinion that a diabetes care goal was indicated for the chronic condition. If the care goal was not indicated, that condition had goal-specific discordance with diabetes. Conditions could be concordant for one diabetes care goal and discordant for another.
Provider consensus opinion for care goal concordance was reached when 60% of respondents agreed that a care goal was indicated for a chronic condition (60% majority opinion threshold). In the Delphi Method, the percentage agreement required to establish consensus is not definitive and typically ranges from 50-80% [25,26], where consensus levels higher than 80% are of unclear benefit . We chose a 60% cut-off because at a higher threshold, over half of respondents would have to change their opinions in subsequent rounds on a specific care goal to move the majority opinion from concordant to discordant. As this is highly unlikely, we concluded that a 60% majority opinion accurately determines goal-specific concordance for a given condition-goal pair. Condition-goal pairs that were not considered concordant by 60% consensus opinion were considered discordant per the concordance and discordance framework, i.e., any condition that is not concordant with diabetes care is considered discordant without needing a separate discordance threshold . The 60% threshold was chosen prior to seeing the survey results (Figure 1.)
After the first round of surveying, condition-goal pairs that did not reach the 60% consensus threshold were re-addressed in a second survey round. We used only 2 rounds of the survey to determine consensus opinion as additional rounds have been shown not to improve outcomes . The second round surveys were individualized, based on each respondent’s unique responses, to include only those condition-goal pairs for which the respondent was not in the majority opinion. The second round was conducted in waves, starting with those respondents who needed to be asked the fewest questions. As items reached consensus through the iterative process, they were dropped from further waves in round 2. This limited the time burden on participants and potential burn-out . Condition-goal pairs that did not reach the 60% consensus threshold for concordance after the second round were defined as discordant.
Overall condition concordance and discordance analysis
We determined each chronic condition’s overall concordance or discordance by assessing whether a majority of care goals were concordant or discordant for each condition. Conditions that were concordant for the majority of care goals were established as having overall concordance with diabetes, and vice versa for discordant conditions.
After the first round of surveys, 339 of the 372 condition-goal pairs were categorized as concordant or discordant. Thirty-three condition-goal pairs did not reach consensus (19 condition-goal pairs were tied and 14 had a slight majority towards concordance) and went to the second round. After the second round of surveying, 9 condition-goal pairs of the 33 remained below the 60% concordance threshold and were declared discordant.
Unsurprisingly, the tobacco cessation counseling goal was unanimously indicated (concordant) for all conditions in the first round. As such, it could not be used to discriminate between conditions based on diabetes concordance, and was excluded from use in determining overall condition concordance. Therefore, overall condition concordance and discordance were established when 3 out of 5 goals were concordant or discordant, respectively.
Overall, 12 conditions were concordant with diabetes and 50 were discordant (see Table 1).The largest clinical group for concordant conditions was cardiovascular, whereas discordant conditions were distributed across multiple clinical groups. Discordant conditions included depression. Six conditions showed goal-specific concordance for all 5 goals except eye exam (Table 2). The other six concordant were concordant for 3 goals each. Thirty-one discordant conditions were discordant on all 5 goals, and 19 were discordant on all but 1 or 2 goals (Table 2).
Our study determined the goal-specific and overall condition concordance and discordance with diabetes of a comprehensive set of chronic conditions using primary care provider expert opinion. Among 62 ambulatory care-relevant chronic conditions, 12 conditions were concordant with diabetes care, and were all diabetes risk factors or complications. The remaining 50 conditions (including depression) were discordant, showing limited overlap with diabetes care goals. Our results show that not all conditions are equal in how they interact with diabetes care, and conditions can be categorized as concordant or discordant overall and specific to diabetes care goals. We also found that not all PCPs’ opinions matched current clinical practice guidelines. This work has implications for future research to improve clinical guidelines, public reports and interventions for patients with diabetes and multiple chronic conditions.
Previous studies on the diabetes concordance-discordance framework have used the study authors’ opinions [27-31] or the nominal group technique  to determine concordance and discordance. In contrast, the Delphi Method mitigates undue influence from other members of the group, as respondents do not know the individual responses of other group members. Our approach employed the professional judgment of PCPs who are experts at managing and coordinating the care of multimorbid patients. As the front-line providers for these complex patients, their opinions on care are most relevant for study, resulting in findings that are highly applicable to clinical practice. A survey of PCPs shows the complex cognitive and clinical realities of caring for multimorbid patients with diabetes . To date, no study has analyzed as large or comprehensive a set of diabetes care goals and chronic conditions [15,27,29-31]. We added to the number of conditions categorized as concordant and discordant, and validated concordance and discordance categorizations done in previous work (Table 3). We compared the overall condition concordance-discordance results to previous literature on the subject . Our results are generally consistent with the limited number of conditions categorized in prior literature.
Clinically, the conditions we found to be concordant comprise vascular, metabolic and renal conditions that share pathophysiology with diabetes and co-occur frequently. We expected these conditions to be concordant based on the conceptual model and previous work. All concordant conditions have goal-specific concordance with diabetes for LDL and blood pressure management, highlighting the importance of cardiovascular risk management for both diabetes and to provide synergistic benefit to common comorbidities.
Most conditions were discordant, even those whose care is considered of special importance in diabetes. For example, depression is a concerning comorbidity in diabetes, as it is associated with worse outcomes and there is a suggestion of a bidirectional relationship for poor control between diabetes and depression . Our study found that depression was viewed as discordant with diabetes, sharing none of the assessed care goals. This suggests that, for a patient with diabetes and depression, diabetes care goals would be addressed without additional cuing from depression (no overlap in care), and that depression care goals could increase the health care workload and lead to missed care. Alternatively, despite the categorization of depression as discordant, patients with depression and diabetes might receive extra attention on diabetes care, as it is fairly well known that depression is associated with worse diabetes outcomes, and providers might put extra emphasis on treating these patients. This was not assessed in our study and should be studied in future work.
Our study also expands the concordance-discordance framework from solely overall condition concordance-discordance to include concordance-discordance of conditions with specific diabetes care goals. Goal-specific concordance and discordance adds potentially valuable clinical detail for use with the concordance-discordance framework. This expanded knowledge can be used in future research to understand the interaction of conditions in more detail than can be done with only overall condition concordance and discordance . Of the 50 discordant conditions, 19 were concordant with diabetes on 1–2 individual diabetes care goals. Although discordant overall, these conditions still have the potential to interact synergistically with diabetes for those care goals with which they are concordant. These conditions might therefore improve diabetes care for these goals while distracting from diabetes care for other goals. Overall condition concordance and discordance remains important to demonstrate and summarize which conditions, as a whole, share care (or not) with diabetes and might be more likely to interact favorably with diabetes care (or not).
Interestingly, there was some discrepancy between PCP perceptions and guideline recommended care, which may reflect lower familiarity with less commonly seen conditions. The discrepancy also highlights an opportunity to correct cognitive models for concordant conditions to improve clinical care for patients whose comorbidities may confer additive risk. For example, PCPs perceived rheumatoid arthritis as discordant with diabetes on the goal level with glycemic, blood pressure, and lipid management. However, rheumatoid arthritis, like diabetes, increases cardiovascular disease risk, and requires attention to blood pressure, lipid control, and glycemic control [35,36]. Our findings align with results of prior work by our group showing that Medicare patients with RA and diabetes actually received fewer A1c tests than diabetes patients without rheumatoid arthritis , fitting with provider perceived discordance despite shared physiologic cardiovascular risk.
The major goal of this study was to provide more information for the future research and subsequent clinical use of the concordant-discordant framework by (1) increasing the number of conditions that can be characterized as concordant or discordant with diabetes, and (2) examining the number of care goals that are shared between concordant and discordant conditions . If future research demonstrates differences in clinical outcomes associated with concordant and discordant conditions, then there are several potential clinical implications for guidelines, interventions and public reporting of this more detailed and thorough application of the concordance-discordance framework for patients with diabetes and multiple chronic conditions. Current guidelines, for the most part, are disease-centric. While recent diabetes guidelines suggest individualized targets for patients with specific chronic conditions, the guidelines still have diabetes as the central condition and discuss diabetes care goals, without incorporating the non-diabetes care goals of most comorbidities, especially discordant comorbidities. One approach to guideline use in multimorbidity is to electronically cross-reference guidelines and create a patient-specific guideline listing all the patient’s conditions and the conditions’ associated care goals . Using goal-specific concordance, this list could be integrated across conditions to list all recommended care goals for that patient, for all conditions, and the conditions the care goals benefit (for example, blood pressure control benefits diabetes and heart failure but not necessarily depression). This approach would also help patients and providers discuss risks and benefits of each care goal across all conditions to see which goal would give the largest health benefit and/or best fits their personal care priorities [9,10]. Interventions to improve care can apply integrated care goal and risk-benefit discussions across all conditions. These interventions can target patients most at risk for suboptimal care based on their concordance and discordant conditions, if future work shows that patients with certain multimorbidity profiles are more at risk than others. Finally, public reports of care quality could be stratified by comorbidity type so that patients are compared to patients more similar to them, giving more meaningful reports. These reports could consider using non-disease specific outcomes, such as reporting on care goals that overlap between conditions, care goals that give the greatest benefit with least risk across conditions, or functional outcomes .
A potential limitation of our study is that our expert panel included a variety of PCPs (e.g., family medicine, internal medicine). While it is possible that the different specialties of the providers could bias our results, we think it is more likely that the diversity of our panelists best represents the range of PCPs and their opinions. Additionally, it is possible that the PCPs who chose to participate in the panel have a special interest in research or quality improvement. If this were the case, we would expect the result to bias towards more conditions having care goals indicated and more concordance. We also chose certain diabetes care goals based on current guidelines and publicly reported quality metrics, but there are other care goals that are relevant to diabetes. We didn’t include in our survey additional diabetes care goals that are used in quality reporting or recommended in diabetes guidelines, including foot care and lifestyle counseling, as achievement of these goals is harder to consistently quantify between providers and health systems. As concordance and discordance in the original framework is defined as sharing management and pathophysiology, or not, this model is inherently provider-centric, based on care processes and treatments that are ordered by the provider and assessed in the clinic. Future work could assess the patient’s perspective on concordance and discordance by assessing patient opinion on overlap in self-care, such as exercise and diet changes. We recognize that more factors than care goal overlap between chronic conditions have an influence care quality and should be considered in the development of future guidelines and interventions. Patient contextual elements must be included such as age and socioeconomic status , since older adults require different care than younger adults  and lower socioeconomic status is a risk factor for an earlier age of onset for chronic conditions . Finally, one goal of our study was to determine care goal overlap. Future work should determine the clinical impact of this overlap, and if concordance and discordance impact care quality differently, especially for different care goals (e.g., HbA1c testing vs. HbA1c control, HbA1c control versus BP control).
Our study shows that PCPs perceive the care of diabetes to overlap with the care of several other chronic conditions, especially chronic conditions that are risk factors for or complications of diabetes. Other chronic conditions, including depression, an important diabetes comorbidity, were not perceived as having overlapping care with diabetes. As our approach differentiates overall condition concordance-discordance and goal-specific concordance-discordance, our results highlight potentially helpful overlaps in care that are not evident with overall condition discordance. This knowledge will be especially useful as we move towards guidelines and interventions that focus on condition interactions in multiple chronic conditions, rather than considering each condition in isolation. Understanding the interactions between conditions at the level of care goals could be a key to identifying patients with diabetes most at-risk for suboptimal care due to their other chronic conditions, and to devising system-level interventions to target and improve care for these patients.
Primary Care Provider
Clinical Classification System
Schneider KM, O’Donnell BE, Dean D. Prevalence of multiple chronic conditions in the United States’ Medicare population. Health Qual Life Outcomes. 2009;7:82.
Laiteerapong N, Huang ES, Chin MH. Prioritization of care in adults with diabetes and comorbidity. Ann N Y Acad Sci. 2011;1243:69–87.
Bae S, Rosenthal MB. Patients with multiple chronic conditions do not receive lower quality of preventive care. J Gen Intern Med. 2008;23(12):1933–9.
Barnett K, Mercer SW, Norbury M, Watt G, Wyke S, Guthrie B. Epidemiology of multimorbidity and implications for health care, research, and medical education: a cross-sectional study. Lancet. 2012;380(9836):37–43.
Higashi T, Wenger NS, Adams JL, Fung C, Roland M, McGlynn EA, et al. Relationship between number of medical conditions and quality of care. N Engl J Med. 2007;356(24):2496–504.
Lee TA, Shields AE, Vogeli C, Gibson TB, Woong-Sohn M, Marder WD, et al. Mortality rate in veterans with multiple chronic conditions. J Gen Intern Med. 2007;22 Suppl 3:403–7.
Piette JD, Kerr EA. The impact of comorbid chronic conditions on diabetes care. Diabetes Care. 2006;29(3):725–31.
Zulman DM, Asch SM, Martins SB, Kerr EA, Hoffman BB, Goldstein MK. Quality of Care for Patients with Multiple Chronic Conditions: The Role of Comorbidity Interrelatedness. J Gen Intern Med. 2014;29(3):529–37.
Guthrie B, Payne K, Alderson P, McMurdo ME, Mercer SW. Adapting clinical guidelines to take account of multimorbidity. BMJ. 2012;345:e6341.
Roland M, Paddison C. Better management of patients with multimorbidity. BMJ. 2013;346:f2510.
Smith MA, Wright A, Queram C, Lamb GC. Public reporting helped drive quality improvement in outpatient diabetes care among Wisconsin physician groups. Health Aff (Millwood). 2012;31(3):570–7.
Tricco AC, Ivers NM, Grimshaw JM, Moher D, Turner L, Galipeau J, et al. Effectiveness of quality improvement strategies on the management of diabetes: a systematic review and meta-analysis. Lancet. 2012;379(9833):2252–61.
Venkatesh A, Goodrich K, Conway PH. Opportunities for quality measurement to improve the value of care for patients with multiple chronic conditions. Ann Intern Med. 2014;161(10 Suppl):S76–80.
Smith SM, Soubhi H, Fortin M, Hudon C, O’Dowd T. Managing patients with multimorbidity: systematic review of interventions in primary care and community settings. BMJ. 2012;345:e5205.
Pentakota SR, Rajan M, Fincke BG, Tseng CL, Miller DR, Christiansen CL, et al. Does diabetes care differ by type of chronic comorbidity?: An evaluation of the Piette and Kerr framework. Diabetes Care. 2012;35(6):1285–92.
Dalkey N, Helmer O. An Experimental Application of the Delphi Method to the Use of Experts. Manag Sci. 1963;9(3):458–67.
Dixon LB, Kreyenbuhl JA, Dickerson FB, Donner TW, Brown CH, Wohlheiter K, et al. A comparison of type 2 diabetes outcomes among persons with and without severe mental illnesses. Psychiatr Serv. 2004;55(8):892–900.
American Diabetes Association. Standards of medical care in diabetes–2014. Diabetes Care. 2014;37 Suppl 1:S14–80.
Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR, Lázaro P, et al. The RAND/UCLA Appropriateness Method User’s Manual. RAND: Santa Monica, CA; 2001.
National Committee for Quality Assurance. HEDIS 2013: Technical Specifications for Physician Measurement. Washington, DC: National Committee for Quality Assurance (NCQA); 2013. p. 1–5.
Hwang W, Weller W, Ireys H, Anderson G. Out-of-pocket medical spending for care of chronic conditions. Health Aff (Millwood). 2001;20(6):267–78.
Naessens JM, Stroebel RJ, Finnie DM, Shah ND, Wagie AE, Litchy WJ, et al. Effect of multiple chronic conditions among working-age adults. Am J Manag Care. 2011;17(2):118–22.
ealthcare Cost and Utilization Project. Clinical classifications software (CCS) for ICD-9-CM. Rockville, MD: Agency for Healthcare Research and Quality; 2008.
Powell C. The Delphi technique: myths and realities. J Adv Nurs. 2003;41(4):376–82.
Zevin B, Levy JS, Satava RM, Grantcharov TP. A Consensus-Based Framework for Design, Validation, and Implementation of Simulation-Based Training Curricula in Surgery. J Am Coll Surg. 2012;215(4):580–6.
Rayens MK, Hahn EJ. Building Consensus Using the Policy Delphi Method. Policy Polit Nurs Pract. 2000;1(4):308–15.
Woodard LD, Urech T, Landrum CR, Wang D, Petersen LA. Impact of comorbidity type on measures of quality for diabetes care. Med Care. 2011;49(6):605–10.
Petersen LA, Woodard LD, Henderson LM, Urech TH, Pietz K. Will hypertension performance measures used for pay-for-performance programs penalize those who care for medically complex patients? Circulation. 2009;119(23):2978–85.
Voorham J, Haaijer-Ruskamp FM, Wolffenbuttel BH, de Zeeuw D, Stolk RP, Denig P. Differential effects of comorbidity on antihypertensive and glucose-regulating treatment in diabetes mellitus–a cohort study. PLoS One. 2012;7(6):e38707.
Kerr EA, Heisler M, Krein SL, Kabeto M, Langa KM, Weir D, et al. Beyond comorbidity counts: how do comorbidity type and severity influence diabetes patients’ treatment priorities and self-management? J Gen Intern Med. 2007;22(12):1635–40.
Meduru P, Helmer D, Rajan M, Tseng CL, Pogach L, Sambamoorthi U. Chronic illness with complexity: implications for performance measurement of optimal glycemic control. J Gen Intern Med. 2007;22 Suppl 3:408–18.
Boyd CM, Fortin M. Future of Multimorbidity Research: How Should Understanding of Multimorbidity Inform Health System Design? Public Health Rev. 2010;32(2):451–74.
Murphy MK, Black NA, Lamping DL, McKee CM, Sanderson CF, Askham J, et al. Consensus development methods, and their use in clinical guideline development. Health Technol Assess. 1998;2(3):i–iv. 1–88.
Ducat L, Philipson LH, Anderson BJ. The mental health comorbidities of diabetes. JAMA. 2014;312(7):691–2.
Yazdany J, Panopalis P, Gillis JZ, Schmajuk G, MacLean CH, Wofsy D, et al. A quality indicator set for systemic lupus erythematosus. Arthritis Rheum. 2009;61(3):370–7.
Peters MJ, Symmons DP, McCarey D, Dijkmans BA, Nicola P, Kvien TK, et al. EULAR evidence-based recommendations for cardiovascular risk management in patients with rheumatoid arthritis and other forms of inflammatory arthritis. Ann Rheum Dis. 2010;69(2):325–31.
Bartels CM, Saucier JM, Thorpe CT, Kind AJ, Pandhi N, Hansen KE, et al. Monitoring diabetes in patients with and without rheumatoid arthritis: a Medicare study. Arthritis Res Ther. 2012;14(4):R166.
Bayliss EA, Bonds DE, Boyd CM, Davis MM, Finke B, Fox MH, et al. Understanding the context of health for persons with multiple chronic conditions: moving from what is the matter to what matters. Ann Fam Med. 2014;12(3):260–9.
The authors thank Colleen Brown, Health Innovation Program, for help with preparing the manuscript.
This project was supported by the Health Innovation Program, the UW School of Medicine and Public Health from The Wisconsin Partnership Program, and the Community‐Academic Partnerships core of the University of Wisconsin Institute for Clinical and Translational Research (UW ICTR) through the National Center for Advancing Translational Sciences (NCATS), grant UL1TR000427. Additional support was provided by grants R21 HS007646 and R01 HS018368 from the Agency for Healthcare Research and Quality (AHRQ). The work presented here was carried out while Dr. Magnan was a Primary Care Research Fellow supported by a National Research Service Award (T32HP10010) from the Health Resources and Services Administration to the University of Wisconsin Department of Family Medicine. Christie Bartels was supported by the Marshfield Clinic Research Foundation, as well as NIH/NIAMS K23 AR062381-02. Heather Johnson was supported by the National Heart, Lung, and Blood Institute of the National Institutes of Health under award number K23HL112907, and also by the University of Wisconsin Centennial Scholars Program of the University of Wisconsin School of Medicine and Public Health. Nancy Pandhi was supported by a National Institute on Aging Mentored Clinical Scientist Research Career Development Award, grant number l K08 AG029527. Elizabeth Jacobs was supported by the National Institutes on Aging R01 AG033172-01. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or AHRQ.
The authors declare that they have no competing interests.
EM and RG contributed to conceptualization of the project and analyzed data. EM wrote the manuscript. RG contributed to writing the introduction, methods and discussion, and critically edited the manuscript. MS contributed to conceptualization and the development of the project, and critically edited the manuscript. CB, HJ, NP, and EJ contributed to the development of the project, wrote segments of the discussion and critically edited the manuscript. All authors read and approved the final manuscript.
EM, CB, HJ, NP and EJ are clinician-researchers in health services research. RG was a research specialist at the Health Innovation Program during this project. MS is the Director of the University of Wisconsin School of Medicine and Public Health’s Health Innovation Program, a multidisciplinary health services research center.