Skip to main content

When is referral from primary care to specialist services appropriate for survivors of stroke? A modified RAND-appropriateness consensus study



There is guidance in the United Kingdom about what long-term care stroke survivors should receive, but a lack of guidance about who should deliver it and where this care should take place. This is a key issue given the evidence that current needs are not well addressed. The purpose of this study was to explore when a referral from generalist to specialist services is appropriate in the long-term management of stroke survivors.


A modified RAND-Appropriateness method was used to gain consensus from a range of stroke specialist and generalist clinicians. Ten panelists rated fictional patient scenarios based on long-term post-stroke needs. Round 1 was an online survey in which panelists rated the scenarios for a) need for referral to specialist care and b) if referral was deemed necessary, need for this to be specifically to a stroke specialist. Round 2 was a face-to-face meeting in which panelists were presented with aggregate scores from round 1, and invited to discuss and then re-rate the scenarios.


Seventeen scenarios comprising 69 referral decisions were discussed. Consensus on whether the patient needed to be referred to a specialist was achieved for 59 (86%) decisions. Of the 44 deemed needing referral to specialists, 18 were judged to need referral to a stroke-specialist and 14 to a different specialist. However, for 12 decisions there was no consensus about which specialist the patient should be referred to. For some scenarios (spasticity; incontinence; physical disability; communication; cognition), referral was deemed to be indicated regardless of severity, whereas indications for referral for topics such as risk factor management and pain depended on complexity and/or severity.


There was broad agreement about when a stroke survivor requires referral to specialist care, but less agreement about destination of referral. Nevertheless, there was agreement that some of the longer-term issues facing stroke survivors are best addressed by stroke specialists, some by other specialists, and some by primary care. This has implications for models of longer-term stroke care, which need to reflect that optimal care requires access to, and better co-ordination between, both generalist and specialist healthcare.

Peer Review reports


Stroke is a major cause of disability-adjusted life years in the UK [1] and carries a large economic cost of health and social care for stroke survivors [2, 3] totalling £3.6 billion in the UK in the first 5 years after admission [4].

Post-stroke survival rates are increasing [5], leading to increased need for long-term care of survivors. Most interventions are targeted in the first few months after stroke and there is a lack of evidence for interventions for longer-term stroke care in the community [6]. However, there is evidence that patient needs continue long after discharge, although these needs are not always identified by health professionals [7, 8].

The UK’s National Institute for Clinical Excellence has produced guidelines for rehabilitation after stroke, but these do not specify where these interventions should happen or when referrals to specialist services should take place [9]. Access to post-acute stroke care varies across the country [10] and there is variation in GP referral rates at both GP and practice level [11]. Post-stroke needs can be many and varied, including those in medical, rehabilitation, social and psychological domains. Increasing financial strain on the NHS in the UK can lead to pressure on clinicians to reduce referrals to specialist care [12] and so there is potential for patient needs being unmet by health care services even when they have been identified. The lack of evidence base for longer-term care of people with stroke in the community means that it is difficult to make firm recommendations.

The evidence that longer-term needs after stroke are not well addressed raises a question for both policy makers and health care professionals about whether the unmet needs would be better met by developing primary care services, or by improving access to specialist services. To inform this debate, we carried out this study addressing the question: from a health care professional perspective when is referral from primary care to specialist services appropriate in the long-term management of problems encountered by stroke survivors?


A modified RAND Appropriateness method was used [13]. This is a consensus method originally developed for establishing criteria for clinical interventions, but has also been used for developing clinical guidance [14] and quality indicators [15]. In brief, a series of patient scenarios were distributed to a panel participants by email. For each scenario, the panelist was asked to indicate on a scale of 1 to 9 the strength of their opinion about whether the patient described in the scenario should be referred to a specialist. The scores were collated and presented at a face-to-face meeting of the panelists. Each scenario was discussed in turn, and the panelists rescored each scenario. The final scores were then analysed with pre-defined rules as to what comprised consensus (see below). The study was approved by the University of Cambridge Psychology Research Ethics Committee (REC Reference PRE.2017.059).


A 10-person expert panel was recruited by the study team to include a range of professional backgrounds and comprised: 4 general practitioners, 3 stroke physicians, 1 physiotherapist, 1 occupational therapist and 1 practice nurse. Initially, a further stroke physician, a clinical psychologist and a representative from the Stroke Association (a UK national charity which provides support for survivors of stroke and their carers) were recruited, but had to withdraw too late for replacements to be found. Participants were nominated by contacts of the study team and carefully selected for their expertise in their field; all had an active research and/or clinical interest in stroke or cardiovascular disease.


Fictional post-stroke scenarios were composed by a writing group comprising clinicians and researchers from the study team for the Improving Primary Care after Stroke Programme, (NIHR Programme Grant PTC-RP-PG-0213-20,001) [16]. Other clinicians known to the study team were asked to contribute where additional expertise was required (a clinical psychologist, stroke physician and a speech and language therapist). These scenarios were based on long-term post-stroke problem areas identified by Philp et al. [17], supplemented by focus group work carried out by the Improving Primary Care After Stroke Programme team [18]. The list of topics encompassed physical, psychological, social and carer issues, and aspects of secondary prevention.

Seventeen scenarios were devised, each broadly based around a topic area and requiring between three and five referral decisions. These were designed so that the severity or impact of the problem increased as the scenario progressed. See Fig. 1 for an example scenario.

Fig. 1

Example fictional scenario

Round 1 – online questionnaire round

Panelists were sent a link via email to an online survey using Qualtrics software [19]. Electronic consent was obtained and each panelist was asked to rate each referral decision for each of the 17 scenarios as follows:

  1. i.)

    Assuming current provision in primary care, should this patient be referred to specialist services? (nine point scale from 1 ‘referral definitely not necessary’ to 9 ‘referral definitely required’)

  2. ii.)

    If you have indicated 7, 8 or 9 above, please indicate whether this referral should be made to a member of a stroke specialist team (1 being referral to a stroke specialist team definitely not necessary, 9 being referral to a stroke specialist team definitely required)

Participants were also asked to rate each complete scenario for clarity as follows:

Is this scenario clear? (1 being not at all clear, 9 being completely clear).

The panelists were specifically instructed to disregard cost in judging whether a referral was or was not necessary. Panelists were not required to provide a rating if they felt the subject was out of their area of expertise.

Round 2 - consensus meeting

Round 2 was a face-to-face meeting held over 2 days. Written consent was obtained from each panelist. Panelists were given the results from the first round presented graphically as the distribution of ratings made by the group for each referral decision, with their own rating additionally indicated. The group was facilitated by an expert in RAND methodology (M.R.) and an expert in stroke research (J.M.) Each referral decision was discussed and participants asked to re-rate each one after the discussion.

Where terms were unclear, definitions were discussed and agreed upon by the panel (Table 1). The voluntary sector was defined as not being a specialist service (i.e. when support from the voluntary sector was recommended, this was not classified as a referral). Definitions were also provided for services that in parts of the UK can be accessed by patients via self-referral (such as physiotherapy or psychological services). It was agreed that access to such services would be defined as a referral outside of the primary care team whether or not the patient could access them without a formal referral from their primary care physician.

Table 1 Definitions

Two researchers took notes of the reasoning behind each decision made. These were later discussed with the two facilitators to agree a short summary for each topic area (Table 2).

Table 2 Quantitative Summary of Referral Decisions made


The ratings were divided into three 3-point regions on a 9 point scale: 1–3 (referral not necessary), 4–6 (unsure about need for referral) and 7–9 (referral required). No scenarios were deemed unclear requiring rewriting.

Not reaching consensus was defined either as “Disagreement” as per the RAND Appropriateness Manual for a 10 person panel [13]: “at least three panelists rate the indication in the 1-3 region, and at least three panelists rate it in the 7-9 region”, or as a median score of 4–6. Consensus for referral was defined as a median score of 7–9 without disagreement. Consensus for non-referral was defined as a median score of 1–3 without disagreement.


Figure 2 shows the decisions made at each stage of the consensus process. Seventy-one referral decisions were discussed in Round 2; two were removed as the panel decided they did not add anything to the clinical picture. Consensus was not reached in 10 out of 69 (14%) referral decisions. For 15 (22%) decisions, referral was not deemed necessary; for 18 (26%), referral to a stroke specialist was deemed appropriate and for 14 (20%) decisions, while referral was appropriate, it did not need to be to a stroke specialist; for 12 (17%) decisions, while there was agreement on the need for referral, there was no consensus as to whether this should or should not be to a stroke specialist.

Fig. 2

Flow chart showing decisions made at each stage of the consensus process

Table 2 summarises the consensus achieved in each of the topic areas and Table 3 shows key points of discussion for each scenario.

Table 3 Explanations for referral decisions made

Consensus by topic area

There was broad consensus on need for referral in most parts of all topic areas. For some scenarios (spasticity; incontinence; physical disability; communication; cognition), referral was deemed to be indicated regardless of level of severity, whereas indications for referral for topics such as risk factor management and pain depended on complexity and/or severity. Common factors where agreement was present were:

Referral to specialist care was deemed appropriate when the intervention required specific resources e.g. in the incontinence scenario, referral was deemed appropriate for all parts of the scenario, even the “less severe”.

Referral to specialist care was deemed appropriate when the intervention was complex and out of the usual scope of practice of primary care clinicians (as agreed by the panel) e.g. incontinence, speech and language, physiotherapy.

Referral was not thought to be required when the topic was within the generally agreed scope of practice for primary care, such as anticoagulation, pain control and control of cardiovascular risk factors. However, there may be instances where the severity cannot be adequately managed in primary care alone, requiring specialist input. For example, while there was consensus that anticoagulation could be managed in primary care, it was agreed that referral was appropriate if there was suspicion of new pathology that might increase the potential harms of treatment. Similarly, complex lipid management merited referral (in line with national guidelines). Pain control was considered a primary care issue, but referral could be considered where impact on day-to-day life was severe, or if there was a failure to respond to standard treatments. While increasing severity was usually considered an indication for referral, this was not always the case. In the example scenario (Fig. 1), one participant felt that the initial presentation merited referral, because it raised the possibility of new neurological pathology. However, with progression of the symptoms, the diagnosis became clearer, so that the participant no longer felt referral was merited.

Referral to specialist stroke services or to other services?

Some services not specific to stroke were thought to be appropriate for stroke survivors if they met the relevant criteria. For example, an individual may experience memory issues following their stroke and meet the criteria for referral to the memory clinic, in which case this is likely to be a more appropriate referral than to a stroke specialist. Similarly, incontinence issues were considered best dealt with by a continence team, rather than stroke specific services.

In some topic areas where a referral was indicated for a specific therapist, such as for speech and language, swallowing difficulties, physical disability and spasticity, the panel felt that ideally the referrals should be made to a stroke specialist team. The common factor here is that the problems faced by the patient are likely to be different from those requiring speech and language therapy or physiotherapy for non-stroke reasons; however, this may be dependent on what services are available locally.

Reasons for lack of consensus

Topic areas in which one or more parts of the scenario did not reach consensus were: cardiovascular risk factors (including blood pressure control), mood, adjustment, pain, swallowing, intimate relationships, mild fatigue, carer needs and employment. Panel comments as well as consensus scores were considered when drawing conclusions from the data (see Tables 2 and 3).

Where there was disagreement about a referral decision, this tended to be with situations where it was generally accepted that primary care could manage the problem, but it was debatable whether or not the patient would derive additional benefit from specialist input. Disagreement was not split along professional lines – of the 10 decisions with disagreement, in none of them did all 4 GPs agree, and only in one did all 3 stroke specialists agree.

Where consensus was reached for need for referral but not referral destination (12 decisions), the discussion often reflected differences in opinion of what the service needed to achieve. For example, regarding referral for post-stroke pain, some of the panelists felt that a stroke specialist would be most appropriate, but others felt that the patient would benefit more from a biopsychosocial approach which would be provided by a pain team.


To our knowledge, this study is the first to explore consensus on referral decisions for people with stroke. The results showed that there was good consensus across professional groups who work with stroke patients in both primary and secondary care about when referral would be appropriate. Less clear was the role of specialist stroke services as opposed to other specialist services; it was not necessarily the specific specialist area, rather the severity of the problem that determined consensus regarding the need for referral.

The face-to-face meeting included in this study allowed the expert panel to discuss case scenarios, resolving any misinterpretations and agreeing definitions where ambiguity existed. The use of fictional scenarios, designed to increase in severity with each referral decision, helped to illustrate the complexity of individual situations post-stroke; however this did make drawing conclusions from the data more difficult as not all scenarios could fit neatly into one clinical topic area and in reality, stroke survivors often have problems in multiple domains. The study team did not attempt to produce guidelines, rather conclusions from the data were drawn to address examples of clinical situations that primary care clinicians might face in practice and illustrate some of the types of problems that would generally be expected to be managed in primary care and those that a specialist would be expected to see within their scope of practice. The consensus group members were chosen because of known interests in stroke rather than aiming to be specifically representative of their professional group. Not all relevant specialties were included, for instance a clinical psychologist was unable to make it at the last minute; we were also unable to find a speech and language therapist able to attend. It is plausible that a different group would have drawn different conclusions.

There are a number of factors which may have accounted for the lack of agreement in some scenarios as to whether or not they could be managed in primary care. Difficulty in accounting for individual primary care clinicians’ expertise, and what services are available locally were two important factors. These may also have contributed to lack of agreement within professional groups as patients in one area may have access to different local services, which would affect the clinical decision making when it comes to referral. The face-to-face meeting allowed differences in interpretation of the scenarios to be clarified, which is an advantage over questionnaire-only consensus methodology.

Policy implications

Stroke survivors and their caregivers may feel abandoned and marginalised by services; this marginalisation occurs because of the passivity of services and the lack of knowledge and skills that stroke survivors have to re-engage [20]. Hospital stroke teams do not routinely continue to follow up stroke survivors, handing over care to primary care and community teams normally within a year after the stroke although stroke survivors have changing needs at different times [7, 21]. There is recognition that the longer-term needs of stroke patients are not well met [22]. This may be due to lack of identification of these needs by health care professionals [23], or lack of effective action once identified [7]. A key consideration is the level of expertise required to address needs. In this study, we devised scenarios that reflected the range of problems most commonly met by stroke survivors in the community and found broad consensus as to which merited specialist referral. For many of the referral decisions, there was agreement that referral was appropriate. This has implications for primary care services, in that they need to be able to identify problems, and have knowledge of local referral pathways. This has been taken into account in the development of the Improving Primary Care After Stroke (IPCAS) model of service delivery for stroke care in the community which includes use of a checklist in primary care [18], mapping local services and how to access them, and improving communication between primary and secondary care [16]. One component of improving communication may be to facilitate contact between the primary care practitioner and the specialist prior to the referral. The current separation between primary care and specialist care may not always be helpful for the individual patient; further work to develop models of care that use a mix of specialist and generalist input which can be tailored to meet the varied individual needs of stroke survivors is likely to be valuable.

There was less agreement over whether or not the referral needed to be to a stroke specialist or to a non-stroke specialist. Given that not all problems faced by stroke survivors require a stroke specialist (for example memory problems may be caused by stroke but may be better served by a memory clinic) and in light of a reported shortage of stroke physicians in the UK [24], it is important to clarify which aspects of care require stroke-specific input.


In the absence of a strong evidence base, consensus work such as this has a role in informing what might be the optimal mix of primary care, stroke services and non-stroke specialist services that are required to better address the longer-term needs of stroke survivors. However, national policy and guidelines should be under-pinned by evidence. It is important that new models of longer-term stroke care continue to be developed and evaluated. This study suggests that such a model should include a role for both generalist and specialist healthcare, and therefore implicitly, better co-ordination and communication between primary and secondary care.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.


  1. 1.

    Newton JN, Briggs AD, Murray CJ, Dicker D, Foreman KJ, Wang H, et al. Changes in health in England, with analysis by English regions and areas of deprivation, 1990–2013: a systematic analysis for the global burden of disease study 2013. Lancet (London, England). 2015;386(10010):2257–74.

    Article  Google Scholar 

  2. 2.

    Evers SM, Struijs JN, Ament AJ, van Genugten ML, Jager JHC, van den Bos GA. International comparison of stroke cost studies. Stroke. 2004;35(5):1209–15.

    Article  Google Scholar 

  3. 3.

    Saka Ö, McGuire A, Wolfe C. Cost of stroke in the United Kingdom. Age Ageing. 2009;38(1):27–32.

    Article  Google Scholar 

  4. 4.

    Xu X-M, Vestesson E, Paley L, Desikan A, Wonderling D, Hoffman A, et al. The economic burden of stroke care in England, Wales and Northern Ireland: Using a national stroke register to estimate and report patient-level health economic outcomes in stroke. Eur Stroke J. 2017;3(1):82–91.

  5. 5.

    Lakshminarayan K, Berger AK, Fuller CC, Jacobs DR, Anderson DC, Steffen LM, et al. Trends in 10-year survival of patients with stroke hospitalized between 1980 and 2000: the Minnesota stroke survey. Stroke. 2014;45(9):2575–81.

    Article  Google Scholar 

  6. 6.

    Chen R, McKevitt C, Crichton SL, Rudd AG, Wolfe CD. Socioeconomic deprivation and provision of acute and long-term care after stroke: the South London stroke register cohort study. J Neurol Neurosurg Psychiatry. 2014;85(12):1294–300.

    Article  Google Scholar 

  7. 7.

    McKevitt C, Fudge N, Redfern J, Sheldenkar A, Crichton S, Rudd AR, et al. Self-reported long-term needs after stroke. Stroke. 2011;42(5):1398–403.

    Article  Google Scholar 

  8. 8.

    Kristensen HK, Tistad M, Von Koch L, Ytterberg C. The importance of patient involvement in stroke rehabilitation. PLoS One. 2016;11(6):e0157149.

    Article  Google Scholar 

  9. 9.

    Dworzynski K, Ritchie G, Fenu E, MacDermott K, Playford ED. Rehabilitation after stroke: summary of NICE guidance. BMJ. 2013;346:f3615.

  10. 10.

    Stroke Association. A new era for stroke 2016. Accessed 21 Oct 2019.

  11. 11.

    O'Donnell CA. Variation in GP referral rates: what can we learn from the literature? Fam Pract. 2000;17(6):462–71.

    CAS  Article  Google Scholar 

  12. 12.

    NHS England. Demand Management Good Practice Guide. Version 1.1 ed 2016. Accessed 21 Oct 2019.

  13. 13.

    Fitch K, Bernstein SJ, Aguilar MD, Burnand B, JR LC. The RAND/UCLA appropriateness method user's manual. Santa Monica CA: RAND CORP; 2001.

    Google Scholar 

  14. 14.

    Shekelle P. The appropriateness method. Med Decis Mak. 2004;24:228–31.

    Article  Google Scholar 

  15. 15.

    Saust LT, Bjerrum L, Arpi M, Hansen MP. Quality indicators for the diagnosis and antibiotic treatment of acute respiratory tract infections in general practice: a RAND appropriateness method. Scand J Prim Health Care. 2017;35(2):192–200.

    Article  Google Scholar 

  16. 16.

    Mullis R, Aquino M, Dawson SN, Johnson V, Jowett S, Kreit E, et al. Improving primary care after stroke (IPCAS) trial: protocol of a randomised controlled trial to evaluate a novel model of care for stroke survivors living in the community. BMJ Open. 2019;9(8):e030285.

    Article  Google Scholar 

  17. 17.

    Philp I, Brainin M, Walker MF, Ward AB, Gillard P, Shields AL, et al. Development of a poststroke checklist to standardize follow-up care for stroke survivors. J Stroke Cerebrovasc Dis. 2013;22(7):e173–e80.

    Article  Google Scholar 

  18. 18.

    Turner GM, Mullis R, Lim L, Kreit L, Mant J. Using a checklist to facilitate management of long-term care needs after stroke: insights from focus groups and a feasibility study. BMC Fam Pract. 2019;20(1):2.

    Article  Google Scholar 

  19. 19.

    Qualtrics software. Provo, Utah, USA. Version May 2017 of the Qualtrics Research Suite. Available from: Accessed 15 Jan 2019.

  20. 20.

    Pindus DM, Mullis R, Lim L, Wellwood I, Rundell AV, Aziz NAA, et al. Stroke survivors' and informal caregivers' experiences of primary care and community healthcare services–a systematic review and meta-ethnography. PLoS One. 2018;13(2):e0192533.

    Article  Google Scholar 

  21. 21.

    Kirkevold M. The unfolding illness trajectory of stroke. Disabil Rehabil. 2002;24(17):887–98.

    CAS  Article  Google Scholar 

  22. 22.

    Chen T, Zhang B, Deng Y, Fan J-C, Zhang L, Song F. Long-term unmet needs after stroke: systematic review of evidence from survey studies. BMJ Open. 2019;9(5):e028137.

    Article  Google Scholar 

  23. 23.

    Tistad M, Ytterberg C, Thama K, von Kocha L. Poor concurrence between disabilities as described by patients and established assessment tools three months after stroke: a mixed methods approach. J Neurol Sci. 2012;313(1):160–16.

    Article  Google Scholar 

  24. 24.

    Gulland A. RCP warns over shortage of stroke physicians. BMJ. 2016;355:i6398.

    Article  Google Scholar 

Download references


JM is an NIHR Senior Investigator.


This paper summarises independent research funded by the National Institute for Health Research (NIHR) under its Programme Grant for Applied Research (PTC-RP-PG-0213-20001). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

Author information




LL, MR, JM and RM contributed to the study conception and design. LL, MR, JM and RM were involved in the interpretation of results. LL drafted the manuscript and MR, JM and RM provided feedback. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Lisa Lim.

Ethics declarations

Ethics approval and consent to participate

The study was approved by the University of Cambridge Psychology Research Ethics Committee (REC) – REC Reference PRE.2017.059.

Electronic consent was obtained from each participant prior to the online questionnaire. Written consent was obtained from each participant at the start of the face-to-face meeting.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Lim, L., Mant, J., Mullis, R. et al. When is referral from primary care to specialist services appropriate for survivors of stroke? A modified RAND-appropriateness consensus study. BMC Fam Pract 21, 66 (2020).

Download citation


  • Consensus
  • Stroke
  • Referral
  • Primary health care
  • Specialist