This article has Open Peer Review reports available.
Measuring access to primary care appointments: a review of methods
© Jones et al; licensee BioMed Central Ltd. 2003
Received: 15 April 2003
Accepted: 07 July 2003
Published: 07 July 2003
Patient access to primary care appointments is not routinely measured despite the increasing interest in this aspect of practice activity. The generation of standardised data (or benchmarks) for access could inform developments within primary care organisations and act as a quality marker for clinical governance. Logically the setting of targets should be based on a sound system of measurement. The practicalities of developing appropriate measures need debate. Therefore we aimed to search for and compare methods that have been published or are being developed to measure patient access to primary care appointments, with particular focus on finding methods using appointment system data.
A search and review was made of the primary care literature from 1990 to 2001, which included an assessment of online resources (websites) and communication with recognised experts. The identified methods were assessed.
The published literature in this specific area was not extensive but revealed emerging interest in the late 1990s. Two broad approaches to the measurement of waiting times to GP appointments were identified. Firstly, appointment systems in primary care organisations were analysed in differing ways to provide numerical data and, secondly, patient perceptions (reports) of access were evaluated using survey techniques. Six different methods were found which were based on appointment systems data.
The two approaches of either using patient questionnaires or appointment system data are methods that represent entirely different aims. The latter method when used to represent patient waiting times for 'routine' elective appointments seems to hold promise as a useful tool and this avoids the definitional problems that surround 'urgent' appointments. The purpose for which the data is being collected needs to be borne in mind and will determine the chosen methods of data retrieval and representation.
Primary care is under scrutiny along with other public services to improve access to its users. Access in primary care is typically conceptualised as the achievable access to appointments with clinical professionals, although it is not routinely measured in most practices. It is widely recognised that it represents an important dimension in determining the quality of care [1, 2]. Nevertheless, it is also known that waiting times can vary widely in differing localities and countries, from patients being seen the same day to a wait of several weeks. Most primary care organisations have no more than a perception of variable demand and no method of comparing fluctuating levels of access to appointments within or between practices. Measuring patient access could generate useful information for patients, clinicians and practice managers. Demand management initiatives and ways to optimise access could then be audited bearing in mind the impact of such initiatives on opportunity costs [3–5].
There are potential difficulties, recognised by a recent discussion document , that have been accentuated by policy and political influences. The NHS Plan in the UK  suggested that patients should have access to primary care services within 48 hours, but the concept lacked sufficient definition. The proposed new GMS contract includes an optional 48 hour target for access to GP appointments. But what exactly should we measure? Access can be measured at many different interfaces, from the wait for traditional services such as appointments with a clinician to the alternative solutions of nurse triage, nurse led clinics, telephone advice (including NHS Direct) or electronic mail responses. Differing interpretation of terms can also cause confusion. How soon for example should a problem that is defined by a patient as urgent be seen in general practice? The perception of urgent differs between patients themselves  and between doctors and patients. The concept of routine appointments is easier to define and quantify. A proposed measure is the waiting time for the next available routine appointment but this provokes debate about whether this should be practice-based or specified for individual clinicians. If the latter, factors such as part-time working, practitioner popularity, the creation of multiple review appointments are likely to rapidly diminish a clinician's accessibility. How transparent would a practice want to be about such data and how useful or acceptable would it be to publish information at the clinician level? 
Recognising the complex nature of this issue, we set out to review the literature. Our main aim was to search the international primary care literature for methods that had been, or were being developed to measure access to GP appointments, focusing on measures using appointment system data. Once identified, existing methods (tools, scales or other instruments) would be compared, with specific attention given to the type and levels of access they aimed to assess.
Preliminary searches indicated that this area did not have an extensive or long-standing research literature; most articles had been published after 1998. A broad but systematic search process was designed to allow for a poorly indexed publication pool. Medline, PubMed, Clin Psyc and ASSIA were searched for relevant publications between 1990 and 2001. The following MeSH terms were used: family practice, health service accessibility (organisation and administration, statistics and numerical data, standards, trends, methods, manpower), appointments and schedules (waiting lists), research design, health service needs and demand, weights and measures, quality of healthcare, management audit, patient satisfaction, health service needs and demand. In addition the following keywords were used: general practice, access, appointments (same day, urgent, routine) appointment systems, measurement, measures, tools, scales, demand, availability, audit and waiting times. Terms were used both singly and in combination. Title searches were used to increase the sensitivity. All citations and abstracts were appraised for relevance and full articles selected for examination by two researchers independently (GE and WJ). Key authors were contacted directly [1, 12, 16, 18, 21, 25, 27] and searches conducted on their previous work. Departments of General Practice in Universities in the United Kingdom were also asked to send details of any relevant research. Conference literature  and non-peer reviewed literature obtained from websites was also appraised. Relevant websites were identified using http://omni.ac.uk and the search terms health service delivery, access to primary care and general practice. The following sites were reviewed: the Royal College of General Practitioners , the National Primary Care Research and Development Centre , The National Primary Care Development Team  and the Centre for Innovation in Primary Care .
Studies or articles were included in the review if they described tools, scales, questionnaires or other methods of measuring actual patient access to appointments. We also included descriptions of methods that were currently being developed in this field, provided they had undertaken pilot studies and had completed one data collection exercise. Articles were excluded if they were purely editorial.
A total of 1763 citations were initially identified and 38 articles retrieved for detailed assessment from the Pubmed and the Medline searches. Clin Psych and ASSIA searches provided some overlap but no new relevant material. The most helpful pointers to relevant publications were obtained from the website searches and personal communications rather than the traditional search engines. Two broad approaches to the measurement of patient access were identified. Firstly, appointment systems in organisations were analysed in differing ways to provide numerical data and, secondly, patient perceptions (reports) of access were evaluated using survey techniques.
Methods using appointment system data
Comparison of methods based on appointments systems to measure access to primary care
Third appointment 
Access Response Index AROS 
The 3rd available routine appointment at 12 midday, one day per week, for every clinician
Date of patient call and appointment provided. GP requested and GP allocated.
Appointments demanded but not available in US style primary care clinics compared to community clinics. Demand versus availability gap coded into 4 categories.
Number of days until next available routine appointment (with any clinician) at 4 pm, every working day
Number of appointments provided at the beginning of the day and the number still available. Total number of patients seen during the day, noting the number of 'extras'
Number of appointments available at the start of the day and the number of patients seen as 'extras' at the end of the day
Frequency of data collection
Once a week
Once a day
Twice a day
Twice a day
Weighted for part time staff
Named clinician access measured
Weekly median score and monthly average
Demand versus availability gap
Computer to work out 5 day moving average
Data related to practice list size, with rates given per 1000 patients
Weekly snapshot of patient access profile
Complete computerised analysis of practice appointment system
Feedback reports generated to clinic staff
Trends across weekly schedules.
Bar charts represent number of appointments offered versus number of patients seen. Start of day appointment availability categorised as low, medium and high
Graphical display of extras versus number of free appointments during the day
Extent of and reason for use
Primary Care Collaborative in England. To inform implementation of advanced access
145 teaching practices Audit
US Military Clinic Study
10 practices To inform improvement
19 practices Research Study
1 practice Research Study
National Primary Care Development Team
Department of General Practice, University of Glasgow.
Healthcare Programs Central Michigan University.
University department of General Practice
University Department of General Practice, Edinburgh.
Department of Primary Health Care, University of Southampton.
Three methods determine appointment availability and/or the satisfaction of demand on a daily basis but do not measure the days wait for appointments. Campbell measured clinician availability by recording the number of provided appointments at the beginning of each day plus the number of these un-booked at this time . At the end of the day numbers seen and numbers of extras were counted, and adjusted for practice list size. Kendrick and Kerry recorded the number of available appointments at the beginning of the day and the number of extras seen at the end . Ledlow also suggested a categoriations system for recording differing levels of unmet daily demand in a military medical service 
Three methods measure access as days waited by representing appointment system data. A computerised package, NEMAS [15, 16] enables practices to audit four areas: practice service provision (including 'appointment availability' and patient satisfaction), chronic disease management, drug monitoring and significant event analysis. The method calculates the mean time waited in days plus the minimum and maximum patient waits. Data can be presented for the whole practice or for individual GPs. Data entry involves using electronic forms to record the date of appointment request, the date the patient was seen, whether it was an elective, forced (i.e. next available clinician) or urgent appointment, the clinician requested and the clinician actually consulted. The costs of collecting data has varied considerably depending on which staff member is employed for the task . Transfer of data from the practice system can be automatic with compatible systems (personal communication).
The National Primary Care Development Team is measuring access as part of the adoption of the system of 'Advanced Access', a system developed in the USA [18, 19], as a response to patient waiting times of 4 – 6 weeks for routine appointments in primary care . This method collects data on one day per week (which can vary) at 12 noon. The number of days to the third available routine appointment for each clinician is recorded and a median figure calculated to represent an access score for the specified week. Over a month, the average of four median values is taken to represent a monthly access score. The third appointment is chosen, rather than the first, in order to negate the effect of sudden cancellations which otherwise could give false impressions of availability if at the moment of measuring there is one sudden cancellation in a schedule that is otherwise booked for several days ahead. The third appointment has been found by trial and improvement to best represent the actual waits involved (personal communication). Embargoed appointments are not included. A system of weighting has been developed  for part time workers to enable their scores to be incorporated.
Methods using patient questionnaires
Patient survey instruments: items used to determine access perceptions
Thinking of times when you want to see a particular doctor:
a) 5 point scale, 1 = same day, 5 = more than 5 days
a) How quickly do you get an appointment?
b) How do you rate this?
b) Range from 1 = very poor, 6 = excellent
Thinking of times when you are willing to see any doctor:
a) How quickly do you get an appointment?
8. Yes / No / Not applicable / Don't know
b). How do you rate this?
If you need an urgent appointment to see your GP can you normally get one on the same day?
What is your opinion of the general practitioner and/or the practice over the past 12 months with respect to:
5 point scale (poor to excellent)
Getting an appointment to suit you?
Providing quick services for urgent health problems?
It can sometimes be difficult to get an appointment with my doctor at this surgery.
5 point agreement scale
It can be hard to get an appointment for medical care right away.
Getting an appointment at a convenient time is easy.
5 point agreement scale
Appointments are easy to make whenever I need them.
It is often difficult to get an appointment with a doctor.
It is easy to see a doctor of my choice.
This review of access measurement reveals the heterogeneous nature of the methods and the lack of any widely accepted conceptualisation of patient access. Identified measures are either practice centred using appointment data or patient orientated via surgery satisfaction questionnaires. It is clear that these two methods represent entirely different aims. It is not possible for episodic patient surveys to provide data that has enough currency or accuracy to inform organisational responses to patient demand.
Also demonstrated are inherent problems over definition of terms – defining what is to be measured and setting of targets. There appear to be three appointment categories over and above emergencies (which by definition cannot be given 'appointments'). Appointments for healthcare could be categorised into urgent, soon, and elective. Urgent appointments are typically seen as requests for same-day consultations. The soon category would fit problems that should be seen within two or three days to prevent escalation or symptom prolongation. Finally, routine or elective appointments suit individuals who value an agreed time window over other factors. The methods that represent patient waiting times for 'routine', i.e. elective appointments seem to hold promise as they avoid the definitional problems that surround 'urgent' appointments, and the different views that patients, clinicians and others have about 'urgency'. The task of deciding whether to represent access profiles for organisations or for individual clinicians also needs careful consideration. It may be more feasible (and less threatening) to routinely measure organisational access, especially if the data is to be used for benchmarking purposes.
To attempt to measure access means to obtain meaningful data from a dynamic system that is not always in equilibrium. The access experienced by an individual varies determined by demand, adequate appointment provision, sudden cancellations and block release of held appointments. We have distinguished two approaches to overcome this, either for a full statistical analysis using compatible systems or data retrieval software, or to use a simple snapshot method, deciding whether data-smoothing methods such as daily moving averages or aggregated weekly median scores are the best portrayal of overall access patterns.
Strengths and weaknesses of the study
Multiple search methods were used to ensure that the breadth of literature and online resources were examined as systematically as possible. The searches proved difficult and reflect the emergent status and the diversity of terms used in this area. We may have overlooked methods developed in other healthcare systems.
Implications of the findings to healthcare services and research
The lack of a widely agreed measurement method to represent patient access to primary care services will make it impossible for practices to compare their response to patient demand with any degree of certainty. The 'third appointment' system is the most widely used method and is currently supported by the National Primary Care Development Team in England (but has no equivalent support in Scotland, Wales or Northern Ireland). It is however a relatively complex manipulation of appointment system data, and incorporates individual clinician availability. It seems from first principles that the important requirements of a tool designed to measure a dynamic concept such as patient access is simplicity and ease of regular data collection, so that longitudinal data patterns capable of indicating trends in organisations can be generated rather than data on individual clinician availability.
A recent survey compared mechanisms used to manage requests for same day appointments. Murray's proposal of doing today's work today (Advanced Access) eliminates appointment categories and the work involved in negotiating urgency by dealing with virtually all demand on the day it arises. Access is an important determinant of healthcare quality but what are the implications of this approach to the balance of overall quality? Continuity of care, whilst not important for some problems is desirable for others [32, 33]. Too drastic a shift in favour of access is likely to be at the cost of reduced continuity and a diminution of other services, such as screening and chronic disease management. Measures of organisational quality need to be aware of the dangers of focusing too much on one dimension, and should work towards the creation of measures that balance scores across internal and external requirements .
An effective consultation with a well trained clinician who knows the patient and who has access to a well structured longitudinal record will probably remain a gold standard and the issue of immediate 'access' should not be elevated above all the other components in this equation. Nevertheless, it seems prudent to develop measures that provide a better understanding of patient access to organisations with similar resources. Measuring the interval to the next available routine appointment, whilst measuring the measurable  will, if a consistent standard can be agreed, provide benchmark data, and mark an important step towards a compendium of methods to assess quality in primary care.
The two approaches of either using patient questionnaires or appointment system data to measure access are methods that represent entirely different aims. The latter method when used to represent patient waiting times for 'routine' elective appointments seems to hold promise as a useful tool and this avoids the definitional problems that surround 'urgent' appointments. The purpose for which the data is being collected needs to be borne in mind and will determine the chosen methods of data retrieval and representation
The partners, manager and receptionists at Ely Bridge Surgery. The AROS Pilot Study Group: New Park Surgery, Talbot Green; Porthcawl Group Practice; Woodlands Medical Centre, Ely, Cardiff; Old School Surgery, Pontyclun; Greenmount Surgery, Ely, Cardiff; Salisbury Rd Surgery, Barry; Llynfi Surgery, Maesteg; Oldcastle Surgery, Bridgend; Park Lane Surgery, Tonyrefail; Meddygfa Teilo, Llandeilo; The Surgery, Overton-on-Dee, Wrexham.
This work was sponsored by CAPRICORN which receives funding from the Welsh Assembly Government
- Roland M, Holden J, Campbell S: Quality Assessment for General Practice: supporting clinical governance in primary care groups. Manchester: National Primary Care Research and Development Centre. 1998, [http://www.npcrdc.man.ac.uk/]Google Scholar
- Campbell SM, Roland MO: Defining quality of care. Soc Sci Med. 2000, 51: 1611-25. 10.1016/S0277-9536(00)00057-5.View ArticlePubMedGoogle Scholar
- Rogers A, Entwistle V, Pencheon D: A patient led NHS: managing demand at the interface between lay and primary care. BMJ. 1998, 316: 1816-9.View ArticlePubMedPubMed CentralGoogle Scholar
- Pencheon D: Matching demand and supply fairly and efficiently. BMJ. 1998, 316: 1665-7.View ArticlePubMedPubMed CentralGoogle Scholar
- Gillam S, Pencheon D: Managing demand in general practice. BMJ. 1998, 316: 1895-8.View ArticlePubMedPubMed CentralGoogle Scholar
- Royal College of General Practitioners: Access to general practice based primary care. 2001, Accessed March 2001. London, [http://www.rcgp.org.uk/]Google Scholar
- The Stationery Office: The NHS Plan. London: The Stationery Office. 2000Google Scholar
- Campbell JL: Patient's perceptions of medical urgency, does deprivation matter?. Fam Pract. 1999, 16: 28-32. 10.1093/fampra/16.1.28.View ArticlePubMedGoogle Scholar
- Marshall MN, Shekelle PG, Leatherman S, Brook RH: The public release of performance data: what do we expect to gain?. JAMA. 2000, 283: 1866-74. 10.1001/jama.283.14.1866.View ArticlePubMedGoogle Scholar
- The National Primary Care Development Team. [http://www.npdt.org/]
- Centre for Innovation in Primary Care. [http://www.innovate.org.uk]
- Campbell JL: General practitioner appointment systems, patient satisfaction, and use of accident and emergency services: a study in one geographical area. Fam Pract. 1994, 11: 438-445.View ArticlePubMedGoogle Scholar
- Kendrick T, Kerry S: How many surgery appointments should be offered to avoid undesirable number of "extras"?. BJGP. 1999, 49: 273-6.PubMedPubMed CentralGoogle Scholar
- Ledlow GR, Bradshaw DM, Shockley C: Primary care access improvement: an empowerment interaction model. Military Medicine. 2000, 165: 390-4.PubMedGoogle Scholar
- NEMAS [program]: Glasgow: West of Scotland Postgraduate Medical Education Board and MI Technology Ltd. 2000Google Scholar
- Lough M, Wilmot M: Combining audit and information technology in the west of Scotland. Practice Computing. 1996, Autumn: 26-8.Google Scholar
- Lough JRM, Wilmot M, Murray TS: Supporting practice-based audit: a price to be paid for collecting data. BJGP. 1999, 49: 793-5.PubMedPubMed CentralGoogle Scholar
- Murray M, Tantau C: Same-day appointments: exploding the access paradigm. Family Practice Management. 2000, 7: 45-50.PubMedGoogle Scholar
- Murray M, Tantau C: Redefining open access to primary care. Managed Care Quarterly. 1999, 7: 45-55.PubMedGoogle Scholar
- Murray M, Tantau C: Must patients wait?. Journal of Quality Improvement. 1998, 24: 423-5.Google Scholar
- Oldham J: Advanced Access in Primary Care. Manchester: National Primary Care Development Team. 2001Google Scholar
- Elwyn G, Jones W, Emmerson M, Edwards P, Edwards A: Developing a measure for benchmarking access in primary care: Access Response (AROS). Journal of Clinical Evaluation in Clinical Practice. 9,1: 33-37.Google Scholar
- Ramsay J, Campbell JL, Schroter S, Green J, Roland M: The General Practice Assessment Survey (GPAS) tests of data quality and Measurement properties. Fam Pract. 2000, 17: 372-9. 10.1093/fampra/17.5.372.View ArticlePubMedGoogle Scholar
- Safran DG, Kosinski M, Tarlov AR, Rogers WH, Taira DH, Lieberman N, Ware JE: The Primary Care Asessment Survey:tests of data quality and measurement performance. Med Care. 1998, 36: 728-39. 10.1097/00005650-199805000-00012.View ArticlePubMedGoogle Scholar
- Grol R, Wensing M, Mainz J, Jung HP, Ferreira P, Hearnshaw H, Hjortdahl P, Olesen F, Reis S, Ribacke M, et al: Patients in Europe evaluate general practice care: an international comparision. Br J Gen Pract. 2000, 50: 882-7.PubMedPubMed CentralGoogle Scholar
- Grol R, Wensing M: Patients evaluate family pracitce. The Europep instrument. 2000, Nijmegen: Centre for Health Care QualityGoogle Scholar
- Baker R: The reliability and criterion validity of a measure of patients satisfaction with their general practice. Family Practice. 1991, 8: 171-7.View ArticlePubMedGoogle Scholar
- Baker R, Streatfield J: What type of general practice do patients prefer? Exploration of practice characteristics influencing patient satisfaction. Br J Gen Pract. 1995, 45: 654-659.PubMedPubMed CentralGoogle Scholar
- Grogan S, Conner M, Willits D, Norman P: Development of a questionnaire to measure patients' satisfaction with general practitioners' services. BJGP. 1995, 45: 525-9.PubMedPubMed CentralGoogle Scholar
- Luthra M, Marshall MN: How do general practices manage requests from patients for "same-day" appointments? A questionnaire survey. BJGP. 2001, 51: 39-41.PubMedPubMed CentralGoogle Scholar
- Murray M: Patient care: access. BMJ. 2000, 320: 1594-6. 10.1136/bmj.320.7249.1594.View ArticlePubMedPubMed CentralGoogle Scholar
- Stott NCH, Kinnersley P, Elwyn G: Measuring general practice-based primary care: generic outcomes. Family Practice. 1997, 14: 486-491. 10.1093/fampra/14.6.486.View ArticlePubMedGoogle Scholar
- Guthrie B, Wyke S: Does continuity in general practice really matter?. BMJ. 2000, 321: 734-6. 10.1136/bmj.321.7263.734.View ArticlePubMedPubMed CentralGoogle Scholar
- Kaplan RS, Norton DP: Using the balanced scorecard as a strategic management system. Harvard Business Review. 1996, 75-85.Google Scholar
- Campbell J, Proctor S: A developmental performance framework for primary care. International Journal of Healthcare Quality Assurance. 1999, 12: 279-286. 10.1108/09526869910287549.View ArticleGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2296/4/8/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article: verbatim copying and redistribution of this article are permitted in all media for any purpose, provided this notice is preserved along with the article's original URL.