Introduction
Background
An important number of calls to medical 9‑1‑1 do not represent time-dependent emergencies. Indeed, a recent study suggested that only six percent of Emergency Medical Service (EMS) responses were for patients requiring time-critical care.Reference Levy, Crowe and Abraham1 The goal of triage in dispatching EMS is to align system response times with the needs of patients.Reference Montandon2 High-acuity patients should receive care sooner than low-acuity patients. Increased morbidity and mortality can result from under-triage when response is delayed by not dispatching the closest available ambulance as quickly as possible. Conversely, care can be delayed due to over-triage when too few ambulances are available to respond because they have been dispatched to non-emergencies.Reference O’Keeffe, Nicholl, Turner and Goodacre3,Reference Swain, Barry, Hoyle, Haywood, Cameron and Larsen4 Differentiating between patients that truly require time-dependent interventions and those who do not is challenging, yet these analyses are important as part of quality assurance to guide system initiatives focused on outcomes.Reference Ball, Williams and Smith5–Reference Hoikka, Länkimäki, Silfvast and Ala-Kokko7
Multiple types of measurements of patient acuity have been used in practice and in research; these include post-transport outcomes, vital-sign-based scores, and other prehospital markers.Reference Castro-Portillo, Soriano and López-Izquierdo8,Reference Peters, Goldberg, Hayes and Cash9
Traditionally, the most important outcome to evaluate has been survival, including survival to hospital discharge for patients transported to hospital.Reference Pap, Lockwood, Stephenson and Simpson10,Reference Turner, Siriwardena and Coster11 However, the relationship between EMS dispatch and hospital outcomes (eg, survival to hospital discharge) may reflect hospital-based care rather than prehospital treatment, where various hospital characteristics, beyond the control of the prehospital care system, significantly influence patient outcomes.Reference Fourny, Lucas and Belle12,Reference Stub, Smith, Bray, Bernard, Duffy and Kaye13 Moreover, some prehospital decisions may not alter some outcomes, as shown by unchanged mortality rates in myocardial infarction despite suboptimal dispatch allocation.Reference Matsuyama, Kiyohara and Kitamura14
Vital-sign-based scores are wide-spread and have been associated with mortality and morbidity but are limited by being fixed to a single benchmark rather than relative to a patient’s baseline. For example, the National Early Warning Score/NEWS2 score is less accurate in predicting outcomes for patients with chronic obstructive pulmonary disease (COPD) during acute exacerbations, likely because it fails to account for the physiological differences between healthy individuals and COPD patients, as well as the associated variations in vital signs.Reference Castro-Portillo, Soriano and López-Izquierdo8
Prehospital markers, such as scene disposition and transport priority, may be a better reflection of dispatch priority, are readily available, and are common in the EMS literature.Reference Ball, Williams and Smith5,Reference Grusd and Kramer-Johansen15 However, they may be confounded by social and other scene circumstances, or proximity to hospital and other transport-related issues such as traffic. An under-studied prehospital measure of the appropriateness of dispatch may be the delivery of time-dependent clinical and pharmacological interventions by first responders and paramedics. Identifying interventions which if provided, and therefore considered a necessity of care, would help identify a call which was a true emergency and should have been dispatched as such. The approach has the advantage that prehospital interventions occur in temporal proximity to dispatch, and data on interventions are readily available from call documentation (“run-sheets”). A disadvantage may be that interventions were not recognized by EMS responders as necessary and therefore not provided, or that EMS responders chose to prioritize rapid evacuation and transport (“scoop and run”).
Study Aim
The purpose of this study was to propose a consensus agreement on a list of first responder and paramedic interventions (clinical and pharmacological) indicative of a true emergency.
Methods
Study Design
The study team conducted a modified Delphi consensus process based on the RAND-UCLA Appropriateness Method, and which followed recommendations on “Conducting and Reporting Delphi Studies.”Reference Fitch, Bernstein and Aguilar16 Expert consensus approaches provide collective judgement when little to no robust evidence exists, including for the development of methodological approaches. The Delphi method is a structured systematic approach using rigorous methodology to achieve convergence and consensus.Reference Diamond, Grant and Feldman17,Reference Gattrell, Hungin and Price18 The process was limited to three rounds and the threshold for consensus was set at the typical 80%.Reference Jünger, Payne, Brine, Radbruch and Brearley19
Expert Panel Selection
The study team recruited a convenience panel of key informants using purposive sampling from paramedic systems in Alberta (Alberta Health Services [AHS] – EMS) and Quebec (Urgences-santé) Canada. The recruited informants represented a full range of scopes of practice, including medical first responders (MFRs), primary-care paramedics (PCPs), advanced-care paramedics (ACPs), EMS medical directors, and emergency care physicians. The study team’s rationale for including experts who represent all levels of training that comprise an EMS system, rather than a perhaps more traditional approach of having only physicians, is the unique perspective that paramedics and MFRs have on whether an intervention is indicative of a true emergency. The study team recognizes that the paramedic profession (often referred to as paramedicine)Reference Williams, Beovich and Olaussen20 is undergoing a period of professionalization, especially in systems that allow professionally autonomous practice, as opposed to the more traditional physician-directed model.Reference Makrides, Ross, Gosling, Acker and O’Meara21 The study team also recognizes that in many systems, paramedics play an important role, either with or without collaboration with a physician, in determining which patients ultimately access further health care and what this care may look like.Reference Lane, Blanchard, Buick, Shaw and McRae22
Because key informants were allowed to vote only on interventions included in their scope of practice, the study team included a decreasing number of key informants by scope of practice (Table 1). The study team purposefully recruited women and men with a range of work experience, a reputation for outstanding professional commitment, and availability throughout the study period. All 22 candidates who received a recruitment email accepted the invitation and provided written consent to participate.
Table 1. Composition of the Expert Panel (n = 22)
Note: Senior and Junior denote greater than or equal to five years of experience and less than five years of experience, respectively, at the designated scope of practice.
a Including medical directors and allied physicians.
Delphi studies in health research regularly include 10 to 100 key informants,Reference Diamond, Grant and Feldman17 with some suggesting a median of 17.Reference Boulkedid, Abdoul, Loustau, Sibony and Alberti23 Given the scope of the study and the need to conduct separate consensus meetings in French and English, the study team opted for a moderately sized panel to balance representation with efficiency.
At the time of writing, AHS EMS was a provincial system that served over 4.4 million people, had seven dispatch centers, over 450 primary-care and advanced-care ground vehicles, approximately 5,600 paramedics, and responded to over 320,000 emergency calls per year. Urgences-santé was a regional system that served 2.5 million people living on the islands of Montreal and Laval, Quebec. The service employed over 1,000 PCPs and over 25 ACPs who treated approximately 292,000 patients per year. In both systems, dispatch centers used the Medical Priority Dispatch System (MPDS; International Academies of Emergency Dispatch [IAED]; Salt Lake City, Utah USA).
Delphi Method
To ground the Delphi panel, prior to reviewing interventions, the authors performed a narrative review of English language literature and co-wrote a definition of time-dependent care that was presented to panel members using an on-line survey platform for free-text comments, following which key informant voted to accept or modify the definition. All comments were anonymized and returned to the key informants for context. The consensus definition was subsequently used for all rounds of the Delphi.
Delphi 1, Delphi 2, and Delphi 3
The study team performed two rounds using self-administered surveys through an on-line survey platform, and a third round using conference calls. Senior paramedics in each system reviewed the provincial medical protocols for first responders, PCPs, and ACPs to identify all clinical and pharmacological interventions included in each scope of practice. The collated lists were reviewed by the medical director in each system to ensure completeness. The interventions and their descriptions were translated by one of the authors (LdeM) and reviewed by two senior paramedics and an EMS physician for face and content validity. The bilingual catalogue of interventions was used to create the Delphi surveys, which were further tested for clarity by non-panel paramedics from each service prior to distribution to panel members. Delphi 1 presented all interventions in practice in Alberta and Quebec on October 1, 2018. Key informants scored each intervention on a scale from one (“not time-dependent”) to five (“acutely time-dependent”) and provided free-text comments. Key informants were asked to choose “not applicable” if the intervention was outside their scope of practice, not used in their service, or they had not been trained in its use. Interventions with votes over the threshold, either as time-dependent (≥80% scoring four or five) or not time-dependent (≥80% scoring one or two), were considered to have reached consensus and were removed from subsequent rounds.
In Delphi 2, key informants were provided for each intervention with summaries of rating scores, a reminder of their own rating score, and asked to rate and comment on the interventions for which the panel had not reached a consensus in the Delphi 1 vote.
In Delphi 3, any remaining interventions without consensus were resolved in real-time discussions. The conference calls occurred in two groups aligned with key informants’ language preferences (all informants chose to participate in the conference calls organized for their province). The sessions were co-moderated by LdeM (Urgences-santé sessions) and IEB (AHS sessions). Each group participated in two conference calls, a first call for clinical interventions and a second for pharmacological interventions. Each of the four calls started with a summary of interventions that had achieved consensus in Delphi 1 and Delphi 2, and which interventions remained for discussion. Iterating through each intervention, the co-moderators presented the result and comments from the Delphi 2 round, invited discussion of the time-dependence of the intervention, then called for a final vote.
Ethical Oversight
Ethical review and oversight were provided by the Conjoint Health Research Ethics Board (CHREB) at the University of Calgary (Calgary, Alberta, Canada; REB 18-1368). Participants provided written informed consent. All methods were carried out in accordance with relevant guidelines and regulations.
Results
The following definition for time-dependent care was proposed:
“A majority of patients would suffer a direct prejudice to their health or safety if the intervention, provided on its own, was not performed within eight minutes of the initial emergency services call.”
With the specifications that “a majority of patients” excludes exceptional or one-off cases; “a direct prejudice to health” includes death, permanent or long-term harm, or extreme pain or distress; and “a direct prejudice to safety” includes any situation that will lead to physical harm. Eight minutes was chosen to align with the common, if arbitrary, performance standard in EMS practice and research. Following key informant feedback, the definition was amended to read:
“A majority of patients who should receive the intervention, according to provincial protocols, would suffer a direct prejudice to their health or safety if the intervention, provided on its own, was not performed within eight minutes of the initial 9-1-1 call.”
The definition achieved approval from 18 out of 22 (81.8%) key informants from a vote with a 100% response rate. The time from the initial communication to achieving consensus was 53 days.
Delphi 1, Delphi 2, and Delphi 3
A total of 63 (29 clinical and 34 pharmacological) prehospital interventions were considered as potentially indicative of the need for time-dependent care. The clinical interventions are presented in Table 2a; the pharmacological interventions are in Table 2b.
Table 2a. Clinical Interventions Evaluated by Key Informants in Delphi 1 (n = 29)
Note: *Individual interventions in this group account for multiple scopes of practice; the scope of practice indicated is the most accessible of the grouping (All > PCP > ACP). Key informants were instructed to vote on the overall group, but only consider interventions that were within their personal scope of practice. Key informants were also invited to propose splitting groups they found problematic. Only two groups of clinical interventions were thus split: Burn Care/Bleeding Control and Suctioning/Gastric Decompression.
Abbreviations: AB, Alberta; ACP, Advanced Care Paramedic; LMA, laryngeal mask airway; NRB, non-rebreather mask; NPA, nasopharyngeal airway; OPA, oropharyngeal airway; PCP, Primary Care Paramedic; QC, Quebec.
Table 2b. Pharmacological Interventions Evaluated by Key Informants in Delphi 1 (n = 34)
Note: *Individual interventions in this group account for multiple scopes of practice; the scope of practice indicated is the most accessible of the grouping (All > PCP > ACP). Key informants were instructed to vote on the overall group, but only consider interventions that were within their personal scope of practice. Key informants were also invited to propose splitting groups they found problematic. No pharmacological interventions were split.
Abbreviations: ACP, Advanced Care Paramedic; PCP, Primary Care Paramedic.
a Quebec PCP protocols include the administration of fentanyl. However, the protocol is not practiced currently by Urgences-santé paramedics.
The response rate and proportion of interventions achieving consensus are outlined in Figure 1. After Delphi 3, the panel had reached consensus on a total of 52 out of 63 (82.5%) interventions, of which 17 (32.7%) were voted time-dependent (11 [64.7%] clinical and six [35.3%] pharmacological). All time-dependent interventions for which consensus was reached are described in Table 3. The interventions for which there was agreement to define as not time-dependent, or for which no consensus was reached, are described in the Supplementary Material (available online only).
Figure 1. Results of the Three-Round Delphi Process.
Table 3. Intervention Reaching Final Consensus (n = 17)
Note: Consensus was defined as 80% or more of the key informants voting that an intervention was time-dependent. *Individual interventions in this group account for multiple scopes of practice; the scope of practice indicated is the most accessible of the grouping (All > PCP > ACP). Key informants were instructed to vote on the overall group, but only consider interventions that were within their personal scope of practice. This includes the Heimlich maneuver and direct laryngoscopy.
Abbreviations: AB, Alberta; ACP, Advanced Care Paramedic; PCP, Primary Care Paramedic; QC, Québec.
Delphi 1 was completed in 17 days, and Delphi 2 was completed in 12 days. Completing the Delphi 3 conference calls required 65 days, with each session lasting two hours. The duration of time from the opening of Delphi 1 to final consensus was 94 days.
Discussion
Defining the Need for Time-Dependent Care
A group of 22 Canadian key informants reached consensus on the definition of time-dependent care, with the understanding that the definition would then be used to frame subsequent decisions on individual clinical and pharmacological interventions as time-dependent or non-time-dependent. After a single amendment to the original wording, the consensus definition proved useful and uncontentious with two exceptions. First, while not explicitly quantified, there was general agreement that it was rarely feasible to deliver certain interventions within an eight-minute window (eg, intravenous medications). Second, key informants did not consistently differentiate between risk severity and risk frequency as required by the “a majority of patients” specification. The panel would apply the definition strictly in some cases (eg, electrocardiogram voted as not time-dependent), and in other cases, the risk of a bad outcome, however infrequent, would prevent consensus (eg, prehospital delivery of a baby as time-dependent).
Consensus-Based Instrument
After three rounds of voting, the panel reached a consensus on 17 interventions, agreeing that 11 clinical and six pharmacological interventions were time-dependent according to the agreed-upon definition. Over one-half of these interventions are included in the MFR scope of practice, meaning that they are practiced by all levels of prehospital emergency care in Alberta and Quebec. With the exception of medium- and high-potency caloric agents, which can be administered by PCPs in Alberta, all other retained interventions are available to ACPs only.
The retained interventions constitute an instrument that can be applied to the retrospective identification of patients with time-dependent emergencies for research and quality assurance purposes. Others have subsequently published an instrument utilizing similar methodsReference Levy, Crowe and Abraham1 as part of a broader United States analysis of dispatch evaluation. They created an instrument where Medical Directors or designates participated in a two-round Delphi with reducing inclusion thresholds. Their definition of “time critical” was relatively less stringent than this study’s definition of time-dependent: “intervention required in a matter of minutes to save life or maintain essential functions….” Ninety-four individual time-critical interventions are included in the Levy, et al (2025) instrument, compared to 17 time-dependent intervention categories retained here. All the intervention categories in the study’s instrument are included in the Levy, et al instrument, except for caloric agents (D50W, D25W, D10W). There were 11 intervention categories listed in Levy, et al that were not included here, most relating to advanced airway or the provision of a drug facilitated advanced airway, and other specific medications in the fibrinolytic, antifibrinolytic, antidote, bronchospasm, and narcotic antagonist categories. The Levy, et al instrument also included intraosseous infusion and pelvic binding as time critical. While the two lists are very similar, the differences observed likely relate to the different definitions of time-critical and time-dependent between Levy, et al and this study, respectively. For example, the advanced airway category was contentious as many suggested advanced airway is likely not time-dependent in all situations, but it could be if basic interventions were not sufficient. This study’s expert panel also recognized that basic airway maneuvers would always proceed advanced airway, and that practically an advanced airway, let alone a drug facilitated advanced airway, would rarely be possible in under eight minutes.
Because an increasing number of EMS systems face the challenge of higher call-volumes and staffing shortages, dispatching emergency responses to a large number of calls that do not require time-dependent interventions reduces the resources available for true emergencies.Reference Kim, Sohn, Hong, Song and Shin24,Reference Jin, Chen, Ge, Li, Zhang and Ma25 Conversely, reliably identifying patients not requiring emergency response opens up alternative pathways of care, such as guided self-care, referral to community clinics, telemedicine, and non-ambulance transport to hospital.Reference Blodgett, Robertson, Pennington, Ratcliffe and Rockwood26 This instrument could be used to explore if and why mismatches occur between the triage and dispatch of calls, and the subsequent interventions provided by responders on scene. Simple reasons why a mismatch may occur include inadequate or incorrect information provided by the caller, a change in patient status during the interval of EMS call and arrival of paramedics, errors in the application of the triage system, or issues with the triage system itself. Even when triage is done accurately, patients may not receive timely interventions because of operational limitations, such as offload delays, surges in call-volume resulting in inadequate available resources, staffing, and vehicle procurement issues.Reference Kim, Sohn, Hong, Song and Shin24–Reference Blanchard, Williamson and Hagel27 Regardless of the reason, identifying patients requiring time-dependent interventions should be an imperative for all EMS systems. Clinical practice and patient improvements would logically follow improvements in triage and timely delivery of care. The logical next step for further research could explore if time-dependent calls are being missed by this list of interventions and why. Robust, valid, and reliable administrative data, which are in short supply in most systems, and/or structured expert review of individual patients based on chart-level data would be required as the gold standard.
Limitations
This study’s expert panel was drawn from only two paramedic services, one representing a large-volume urban system, and the other a provincial system including urban, sub-urban, rural, and remote communities. The consensus reached by the panel may not reflect the opinion of key informants from other systems or geographies. The time constraint of eight minutes may not be a standard achievable outside of urban centers.
Using the typical threshold for Delphi studies (80% of key informants), consensus, whether on time-dependence or non-time-dependence, was reached for 83% of interventions. Moreover, there was a very low attrition rate, albeit at the cost of a relatively lengthy consultation process.
The study team was unable to obscure the level of practice of key informants during the Delphi 3 due to voting being constrained by level of practice. The study team has worked to minimize “dominant voice” by ensuring that each panel member was invited explicitly to speak and varying the sequence in which they had an opportunity to speak. Despite the limitation of running the final round as conference calls, the opportunity for real-time discussion between members of the panel provided invaluable insight into the thought processes behind the categorization of interventions, which would not have been achievable using standard Delphi methods.
As with any study using the Delphi technique, this study was limited by the fact that the Delphi process relies on expert knowledge. Therefore, its outcomes can only be as valid as the participating key informants and the information on which they base their opinions.Reference Gattrell, Hungin and Price18 Since the Delphi, there has been only one change to the medical guidelines that may have affected the final list of time-dependent interventions. That is the use of sodium bicarbonate for cerebral herniation syndrome in Alberta; it is possible this additional indication would have influenced the panel.
Though the study team provided the key informants with a definition of time-dependent care, elements of the definition may have been interpreted differently between key informants (eg, majority of patients, extreme pain or distress, situation that will lead to physical harm), which may have introduced heterogeneity into the responses.
Finally, and as previously described, like all methods to determine time-dependent emergencies, there are limitations in this approach. For example, there may be a mismatch between patient need for a time-dependent intervention, and the patient receiving a time-dependent intervention from geographic and operational limitations that preclude the paramedic from actually delivering the intervention (eg, unrecognized indication for intervention, load-and-go, proximity to hospital, limited Advanced Life Support trained personnel). The study team believes that this risk is low, but may support combining this approach with other instruments when data are available.
Conclusions
By using a structured approach to achieving consensus – first on a working definition of the need for time-dependent care, second on clinical and pharmaceutical interventions indicative of time-dependent patients – the study team developed an instrument that can be used retrospectively to determine a true medical emergency. In combination with outcome measures, vital-sign-based instruments, or prehospital markers, or by itself, this expert consensus derived, intervention-based instrument could be used to identify time-dependent patients.
Acknowledgements
Thanks first and foremost to the panel of experts who contributed generously, reliably, and with good humor. Thanks also to Chief Darren Sandbeck, Dr. Mark MacKenzie, and Dr. Gilles Bourgeois for their unwavering support of paramedic research. Finally, thanks to the paramedics, the technician, and the allied physician who acted as beta-testers and domain experts: Jennifer Allison, Éric Bruneau, Nathan Hart, Victor Kang, Éric Lareau, Sébastien Légaré, Alex Onos, Ian Pratt, and Eli Segal; and also to Pat Weigel for administrative support.
Conflicts of interest
Please note that none of the authors have any conflicts of interest in the conduct of this research.
Open access
