Validation of the Emotional Content and Cooperation Score used in emergency call-taking research
|
Introduction: EMS prioritize patients based on information provided by 9-1-1 callers to emergency medical dispatchers (EMD). Call-taking effectiveness and efficiency can be adversely affected by a non-cooperative caller. Although the emotional content and cooperation score (ECCS) is the standard measure of a caller’s cooperation with EMD questioning in emergency call-taking research, the scale has not been thoroughly validated. Objective: The study aimed to measure the inter-rater agreement of the ECCS applied to medical 9-1-1 calls. Methods: Using anonymized audio recordings of actual medical 9-1-1 calls, a panel of experts developed a working definition for five levels of emotional-content: 1) normal conversation; 2) anxious but cooperative; 3) moderately upset but cooperative; 4) uncooperative, not listening, yelling; and 5) uncontrollable, hysterical. The 11 panel members then rated 15-second clips from a random sample of 220 calls. Agreement between the ratings was assessed using the kappa statistic. Results: An average of 60.8% of the scores assigned by all the panel members were a level 1 (conversational tone) and 1.7% were a level 4 (uncooperative caller). None of the callers were rated as uncontrollable. Agreement across the raters was low (K=0.32) for the 5-level scale. Dichotomising the clips into cooperative (levels 1, 2 and 3) and uncooperative (levels 4 and 5) yielded near-perfect agreement (K=0.97). Conclusions: Our results align with the call-taking literature in suggesting that a high proportion of 9-1-1 callers are cooperative when responding to an EMD. However, our low inter-rater agreement should caution researchers in the use of the 5-level ECCS, as it is questionable whether multiple raters can reliably discriminate that level of nuance. Our results suggest that a binary classification (cooperative vs uncooperative) is reliable. Future research will assess the effects of a caller’s expressed emotions on an EMD’s ability to process the call in a timely and accurate way. |
About the author:
 |
| Thérésa Choisi is co-coordinator of the Research Program at Urgences-santé, one of the largest emergency medical services in Canada. She has eight years’ experience as an Emergency Medical Dispatcher (EMD), being promoted to Team Leader, then Quality Assurance Specialist (EMD-Q). She holds a Bachelor’s degree in Health Sciences from the University René Descartes (Paris V) and an Honours Master’s degree in Experimental Health Sciences from the University of Québec. In addition to her operational expertise, she coordinates interventional, qualitative and comparative-effectiveness research, with an emphasis on data management and survey methods. Her research interests include out of hospital cardiac arrest, stroke and mental wellness of healthcare workers. Currently, she leads three studies focusing on agreement between information collected by EMD and what is observed by the paramedics and/or diagnosed at the hospital. |
Commentary