Methodology for Developing Quality Indicators for the Care of Older People in the Emergency Department

Melinda Martin-Khan; Ellen Burkett; Linda Schnitker; Richard N Jones; Leonard C Gray


BMC Emerg Med. 2013;13(23) 

In This Article

Phase 1: Review of the Literature


The purpose of this phase is to develop a preliminary QI set through a process of evaluation of available scientific literature, analysis of data collected from a pilot study,[32] and finally, expert panel input. There will be a focus on utilising structural, process and outcome measures. Specific areas of interest include: triage, clinical assessment, cognition and cognitive assessment, delirium, palliative care, medication and other geriatric specific syndromes; the expert panel will be able to nominate additional topic areas believed to be of high priority.

Expert Panel

A range of stakeholders will be sought to establish the ED expert panel. The study team developed a list of stakeholder categories to identify the range of expertise required, such as physicians, nurses, dementia specialists or QI development experts (Table 1). In the first instance, one representative from each data collection site (field study) will be invited to participate in the panel. Purposeful sampling will follow, to populate each category with at least one representative. The total panel will include 12–18 participants. Potential participants will be contacted by email with an explanation of the study and an invitation to join the expert panel. Panel members will be required to participate in two face-to-face expert panel meetings and a formal voting process, which will be conducted after the second panel meeting. Final Distribution of panel members is noted in Table 1.


The scientific literature will be evaluated systematically to address 4 core concept areas:

  1. Profile of elderly patients presenting to EDs including: patient characteristics; presenting complaints; discharge diagnoses; discharge destinations; predictors of failed discharge from ED in elderly; predictors of morbidity & mortality within 28 days subsequent to ED discharge of elders

  2. Descriptors of best practice in assessment and management of geriatric ED patients, in terms of process, environment and structure including strength of relationship of each to desired outcomes

  3. Existing QIs for elderly patients in ED and, where relevant, non-ED settings

  4. Quality management in ED including: structure and feasibility of QIs; barriers to achieving quality of care in EDs; benchmarking in EDs; quality improvement projects in EDs.

National Health and Medical Research Council (NHMRC) guidelines for systematic review of scientific literature will be followed for each core concept.[33] This will include the identification of relevant MeSH/search terms; a search of the peer-reviewed and gray literature; and a hand search of bibliography and reference lists. Using the identified literature, a preliminary list of potential domains for sourcing QIs will be formulated (EB, LS). The resultant literature summary and the preliminary list of potential QI-domains will then be distributed to an expert panel for review, and to initiate discussion at the expert panel meeting.

The first time, the expert panel will meet for two days. The meeting will commence with a presentation of the study, an overview of QI development methodology and a discussion of potential data collection tools. For the remaining time, the Chair (MMK) will lead the panel through a formal process of review for each domain. This will include: a general discussion of the literature; review of existing QIs (if any) with suggestions for modification if required; consideration of new potential QIs based on the study team's review of the literature;[12,34] and opportunity for the panel to recommend new QIs. The resultant preliminary indicators will aim to encompass assessment of emergency department structure (including the physical environment and the policies related to the care of older persons), process and outcomes.

Data Collection

Throughout the meeting, three scribes will record decisions and concepts resulting from the discussion; and each panel member will informally rate potential QIs based on three criteria, including validity, significance, and responsibility. These ratings will be recorded on individual data collection sheets. This will be used as an additional resource to ensure that the scribes captured all relevant discussion points.

Data Compilation

After completion of the first expert panel meeting, three investigators (EB, LS, MMK) will review all the preliminary indicators. A working manual for each indicator set will be established (structural, process, outcome). Each preliminary QI will be defined - this includes detailed specification of the numerator, denominator, exclusion characteristics and any factors that will be significant for risk adjustment. The feedback from the expert panel will be incorporated into the manual alongside each indicator. Any preliminary indicators rejected at the expert panel meeting as clearly unsuitable will be recorded, along with the justification for exclusion, in a separate manual, known as The Excluded Indicators Manual.