Association Between Trauma Quality Indicators and Outcomes for Injured Patients FREE

Traumatic injuries are the leading cause of death in patients younger than 45 years and are the fifth most common cause of mortality overall in the United States.¹ In 2000, 10% of hospital discharges were owing to injuries, and the direct cost of treating 50 million injury cases was $80.2 billion, with an estimated additional $326 billion in indirect costs.² In light of the substantial mortality, morbidity, and cost of caring for injured patients, improving the care of trauma patients is an important national priority.³

The release of the landmark Institute of Medicine report on medical errors⁴ has resulted in public and private initiatives to improve health care quality through performance measurement and promoting adherence to evidence-based practices.^5- 6 The wide variability in clinical outcomes across hospitals treating trauma patients,^7- 8 even among designated level I trauma centers,⁹ is evidence of large gaps in trauma care quality across hospitals. It is unknown to what extent this variability is caused by differences in clinical practice across trauma centers. More than 10 years before the release of the Institute of Medicine report, the American College of Surgeons Committee on Trauma (ACSCOT) created a set of quality indicators (audit filters), based on expert consensus,^10- 11 to measure adherence to best practices and facilitate quality improvement.¹² In 2006, ACSCOT created the Trauma Quality Improvement Program (TQIP) to develop a national reporting and quality improvement program for trauma patients.³ As part of its mandate, TQIP will seek to identify and validate best practices in trauma care¹³ and has proposed several candidate process measures for evaluation and inclusion in TQIP.³

Because process measures directly measure clinical practice, they are actionable; poorly performing hospitals can focus on improving compliance with a specific measure (eg, timely administration of antibiotics). Process measures are also an effective mechanism for achieving rapid adoption of new evidence into clinical practice and promoting standardization.¹⁴ However, process measures must first be validated to ensure that improved adherence to a recommended practice standard is associated with optimal patient outcomes. Recent negative findings from studies examining the results of 2 national patient safety projects, Leapfrog Safe Practices Scores⁶ and the Surgical Care Improvement Project,¹⁵ highlight the need to establish a strong evidence-based link between practice standards and clinical outcomes before adopting them as performance measures.

In addition to the process measures proposed by the American College of Surgeons TQIP, recent comprehensive reviews of existing trauma quality indicators have also identified candidate process measures for evaluating trauma center quality.^16- 17 Many of the proposed candidate measures are similar to existing ACSCOT audit filters. The 2 largest previous studies investigating trauma audit filters are based on data nearly 20 years old and may not reflect contemporary clinical practice.^18- 19Quiz Ref IDAs part of an Agency for Healthcare Research and Quality–funded study to help develop the national infrastructure for trauma quality reporting,⁷ the current study examined the association between existing quality indicators and clinical outcomes using a large population-based trauma outcomes registry. We used data from Pennsylvania trauma centers to examine whether ACSCOT audit filters are associated with mortality and morbidity. Data on the validity of these expert-based performance measures may help inform the development of future trauma performance measures.

This study was based on data from the Pennsylvania Trauma Outcome Study (PTOS), obtained from the Pennsylvania Trauma Systems Foundation, on patients with traumatic injuries admitted to Pennsylvania trauma centers between 2000 and 2009. The Pennsylvania population, which makes up 4.2% of the US total population and includes rural and urban areas, provides a representative case mix of injured patients.²⁰Quiz Ref IDThe PTOS database is a statewide trauma registry that includes data on all trauma admissions at accredited trauma centers meeting PTOS inclusion criteria: admission to the intensive care unit or step-down unit, hospital length of stay greater than 48 hours, hospital admissions transferred from another hospital, and transfers to an accredited trauma center.²¹ The PTOS database includes deidentified data on patient demographics, Abbreviated Injury Score codes and International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes, mechanism of injury (based on ICD-9-CM E codes), comorbidities, physiology information, mechanisms of injury, in-hospital mortality and complications, transfer status, processes of care, and encrypted hospital identifiers.

The study population consisted of trauma patients older than 16 years in the PTOS database, after excluding patients with burns, hypothermia, isolated hip fractures, superficial injuries, unspecified injuries, and nontraumatic mechanism of injury. From this initial cohort of 228 851 patient observations, we excluded patients with missing information on transfer status (291), demographics (174); invalid Abbreviated Injury Score codes (12 732); and patients transferred out to another hospital (4712). The final study cohort consisted of 210 942 patients in 35 trauma centers. This study was approved by the institutional review board at the University of Rochester School of Medicine.

The goal of this study was to estimate the association between the ACSCOT audit filters (Table 1) and (1) in-hospital mortality and (2) death or major complication. The unit of analysis was the patient. Initial exploratory analyses were conducted using univariate logistic regression.

In the first set of analyses, separate logistic regression models were estimated to test the independent effect of each of the ACSCOT audit filters on in-hospital mortality. We used the previously validated Trauma Mortality Probability Model (TMPM-AIS)²² modified by the addition of age, sex, comorbidities, mechanism of injury (based on E codes), transfer status, the Glasgow Coma Scale (GCS) motor component score, systolic blood pressure, and year of admission to control for confounding. Backward stepwise selection was used to determine which of the comorbidities to include. Fractional polynomial analysis was used to determine the optimal specification for continuous predictor variables.²³ The STATA implementation of the MICE method of multiple imputation described by van Buuren et al²⁴ was used to impute missing values of the motor component of the GCS and systolic blood pressure. Model parameters estimated in the 5 imputed data sets were combined using Rubin's rule.²⁵ Each analysis was based on the applicable trauma patient population eligible for a particular audit filter (Table 2).

In the second set of analyses, we tested the independent association between each of the ACSCOT audit filters and death or major complications. We defined this composite complication outcome if any of the following occurred after hospital admission: death, acute respiratory distress syndrome, acute myocardial infarction, acute respiratory failure requiring more than 48 hours of ventilatory support, aspiration pneumonia, pneumonia, pulmonary embolism, fat embolism syndrome, acute renal failure, central nervous system infection, progression of original neurological insult, liver failure, sepsis, septicemia, empyema, dehiscence, gastrointestinal bleeding, small-bowel obstruction, compartment syndrome, arterial occlusion, and postoperative hemorrhage. We controlled for age, sex, injury severity, mechanism of injury, the motor component score of the GCS, systolic blood pressure, comorbidities, transfer status, and year of admission, using the modified TMPM-AIS, as described earlier.

Data management and statistical analyses were performed using STATA SE/MP version 11.0 (StataCorp). Robust variance estimators were used to account for the nonindependence of observations within hospitals. All statistical tests were 2-tailed and P values less than .05 were considered significant. The performance of the logistic regression models (for mortality) was assessed using measures of discrimination (C statistic) and calibration (calibration curves).

The study sample consisted of 210 942 patients admitted to 35 trauma centers in Pennsylvania between 2000 and 2009. Population characteristics are displayed in Table 3. The median age of patients in the study sample was 47 years and 63% were male. Twenty-seven percent of the patients were transferred from another hospital. The 3 leading causes of injury were blunt trauma (41.4%), motor vehicle collisions (26.4%), and low falls (13.7%). Quiz Ref IDThe overall mortality rate for this population was 6.3% and the rate of major complications was 7.2%. The median length of stay was 4 days. Gunshot wounds were associated with the highest overall mortality rates (24%) and the highest rate of death or major complications (34%). Patients with GCS motor component scores between 1 and 4 had mortality rates ranging from 22% to 46%. Patients with significant hypotension (systolic blood pressure <90 mm Hg) had mortality rates between 21% and 95%.

The number of patients eligible for each of the audit filters and proportion of patients flagged for nonadherence are listed in Table 2. The rate of missing data exceeded 10% for 5 of the audit filters, primarily because of missing time elements. The results of the analyses for these audit filters are presented in the Tables but will not be discussed further.

Unadjusted and adjusted outcomes for patients flagged by the ACSCOT audit filters compared with unflagged patients are displayed in Table 2 and Table 4. The statistical performance of the mortality and major complication or death models used to adjust for confounding were examined. Both models exhibited excellent calibration, as shown in Figure 1 and Figure 2. Both the TMPM mortality model and the TMPM death or major complication model had excellent discrimination, with C statistics of 0.96 and 0.89, respectively.

Twenty percent of patients with an admission GCS score less than 13 did not receive a head computed tomography (CT) scan. This filter was associated with large increases in mortality (adjusted odds ratio [AOR], 4.39; 95% confidence interval [CI], 3.18-6.07) and death or major complications (AOR, 2.76; 95% CI, 2.05-3.72).

Of those patients admitted with a gunshot wound to the abdomen, 8% were flagged because they were managed nonoperatively. These patients had a nearly 5-fold increase in their risk of mortality (AOR, 4.80; 95% CI, 2.95-7.81).

Fifteen percent of trauma patients with femoral diaphyseal fractures were treated with nonfixation. These patients had a 4-fold increased risk of mortality (AOR, 4.08; 95% CI, 2.50-6.66) and greater than 2-fold greater risk of death or major complication (AOR, 2.54; 95% CI, 1.96-3.31).

Less than 1% of patients with an emergency department (ED) discharge GCS score of 8 or less left the ED without a definitive airway. Flagged patients were at higher risk of death (AOR, 1.33; 95% CI, 0.97-1.82) but were less likely to experience the composite outcome of death or major complication (AOR, 0.63; 95% CI, 0.47-0.85).

Only 6% of trauma patients were admitted by nonsurgeons. This audit filter was an independent predictor of a lower risk of death or major complication (AOR, 0.75; 95% CI, 0.65-0.86).

The percentage of patients requiring a laparotomy that was not performed within 2 hours of ED arrival was 19%. Delayed laparotomy was associated with reduced risk of mortality (AOR, 0.74; 95% CI, 0.60-0.91) but a higher risk of death or major complication (AOR, 1.42; 95% CI, 1.19-1.69).

Nearly 40% of patients transported to the hospital from the hospital scene by ambulance or helicopter were missing an ambulance record in the medical record. Patients flagged with missing ambulance records had a lower in-hospital mortality rate (AOR, 0.79; 95% CI, 0.72-0.88).

Hourly documentation of vital signs in the ED was absent in almost 4% of the patients. Missing ED documentation was associated with lower mortality (AOR, 0.75; 95% CI, 0.69-0.82) and a lower rate of death or major complications (AOR, 0.76; 95% CI, 0.69-0.83).

Approximately 7% of patients experienced a major complication. There was a nonsignificant trend toward a higher mortality rate in this patient group (AOR, 1.18; 95% CI, 0.98-1.42).

Patients who were flagged because they required reintubation within 48 hours of extubation (8%) were less likely to die (AOR, 0.45; 95% CI, 0.37-0.54) but experienced a nearly 5-fold increase in the risk of the composite outcome of death or major complication (AOR, 4.46; 95% CI, 3.47-5.75).

Less than 1% of patients were flagged because of a discharge diagnosis of cervical spine injury that was not diagnosed on admission. These patients had a 2-fold higher risk of death or major complications (AOR, 1.96; 95% CI, 1.41-2.74).

Quiz Ref IDIn this large population-based observational study of the association between the ACSCOT audit filters and clinical outcomes, we find evidence that 6 of the audit filters are predictive of increased mortality or the composite outcome death or major complications. For some of the filters, the clinical impact appears to be very strong. For example, trauma patients with an admission GCS score less than 13 who did not receive a head CT scan had a 4-fold increased risk of mortality and a nearly 3-fold higher risk of death or major complications. Similarly, patients admitted with a gunshot wound to the abdomen who were managed nonoperatively experienced a 5-fold higher odds of mortality compared with those undergoing surgery. In comparison, there is no evidence that incomplete documentation is associated with worse outcomes.

The ACSCOT audit filters, first introduced in 1987 and then subsequently revised in 1990 and 1993, are based on expert consensus.^11,18 In comparison with many of the process measures developed by the American College of Cardiology, trauma process measures are not based on data from large multicenter randomized controlled trials.¹⁶ Process measures can be an effective means of promoting best practices²⁶ and improving quality^27- 28 and have been adopted by several large national initiatives to improve quality of care across a wide spectrum of medical and surgical conditions.^15,28 However, in some cases, the absence of a strong evidence base linking specific processes of care and clinical outcomes limits the value of performance measures as a tool to improve the quality of health care.^15,29- 30 Viewed in this light, ongoing efforts by the ACSCOT to evaluate process measures for inclusion in TQIP may benefit from the findings of our exploratory analysis examining the association between existing ACSCOT trauma audit filters and outcome.

To our knowledge, there are only 2 other large-scale observational studies that have investigated the link between the ACSCOT audit filters and outcomes.^18- 19 Both of these studies were conducted more than 15 years ago, and one did not adjust for differences in case mix.¹⁸ The other study, by Copes and colleagues,¹⁹ was also based on the PTOS database and used a case-control design to control for case mix differences. However, 80% of the audit filters examined by Copes et al did not have matches for 20% or more of the patients flagged by the audit filter. Ignoring these observations in the case-control study may have introduced significant bias.³¹ The lack of risk adjustment in one study, and the large amount of missing matches in the second, makes it difficult to compare our study findings with those of these prior studies. In addition, the period separating our study from prior studies limits the utility of such a comparison.

Process measures can be used to improve quality and clinical outcomes by promoting adherence to best practices.²⁶ Unlike outcome measures, which can identify a quality problem but not its root cause, process measures are directly actionable because they quantify adherence to a practice guideline based on best practices.³² However, the Achilles' heel of process measures is the lack of strong evidence supporting large areas of clinical practice.³² Even in the field of cardiology, which has a very strong evidence base grounded in many large experimental trials, nearly 50% of the practice guidelines developed by the American College of Cardiology are based on the lowest quality of evidence.³³ The increasing pressure to create process measures for hospitals and physicians and link these to financial incentives through pay for performance and Centers for Medicare & Medicaid Services value-based purchasing³⁴ may have important unintended consequences.³⁵ Recent research findings on the effectiveness of measures designed to reduce the incidence of surgical infections,¹⁵ control blood glucose levels, and improve perioperative cardiac outcomes through the use of β-blockers³⁶ highlight the potential downside of creating performance measures based on incomplete evidence. The challenge facing the ACSCOT in creating process measures for TQIP is to move beyond expert consensus. To do so, the National Trauma Databank should be expanded to include more information on processes of care to facilitate comparative effectiveness research. Beyond this, trauma surgeons may need to consider testing the efficacy of best practices, based on the initial results of comparative effectiveness research, using large multicenter randomized controlled trials. Results of these trials may help provide the evidence base for creating the next generation of ACSCOT quality measures.

Quiz Ref IDThe implications of our findings with respect to the future development of process measures for trauma care are not straightforward. Although some of the American College of Surgeons audit filters are strongly associated with increased mortality, these same audit filters may not be suitable as performance measures. For example, despite the finding that there is a strong association between mortality and no head CT scan, it is likely that in many cases CT scanning was not performed because other aspects of care were assigned a higher clinical priority. In particular, patients requiring emergent laparotomy to control hemorrhage may not have time for a head CT scan on admission. Nonfixation of femur fractures may also reflect the need to prioritize clinical care in complex patients with multiple injuries. These issues will need to be addressed by TQIP as they evaluate several proposed measures such as the use of intracranial pressure monitoring in patients with severe traumatic brain injury, time to operative fixation in patients with long bone fractures, and time to hemorrhage control.³⁷ Failure to carry out specific best practices in individual patients may reflect appropriate clinical decision making in complex patients with multiple injuries as opposed to substandard care.

This study has several limitations. First, this study is based on data from Pennsylvania and may not be generalizable to the rest of the United States. Alternative data sources such as the National Trauma Databank and the Healthcare Cost and Utilization Project Nationwide Inpatient Sample, however, do not include the necessary patient-level information on the ACSCOT audit filters. Second, this study, like other observational studies, is potentially biased owing to unmeasured confounders. For example, the findings that patients with an abnormal GCS score who do not receive a head CT scan are more likely to die may reflect confounding by indication; patients with abnormal neurological examination findings who did not undergo CT scanning may have been too unstable to spend time in the radiology suite as opposed to dying because they did not receive indicated care. It is unlikely that a propensity-based analysis would eliminate this potential source of bias, since propensity scoring can only balance groups using available data and cannot adjust for unmeasured risk factors. Although it is impossible to rule out bias as a threat to the internal validity of this study, the excellent statistical performance of our risk adjustment model reduces the likelihood of this type of bias.

Third, we did not explore the association between the ACSCOT audit filters and functional outcomes. As captured by the Institute of Medicine definition of quality, the goal of health care is to “increase the likelihood of desired health outcomes.”³⁸ Future work to validate revised trauma performance measures should include functional outcomes as one of the quality domains.

Fourth, the ACSCOT audit filters were first designed as quality indicators to identify patients for peer review, as opposed to performance measures to use for hospital benchmarking. However, as is the case with the Agency for Healthcare Research and Quality Patient Safety Indicators, quality indicators originally intended to facilitate peer review are now routinely used for performance measurement.³⁹ Thus, from a practical standpoint, assessing the validity of the ACSCOT audit filters as performance measures is reasonable. Finally, we examined the association between audit filters that were developed more than 20 years ago and whose relevance to modern trauma management may be challenged. However, some of the process measures currently under review by TQIP overlap with existing ACSCOT measures (eg, deep venous thrombosis prophylaxis and time to operative fixation for fractures). Since TQIP intends to incorporate process measures as part of its reporting system, our exploratory analysis to examine the validity of the ACSCOT audit filters using a contemporary data set should help facilitate the development of an updated set of trauma performance measures. The primary challenge to the construction of process measures for TQIP is the same today as it was 20 years ago: identifying best practices in trauma care using the best available evidence, which unfortunately remains quite limited.

Despite its limitations, we believe that this study will help inform efforts by the American College of Surgeons to develop new performance measures for trauma care. The large variability in outcomes across specialized trauma centers challenges the trauma community to develop standardized treatment approaches, based on best practices, to improve the overall quality of trauma care in the United States. Although performance measurement is an integral part of efforts to improve trauma outcomes, it is imperative that we focus our measurement and standardization efforts on clinical practices that have been demonstrated to lead to better outcomes. The next version of ACSCOT process measures should be based on the best available evidence and should be carefully validated before accepting them as the basis for trauma center evaluation and quality improvement. Because of the complexity of trauma care, the goal of creating evidence-based and clinically valid process measures is likely to prove very challenging.