0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Research Letter | Association of VA Surgeons

Statistical Methods of Risk-Adjusted Statistical Process Control Charts to Assess Surgical Performance in Consecutive Colorectal Operations at a Single Institution FREE

Monique Boudreaux-Kelly, PhD1; Mark Wilson, MD, PhD2; Malak Bokhari, MD, MPH2
[+] Author Affiliations
1Pittsburgh Research Office, Research StatCore, Pittsburgh, Pennsylvania
2Department of Surgery, VA Pittsburgh Healthcare System, Pittsburgh, Pennsylvania
JAMA Surg. 2015;150(3):271-272. doi:10.1001/jamasurg.2014.1773.
Text Size: A A A
Published online

Risk-adjusted cumulative summation is an iterative statistical process control chart centered on 0 that allows for local and individualized risk adjustment.13 Risk-adjusted cumulative summation scores signal special-cause variation if data points cross predefined control limits. We used cumulative summation charts and individual patient-level data at the Veterans Affairs (VA) Pittsburgh Healthcare System to visualize common- and special-cause variations in surgical outcomes in a cohort of patients who underwent colorectal operations. The cumulative summation scores were adjusted for individual risk based on differences between surgical outcome observed (O) (0 = no complication; 1 = any complication) and expected (E) risk derived from the predicted probability of complication (from the VA Surgical Quality Improvement Program [VASQIP]). Risk-adjusted (O − E) cumulative summation charts based on VASQIP probability are a novel and intuitive method of objectively assessing performance in real time with a minimum of 20 patients.

We created 2 risk-adjusted cumulative summation statistic charts: 30-day postoperative morbidity (n = 193) and mortality (n = 241) among patients undergoing sequential colorectal operations between October 1, 2006, and September 30, 2012 (n = 241, abstracted for the VASQIP). If a patient was assessed to be at high risk for complications (with a score of 0.78) and if the patient did indeed have complications (score of 1), then the patient’s adjusted score would be 0.22 (1 − 0.78). However, if another patient had the same risk for complications (a score of 0.78) but did not have complications (a score of 0), then that patient’s risk-adjusted score would be −0.78 (0 − 0.78). Graphically, the presence of complications is presented as upward spikes. To test for excess risk, two 1-sided C curves, CUSUM statistics C+ and C, were calculated using each patient’s risk-adjusted cumulative summation score so as to be sensitive to the variability of excess successes (C) and failures (C+). Surrounding C+ and C were upper and lower control limits that, when crossed, indicate that variation in the curves surpassed what was expected (special-cause variation).

The cumulative summation chart for 30-day morbidity (75 complications) indicated that excess morbidity occurred between February 21, 2007, and April 3, 2007 (cases 25 and 26). Immediately afterward, there was a sharp decline in the number of complications, and the number of complications continued to decrease until case 36 (June 16, 2007), when the lower limit was crossed to indicate significantly lower-than-expected morbidity until July 17, 2007. Beyond July 17, 2007, morbidity occurred at expected rates, with actual outcome getting progressively closer to expected outcome (ie, both curves hovered closer to 0 over time) (Figure 1).

Place holder to copy figure label and caption
Figure 1.
CUSUM Graph of Risk-Adjusted Morbidity of 193 Sequential Cases
Graphic Jump Location

The curve patterns for 30-day mortality (with a total of 10 patients) showed that mortality occurred as expected up until case 96, although that did not signal special-cause variation. Excess mortality was indicated when 2 patients with a less than 10% expected risk of death died within a short time span (cases 124 and 127 in August 2009). The other 8 deaths were spread out randomly over time and did not signal excess death (Figure 2).

Place holder to copy figure label and caption
Figure 2.
CUSUM Graph of Risk-Adjusted Mortality of 241 Sequential Cases

FIR indicates fast initial response.

Graphic Jump Location

The VA uses the O to E ratio method, which estimates performance by using group-level data that produces an O to E ratio that almost always has a value greater than 0. When comparing patient-level outcomes, the ability to adjust for risk using the O to E ratio is limited for patients for whom no complications occurred, and the curve is more dependent on the number of events in a short time span, rather than being dependent on each patient’s outcome related to presurgical risk.

The visual presentation of the risk-adjusted cumulative summation chart (O − E) allows for an examination of the process in control events, in better-than-expected performance, and in out-of-control events. The use of real-time cumulative summation (O − E) complements the existing O to E ratio method to provide an informative, real-time visual representation of ongoing surgical performance. It may provide a more intuitive presentation of the process of care to health care providers and administrators.

Corresponding Author: Malak Bokhari, MD, MPH, Department of Surgery, VA Pittsburgh Healthcare System, University Drive Campus (112 U), Pittsburgh, PA 15240 (malak.bokhari@va.gov).

Published Online: January 21, 2015. doi:10.1001/jamasurg.2014.1773.

Author Contributions: Dr Bokhari had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: All authors.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Boudreaux-Kelly, Bokhari.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Boudreaux-Kelly.

Administrative, technical, or material support: Wilson, Bokhari.

Study supervision: Bokhari.

Conflict of Interest Disclosures: None reported.

Previous Presentation: This paper was presented at the 38th Annual Surgical Symposium of the Association of VA Surgeons; April 6, 2014; New Haven, Connecticut.

Maguire  T, Mayne  CJ, Terry  T, Tincello  DG.  Analysis of the surgical learning curve using the cumulative sum (CUSUM) method. Neurourol Urodyn. 2013;32(7):964-967.
PubMed   |  Link to Article
Provost  LP, Murray  SK. The Health Care Data Guide: Learning From Data Improvement. San Francisco, CA: John Wiley & Sons, Inc; 2011.
Koshti  VV.  Cumulative sum control chart. Int J Physics Math Sci. 2011;1(1):28-32. http://www.cibtech.org/J-PHYSICS-MATHEMATICAL-SCIENCES/PUBLICATIONS/2011/Vol%201%20No.%201/01-Koshti.pdf. Accessed November 20, 2014.

Figures

Place holder to copy figure label and caption
Figure 1.
CUSUM Graph of Risk-Adjusted Morbidity of 193 Sequential Cases
Graphic Jump Location
Place holder to copy figure label and caption
Figure 2.
CUSUM Graph of Risk-Adjusted Mortality of 241 Sequential Cases

FIR indicates fast initial response.

Graphic Jump Location

Tables

References

Maguire  T, Mayne  CJ, Terry  T, Tincello  DG.  Analysis of the surgical learning curve using the cumulative sum (CUSUM) method. Neurourol Urodyn. 2013;32(7):964-967.
PubMed   |  Link to Article
Provost  LP, Murray  SK. The Health Care Data Guide: Learning From Data Improvement. San Francisco, CA: John Wiley & Sons, Inc; 2011.
Koshti  VV.  Cumulative sum control chart. Int J Physics Math Sci. 2011;1(1):28-32. http://www.cibtech.org/J-PHYSICS-MATHEMATICAL-SCIENCES/PUBLICATIONS/2011/Vol%201%20No.%201/01-Koshti.pdf. Accessed November 20, 2014.

Correspondence

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

472 Views
0 Citations
×

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
PubMed Articles
Jobs