0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Research Letter |

Avoiding Immortal Time Bias in the American College of Surgeons National Surgical Quality Improvement Program Readmission Measure FREE

Donald J. Lucas, MD, MPH1; Elliott R. Haut, MD2; Elizabeth M. Hechenbleikner, MD3; Elizabeth C. Wick, MD2; Timothy M. Pawlik, MD, MPH, PhD2
[+] Author Affiliations
1Department of Surgery, Walter Reed National Military Medical Center, Bethesda, Maryland
2Department of Surgery, Johns Hopkins Hospital, Baltimore, Maryland
3Department of Surgery, Georgetown University Hospital, Washington, DC
JAMA Surg. 2014;149(8):875-877. doi:10.1001/jamasurg.2014.115.
Text Size: A A A
Published online

Readmission has become a key quality metric because it is a frequent and costly adverse event for patients.1 Medicare penalizes hospitals if they have excess numbers of readmissions for certain diagnoses, including some within surgery. The American College of Surgeons National Surgical Quality Improvement Program (NSQIP) began tracking readmission rates in 2011.2

While conducting research on readmission after surgery, we noted a problem with the NSQIP’s definition of readmission.3 The NSQIP only counts readmissions during the 30 days following surgery, consistent with the interval they use for all postoperative outcomes. However, the standard period for readmission used by Medicare and others is 30 days after hospital discharge. This discrepancy creates an immortal time bias—patients cannot be readmitted before hospital discharge and are therefore “immortal” for this outcome until they leave the hospital.4 Including immortal time when calculating readmission rates leads to a bias between the measured value and the true value for readmission rates. Logically, 30-day postoperative readmission should systematically undercount 30-day postdischarge readmission. In addition, this bias should worsen for a longer length of stay. In our previous NSQIP analysis, we noted that readmission rates increased with longer lengths of stay, peaked at 13 days, and decreased thereafter. We hypothesized that readmission rates would increase continuously with longer lengths of stay if this immortal time bias were not present. Here we report data supporting this hypothesis and demonstrate how to avoid this bias.

We examined patients who had a colorectal resection at Johns Hopkins Hospital in Baltimore, Maryland, between 2009 and 2011, survived to hospital discharge, and were captured by the NSQIP. Dates of admission, discharge, and readmission were obtained from the NSQIP, medical record review, and administrative data, as previously described.5 The NSQIP 30-day postoperative readmission rate was compared with the standard 30-day postdischarge readmission rates obtained from medical record review. A Kaplan-Meier method to estimate the true 30-day postdischarge readmission rate using only data collected by the NSQIP was then evaluated.6 Our study was approved by the Johns Hopkins Hospital institutional review board. Consent was waived owing to the retrospective nature of the analysis. Patients were not compensated.

We identified 708 patients, whose characteristics were described previously.6 Of these 708 patients, 134 (18.9%) were readmitted within 30 days after hospital discharge, but only 114 (16.1%) were readmitted within 30 days after surgery. Therefore, the NSQIP method had a sensitivity of 85% for 30-day postdischarge readmission in this population. This undercounting worsened with longer lengths of stay and was statistically significant at 25 days (P = .01, determined by use of the Fisher exact test) (Figure).

Place holder to copy figure label and caption
Figure.
True vs NSQIP 30-Day Readmission Rates

The National Surgical Quality Improvement Program (NSQIP) method (30-day postoperative readmission) undercounts true 30-day postdischarge readmission. This effect worsens with longer lengths of stay and is statistically significant at 25 days or longer (P = .012, determined by use of the Fisher exact test).

Graphic Jump Location

The NSQIP began reporting the day of readmission in 2012. This information can be used to overcome the immortal time bias. Truncating our data at 30 days after surgery as if it were from the NSQIP, we estimated the 30-day postdischarge readmission rate using Kaplan-Meier methods. This resulted in an estimated readmission rate of 17.2% (95% CI, 14.5%-20.3%), which included the true value of 18.9%.

The NSQIP systematically undercounts 30-day postdischarge readmissions, and this bias worsens with longer lengths of stay. The Medicare Hospital Readmissions Reduction Program enforces larger and more widespread financial penalties than any other quality measure of any type, and it is expanding into surgery. The NSQIP and other programs like it, which are designed to help hospitals improve quality, should “teach to the test” and adopt the 30-day postdischarge readmission definition. Until this occurs, research on readmission using NSQIP data should use Kaplan-Meier methods to avoid underestimating readmission rates.

Corresponding Author: Timothy M. Pawlik, MD, MPH, PhD, Department of Surgery, Johns Hopkins Hospital, 600 N Wolfe St, Blalock 688, Baltimore, MD 21287 (tpawlik1@jhmi.edu).

Published Online: July 2, 2014. doi:10.1001/jamasurg.2014.115.

Author Contributions: Drs Lucas and Pawlik had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Lucas, Haut, Pawlik.

Acquisition, analysis, or interpretation of data: Lucas, Hechenbleikner, Wick, Pawlik.

Drafting of the manuscript: Lucas, Hechenbleikner, Pawlik.

Critical revision of the manuscript for important intellectual content: Lucas, Haut, Wick, Pawlik.

Statistical analysis: Lucas, Pawlik.

Administrative, technical, or material support: Hechenbleikner.

Study supervision: Haut.

Conflict of Interest Disclosures: Dr Haut receives royalties from Lippincott Williams & Wilkins for a book he coauthored (Avoiding Common ICU Errors). He has received honoraria for various speaking engagements regarding clinical and quality and safety topics and has given expert witness testimony in various medical malpractice cases. No other disclosures are reported.

Disclaimer: The views expressed in this article are those of the authors and do not represent the official policy of the US Navy, the Department of Defense, or the US government.

Additional Information: Dr Haut is the primary investigator of the Mentored Clinician Scientist Development Award K08 1K08HS017952-01 from the Agency for Healthcare Research and Quality entitled “Does Screening Variability Make DVT an Unreliable Quality Measure of Trauma Care?” Dr Haut is the primary investigator of a contract with The Patient-Centered Outcomes Research Institute entitled “Preventing Venous Thromboembolism: Empowering Patients and Enabling Patient-Centered Care via Health Information Technology.”

Jencks  SF, Williams  MV, Coleman  EA.  Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med. 2009;360(14):1418-1428.
PubMed   |  Link to Article
American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP). User guide for the 2011 Participant Use Data File. http://site.acsnsqip.org/wp-content/uploads/2012/03/2011-User-Guide_Final.pdf. Published October 2012. Accessed May 28, 2013.
Lucas  DJ, Haider  A, Haut  E,  et al.  Assessing readmission after general, vascular, and thoracic surgery using ACS-NSQIP. Ann Surg. 2013;258(3):430-439.
PubMed   |  Link to Article
Suissa  S.  Immortal time bias in pharmaco-epidemiology. Am J Epidemiol. 2008;167(4):492-499.
PubMed   |  Link to Article
Hechenbleikner  EM, Makary  MA, Samarov  DV,  et al.  Hospital readmission by method of data collection. J Am Coll Surg. 2013;216(6):1150-1158.
PubMed   |  Link to Article
Kaplan  EL, Meier  P.  Nonparametric estimation from incomplete observations. J Am Stat Assoc. 1958;53(282):457-481. doi:10.1080/01621459.1958.10501452.
Link to Article

Figures

Place holder to copy figure label and caption
Figure.
True vs NSQIP 30-Day Readmission Rates

The National Surgical Quality Improvement Program (NSQIP) method (30-day postoperative readmission) undercounts true 30-day postdischarge readmission. This effect worsens with longer lengths of stay and is statistically significant at 25 days or longer (P = .012, determined by use of the Fisher exact test).

Graphic Jump Location

Tables

References

Jencks  SF, Williams  MV, Coleman  EA.  Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med. 2009;360(14):1418-1428.
PubMed   |  Link to Article
American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP). User guide for the 2011 Participant Use Data File. http://site.acsnsqip.org/wp-content/uploads/2012/03/2011-User-Guide_Final.pdf. Published October 2012. Accessed May 28, 2013.
Lucas  DJ, Haider  A, Haut  E,  et al.  Assessing readmission after general, vascular, and thoracic surgery using ACS-NSQIP. Ann Surg. 2013;258(3):430-439.
PubMed   |  Link to Article
Suissa  S.  Immortal time bias in pharmaco-epidemiology. Am J Epidemiol. 2008;167(4):492-499.
PubMed   |  Link to Article
Hechenbleikner  EM, Makary  MA, Samarov  DV,  et al.  Hospital readmission by method of data collection. J Am Coll Surg. 2013;216(6):1150-1158.
PubMed   |  Link to Article
Kaplan  EL, Meier  P.  Nonparametric estimation from incomplete observations. J Am Stat Assoc. 1958;53(282):457-481. doi:10.1080/01621459.1958.10501452.
Link to Article

Correspondence

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
Submit a Comment

Multimedia

Some tools below are only available to our subscribers or users with an online account.

307 Views
0 Citations

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
PubMed Articles
Jobs
×