0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Research Letter | Pacific Coast Surgical Association

Comparing the National Surgical Quality Improvement Program With the Nationwide Inpatient Sample Database FREE

Anna Weiss, MD1; Jamie E. Anderson, MD, MPH2; David C. Chang, PhD, MBA, MPH3
[+] Author Affiliations
1Department of Surgery, University of California, San Diego
2Department of Surgery, University of California, Davis, Sacramento
3Department of Surgery, Massachusetts General Hospital, Boston
JAMA Surg. 2015;150(8):815-816. doi:10.1001/jamasurg.2015.0962.
Text Size: A A A
Published online

Both raw and risk-adjusted outcomes are increasingly being made publicly available.13 The American College of Surgeons National Surgical Quality Improvement Program (NSQIP) is heralded as the most robust database to examine surgical outcomes. However, enrollment in the NSQIP is expensive, and the use of administrative databases may be more cost-effective.24

In our study, we compare the receiver operating characteristic curves of the Nationwide Inpatient Sample (NIS) with those of the NSQIP to determine which is superior at performing analyses of risk-adjusted outcomes for several operations.

Our study uses 2010 data from both the NIS and the NSQIP. Inpatients older than 18 years of age were included. Patients were identified by International Classification of Diseases, Ninth Revision codes (NIS) and Current Procedural Terminology codes (NSQIP): abdominal aortic aneurysm repair, appendectomy, aortic valve replacement, coronary artery bypass graft, carotid endarterectomy, laparoscopic cholecystectomy, total and partial colectomy, esophagectomy, sleeve gastrectomy, pancreatectomy, and ventral hernia repair. Outcomes included inpatient death and complications. Patients were classified as having a complication if they had one of the following: infection (surgical site, deep incisional, and organ/space), wound disruption, pneumonia, pulmonary embolism, acute renal failure, urinary tract infection, cerebrovascular accident, myocardial infarction, and blood loss requiring transfusion.

We performed multivariate logistic regressions predicting inpatient mortality and complication by procedure. In the NSQIP, the models were adjusted for preoperative risk variables in the database. In the NIS, the models were adjusted for age, race, sex, insurance status, and Charlson comorbidity index. The area under the receiver operating characteristic curve (C statistic) was calculated for each model. Statistical analysis was performed using STATA 64-bit Special Edition, version 11.2 (StataCorp). Our study was exempt from review as designated by the University of California, San Diego Human Research Protections Program.

There were 242 584 patients in the NIS and 73 130 patients in the NSQIP. Unadjusted complication rates were higher in the NIS than in the NSQIP for 7 surgical procedures. Mortality rates were higher for most procedures in the NIS; they were similar for appendectomy, laparoscopic cholecystectomy, and coronary artery bypass graft (Table 1). The C statistic was much higher in most logistic regressions for both mortality and complications in the NSQIP database (Table 2).

Table Graphic Jump LocationTable 1.  Comparison of Unadjusted Outcomes
Table Graphic Jump LocationTable 2.  Comparison of C Statistic By Procedure Between the Nationwide Inpatient Sample (NIS) and National Surgical Quality Improvement Program (NSQIP) Databases

Our study shows that the NSQIP is superior to the NIS administrative database as represented by higher C statistic values. Our study also finds that both mortality and complication rates were higher in the NIS than in the NSQIP. It is possible that hospitals participating in the NSQIP have lower mortality rates because they systematically examine their surgical outcomes. However, an alternative explanation is that hospitals in the NSQIP underreport their complications. Nurse abstractors are able to reason and exclude complication rates or mortality rates that are not directly related to a procedure. This is not true of the NIS. As the landscape for postoperative complication reimbursement changes, it will be prudent to repeat our study in several years and note if complication rates decrease.

Our study is limited by differences in coding. The NIS relies on automated data extraction from discharge diagnoses, whereas the NSQIP relies on trained nurses to manually extract information. Variations exist between International Classification of Diseases, Ninth Revision and Current Procedural Terminology codes. Also, each database was developed for different purposes and may not correlate.3 The NIS was developed for reimbursement purposes, whereas the NSQIP was developed to evaluate and improve outcomes.1 These databases also include different sample hospitals. The hospitals involved in the NSQIP tend to be academic centers, whereas the NIS includes all US hospital discharges. This may create a sampling error by comparing a consistent NSQIP patient cohort with a relatively varied NIS cohort. In conclusion, the NSQIP allows for a more robust risk-adjusted analysis compared with the NIS, as evidenced by higher C statistic values. Wider participation in the NSQIP could allow more hospitals to participate in robust research in surgical outcomes and quality.

Corresponding Author: Anna Weiss, MD, Department of Surgery, University of California, San Diego, 200 W Arbor Dr, 8402, San Diego, CA 92103 (a3weiss@ucsd.edu).

Published Online: June 10, 2015. doi:10.1001/jamasurg.2015.0962.

Author Contributions: Dr Weiss had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: All authors.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Weiss, Anderson.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Anderson.

Administrative, technical, or material support: Weiss, Chang.

Study supervision: Weiss, Chang.

Conflict of Interest Disclosures: None reported.

Previous Presentation: This paper was presented at the 86th Annual Meeting of the Pacific Coast Surgical Association; February 19, 2015; Monterey, California.

Henderson  WG, Daley  J.  Design and statistical methodology of the National Surgical Quality Improvement Program: why is it what it is? Am J Surg. 2009;198(5 suppl):S19-S27.
PubMed   |  Link to Article
Davenport  DL, Holsapple  CW, Conigliaro  J.  Assessing surgical quality using administrative and clinical data sets: a direct comparison of the University HealthSystem Consortium Clinical Database and the National Surgical Quality Improvement Program data set. Am J Med Qual. 2009;24(5):395-402.
PubMed   |  Link to Article
Atherly  A, Fink  AS, Campbell  DC,  et al.  Evaluating alternative risk-adjustment strategies for surgery. Am J Surg. 2004;188(5):566-570.
PubMed   |  Link to Article
Hall  BL, Hirbe  M, Waterman  B, Boslaugh  S, Dunagan  WC.  Comparison of mortality risk adjustment using a clinical data algorithm (American College of Surgeons National Surgical Quality Improvement Program) and an administrative data algorithm (Solucient) at the case level within a single institution. J Am Coll Surg. 2007;205(6):767-777.
PubMed   |  Link to Article

Figures

Tables

Table Graphic Jump LocationTable 1.  Comparison of Unadjusted Outcomes
Table Graphic Jump LocationTable 2.  Comparison of C Statistic By Procedure Between the Nationwide Inpatient Sample (NIS) and National Surgical Quality Improvement Program (NSQIP) Databases

References

Henderson  WG, Daley  J.  Design and statistical methodology of the National Surgical Quality Improvement Program: why is it what it is? Am J Surg. 2009;198(5 suppl):S19-S27.
PubMed   |  Link to Article
Davenport  DL, Holsapple  CW, Conigliaro  J.  Assessing surgical quality using administrative and clinical data sets: a direct comparison of the University HealthSystem Consortium Clinical Database and the National Surgical Quality Improvement Program data set. Am J Med Qual. 2009;24(5):395-402.
PubMed   |  Link to Article
Atherly  A, Fink  AS, Campbell  DC,  et al.  Evaluating alternative risk-adjustment strategies for surgery. Am J Surg. 2004;188(5):566-570.
PubMed   |  Link to Article
Hall  BL, Hirbe  M, Waterman  B, Boslaugh  S, Dunagan  WC.  Comparison of mortality risk adjustment using a clinical data algorithm (American College of Surgeons National Surgical Quality Improvement Program) and an administrative data algorithm (Solucient) at the case level within a single institution. J Am Coll Surg. 2007;205(6):767-777.
PubMed   |  Link to Article

Correspondence

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

526 Views
3 Citations
×

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
PubMed Articles
Jobs
JAMAevidence.com

Care at the Close of Life: Evidence and Experience
What Should the Physician Expect of the Hospital-Based Palliative Care Service?