From the Departments of Surgery (Drs Olcott and Zarins), Cardiothoracic Surgery (Dr Mitchell), and Neurosurgery (Dr Steinberg), Stanford University School of Medicine, Stanford, Calif.
Surgeon-directed institutional peer review, associated with positive physician feedback, can decrease the morbidity and mortality rates associated with carotid endarterectomy.
Tertiary care university teaching hospital.
All patients undergoing carotid endarterectomy at our institution during a 5-year period ending August 1998.
Stroke rate decreased from 3.8% (1993-1994) to 0% (1997-1998). The mortality rate decreased from 2.8% (1993-1994) to 0% (1997-1998). Length of stay decreased from 4.7 days (1993-1994) to 2.6 days (1997-1998). The total cost decreased from $13,344 (1993-1994) to $9548 (1997-1998).
An objective, confidential peer review process that provides ongoing feedback of performance to surgeons and documents that performance in relationship with that of peers seems to be effective in reducing the morbidity and mortality rate associated with carotid endarterectomy. In addition, the review process lowered the hospital cost of performing carotid endarterectomy.
DURING THE past decade there has been an increased interest in documenting the outcomes and costs of surgical procedures. This has been stimulated in part by the surgeon's desire to achieve excellence, but also by governmental and other third-party payers who are concerned about results and costs. This information is important to them in determining how health care will be delivered and by whom. For example, the Health Care Financing Administration established the Health Care Quality Improvement Initiative in an attempt to improve care for Medicare beneficiaries by redirecting the work of peer review organizations to the areas of continuous quality improvement and practice guidelines development.1 The Health Care Quality Improvement Initiative focuses on improving the processes used in delivering health care and monitoring patterns of delivery and outcomes.
In 1993, we began to critically evaluate the outcomes and costs of major vascular procedures. As part of this process, a peer review team was established to monitor the outcomes associated with treating carotid occlusive disease. We initiated our peer review process by studying carotid endarterectomy because of the large number of cases performed at our institution and the fact that there is general agreement as to the indications and anticipated results of this procedure. Carotid endarterectomy, primarily because its end points are easily recognized, has been the subject of significant clinical research and many randomized clinical trials.2- 8 The indications for operation and the accepted incidence of morbidity and mortality are well-established and have been published by the American Heart Association, Dallas, Tex.9 This report documents our experience with an institutional peer review process for carotid endarterectomy.
All patients who underwent carotid endarterectomy at Stanford University Hospital, Stanford, Calif, between September 1993 and August 1998 were identified by selecting those patients with the International Classification of Diseases, Ninth Revision procedure code 38.12 in either the primary or secondary position. This included patients undergoing either a primary procedure or a "redo" procedure. Patients who had vertebral reconstructions or intracranial procedures were excluded, as were those who underwent multiple simultaneous procedures (eg, coronary artery bypass grafting and carotid thromboendarterectomy). At the same time, a carotid quality assurance team was assembled. This team included 3 registered nurses from care review and risk management and was directed by the chief of vascular surgery (C.K.Z.). All medical records of identified patients were reviewed annually by the quality assurance team and relevant data were collected. Indicators included in the study were death, stroke, transient ischemic attack, myocardial infarction, length of stay, need for reoperation, and cost. Follow-up was limited to the first 30 postoperative days. Cost was the sum of total hospital cost including overhead, and indirect costs as determined by the hospital's cost accounting system. Professional fees were excluded from this analysis.
Ten experienced surgeons from 3 separate disciplines—vascular surgery, cardiothoracic surgery, and neurosurgery—participated in the study. The same group of surgeons participated in the entire study and performed all carotid endarterectomies at the institution. On an annual basis, each surgeon received his morbidity and mortality rate report along with the average morbidity and mortality rates for the institution. Confidentiality of each surgeon's results was maintained. Results and trends were reviewed at an annual meeting of all participants. At this time, methods for improving care and controlling costs were also reviewed, and decisions were made regarding their implementation.
During the 5-year study period, 763 carotid reconstructions were performed at Stanford University Medical Center. The patient age (mean, 71.3 years), percentage of asymptomatic patients (mean, 35.4%), redo procedures (mean, 8.3%), and shunt utilization (mean, 73.5%) did not vary significantly from year to year. The mortality rate, stroke rate, length of stay, and cost all decreased during the 5-year period (Table 1). During the final year of the study (1997-1998) there were no deaths and no strokes. The incidence of postoperative transient ischemic attack and frequency of reexploration also declined during the study period. There was no decline in the incidence of postoperative myocardial infarction, although it did vary from year to year.
Cost and length of stay also decreased during the study period. The decrease in cost was largely owing to 2 factors: shortened length of stay and decreased use of preoperative angiography. During the fiscal years from 1994 through 1997, the average length of stay for patients who had angiograms immediately preoperatively was 3.8 days, with an average cost of $14,044. For those not having angiograms, the length of stay averaged 2.2 days, with an average cost of $7307. Angiography was also associated with a 1.3% incidence of stroke, the cost of which was not included in this study.
The effectiveness of carotid endarterectomy and the expected morbidity and mortality rate have been well documented by multiple institutional reports as well as several well-publicized randomized trials.2- 8 Also, peer review has been developed to monitor outcomes and to assist in educating physicians on obtainable results.10,11 Our study was undertaken to determine if confidential peer review could improve results and, as a secondary benefit, lower costs.
The natural history of carotid stenosis has been well documented. Patients with asymptomatic carotid stenosis of greater than 75% have a 2% to 5% risk of stroke within the first year of observation.12,13 Patients with hemodynamically significant carotid stenosis and transient ischemic attacks have a 12% to 13% risk of stroke in the first year following the onset of symptoms, and a 30% to 35% cumulative 5-year risk of stroke.14,15 Various randomized trials have documented significant improvement with surgical therapy for selected patients. The North American Symptomatic Carotid Endarterectomy Trial7 documented that in symptomatic patients with 70% or greater stenosis, carotid endarterectomy reduced the risk of ipsilateral stroke from 26% to 9% during a 2-year period. This is a relative risk reduction of 65%.7 The Asymptomatic Carotid Atherosclerosis Study8 demonstrated an 11% incidence of ipsilateral stroke in the medically treated group and a 5.1% incidence in the surgical group. This presents a risk reduction of 53%.8
While these and other studies have documented the effectiveness of carotid thromboendarterectomy, it is generally conceded that the procedure must be carried out within acceptable morbidity and mortality rate guidelines. The Stroke Council of the American Heart Association has developed standards for carotid artery surgery using the combined end points of stroke and/or death to define acceptable limits for carotid endarterectomy.9 The upper limits are 3% for asymptomatic patients, 5% for patients with transient ischemic attacks, 7% for patients with a prior stroke, and 10% for patients with a recurrent stenosis. These are obtainable goals and several reports document results similar to or even superior to these guidelines. Hertzer et al,2 in their review of 2228 consecutive carotid artery reconstructions at the Cleveland Clinic Foundation (Cleveland, Ohio) between 1989 and 1995, demonstrated a mortality rate of 0.5%, a stroke rate of 1.8% for a combined stroke, and a death rate of 2.3%. For patients undergoing reoperation, the stroke rate was 4.4% and the mortality rate was 1.5%. A recent review of 40 consecutive redo carotid operations throughout a 5-year period in our own institution revealed no deaths and no strokes.16
Multiple factors have been demonstrated to influence the results of carotid endarterectomy, including surgeon's caseload, institutional volume, and vascular surgery certification.10,11,17- 19 On the other hand, Holloway et al,20 in a recent review of the outcomes for carotid thromboendarterectomy in academic medical centers, demonstrated that shortened length of stay, decreased use of intensive care units, and decreased use of preoperative angiography did not negatively affect the results of carotid reconstruction.
Our study was performed to determine if in-house peer review carried out by the surgeons involved in performing carotid endarterectomy could improve results with this procedure. A definite improvement in mortality rate and morbidity was achieved during the peer review process. An additional benefit was a decrease in the hospital cost of performing carotid thromboendarterectomies. As a result of this study, we are expanding our peer review process to include all treatment modalities for carotid artery disease, including carotid artery angioplasty and stenting.
Based on review of the morbidity and mortality conference data, it would seem that the improvement noted over the course of this study was owing primarily to better case selection and perhaps more meticulous surgical technique. During the study period, there was no change in surgical personnel (all of whom were experienced surgeons) or in anesthetic management. As the peer review process is confidential, it is not known if minor changes in technique occurred during the study. However, the surgeons involved in our study have stated that they did not change their basic surgical technique, their indications for shunting or patching, or their method of intraoperative monitoring. Physician response to this process was uniformly very positive. As the data were handled in a confidential manner, no surgeon felt threatened or intimidated by the process. All participants were pleased with the improvement achieved. Also, the surgeons involved believed that the improved results made them and the institution more competitive in the marketplace. As Hertzer said, " . . . results mean everything."21
Presented at the 70th Annual Session of the Pacific Coast Surgical Association, San Francisco, Calif, February 20, 2000.
Corresponding author: Cornelius Olcott IV, MD, Division of Vascular Surgery, Stanford University Medical Center, Mail Stop H3636, Stanford, CA 94305-5642 (e-mail: firstname.lastname@example.org).
Ralph B. Dilley, MD, La Jolla, Calif: In this nicely presented paper by Dr Olcott and his associates, an attempt was made to attribute improved outcomes of carotid endarterectomy with regard to stroke rate, mortality rate, incidence of myocardial infarction, return to the operating room, and length of stay to the institution of a confidential peer review process. They have shown that the stroke and mortality rate, the incidence of postoperative transient ischemic attacks, the return of patients to the operating room, as well as the length of stay have all decreased as a result of this peer review process. Although there is no statistical analysis applied to the data, the trends are clear and more than justify such a policy. I would certainly agree with their premise, since outcome analysis is critically important, not only for maintaining quality in the hospital, but also for being able to produce results when negotiating for health care contracts or satisfying regulatory agencies. As Dr Olcott pointed out, carotid endarterectomy is an ideal index operative procedure for such analysis, since I believe it is one of the most extensively studied operations in terms of indications, expected benefits, and risks.
The Stroke Council of the American Heart Association has defined acceptable limits for the combined end points of stroke and death for carotid endarterectomy, and, as Dr Olcott stated, these are 3% for asymptomatic patients, 5% for patients with transient ischemic attacks, and 7% for patients with prior strokes. As one can see from the results today, these percentages can be markedly reduced, and I suspect that programs such as the one described by Dr Olcott should be instrumental in achieving these low rates of complications.
As surgeons, we have traditionally stood for good results above all other considerations, and in order to improve our results, we first need to identify our poor results. Whether this is done through entering all patients in a registry with regular reviews or a yearly peer review process as described today, is less important than the fact that poor results are examined. At Scripps Clinic, we have adopted a similar approach where stroke rate, mortality rate, incidence of other postoperative transient ischemic attacks, myocardial infarction, and other complications are monitored on an ongoing basis in a vascular registry and reviewed quarterly by the Quality Council for Cardiovascular Diseases, which includes not only cardiac surgeons and vascular surgeons, but also cardiologists and critical care specialists (ie, peer review based on organ systems rather than the traditional departmental structures).
I have several questions for the authors. First, I note that there are 10 surgeons and 3 divisions doing carotid endarterectomy at your institution with approximately 130 to 150 operations per year. At the beginning of this effort, were there certain surgeons who had poorer results than others, or were you able to determine whether the number of operations per surgeon or specialty had any relationship to outcome? Are residents and fellows from all 3 divisions trained to do this operation?
Second, at the annual meeting of the involved surgeons, exactly what types of discussions were held? Did these include indications for operation, technical issues, and postoperative management? Was it easy to get agreement among the participants, and were residents and fellows included in these discussions?
Third, the decision to limit the use of certain hospital resources such as not utilizing an ICU postoperatively, decreasing the length of stay, and decreasing the use of arteriograms seems unlikely to have come from the peer review process but more likely to have been driven by cost considerations. At least that was our impression at Scripps Clinic when we made these changes. What prompted these changes at Stanford?
Finally, I don't believe you have demonstrated cause and effect. Is it possible that most of the improvements noted are coincidental to your confidential peer review and are the result of improved practice patterns that most of us achieve with time?
I believe the process reported on today will be similar to one that most surgeons at hospitals will have to develop in order to justify doing these operations.
Kaj H. Johansen, MD, Seattle, Wash: Demonstration of an association is completely different from proof of causation. I propose to Dr Olcott that while the institution of a "feel-good," expensive, and time-consuming intervention like a formalized peer review program may have reduced the incidence of complications associated with carotid reconstruction, that hypothesis has not been proven here. Alternatively, it could just be that surgeons selected their patients better or improved technically.
Second, Dr Olcott has suggested that the Stanford surgeons have become more competitive by means of these better results. Why then a 50% reduction in the total number of carotid operations performed in the last year of this study as compared to the prior year? That doesn't sound competitive to me.
William P. Shecter, MD, San Francisco, Calif: I am interested in the relationship between the bimonthly morbidity and mortality conference and the yearly report card. Did you make any systemic changes at the yearly conference, or do you think the changes were just changes in individual behavior, reducing errors in technique and judgment? If the latter is true, why wouldn't the bimonthly M&M conference address those problems?
James J. Peck, MD, Portland, Ore: My question has to do with the peer pressure and the outliers. Did any of the 10 surgeons who were originally doing carotid surgery stop or retire or move on to another institution during the study? Was there a switch in the service that was doing carotids? In other words, in the beginning of the study, were cardiovascular and neurosurgery doing more carotid surgery, and vascular surgery doing more of these cases later on?
Dr Olcott: To address Ralph Dilley's questions first, there were some differences between surgeons obviously. That was one of the reasons for the peer review process. As this was a confidential study, and I don't think it would be appropriate to really discuss the differences between surgeons or specialty; suffice it to say that there were some differences and that these were resolved during the study. We do think that a registry is important, and we are also in the process of setting one up. This is not an easy thing as it turns out, but this will allow us to have contemporaneous monitoring of results; not only results of carotid surgery, but of other vascular procedures as well.
The annual meeting that was held complemented the M&M meeting that we had on a bimonthly basis. The M&M meeting is primarily a vascular surgical service meeting attended by the vascular attendings and trainees, residents, and fellows. The 2 cardiac surgeons who do vascular surgery at our institution also attend that meeting. The neurosurgeons do not. The annual meeting does include the neurosurgeons. Regarding Dr Johansen's question about association and causation, both the M&M conference and the annual meeting probably play a role, and certainly there is a component of peer pressure. Surgeons realize that their results are being scrutinized. Exactly what factor is responsible for the improved results is hard to know. Suffice it to say that with this peer review process, the results did improve.
Regarding Dr Dilley's question about cost considerations and what drives them, certainly the institution was interested in cost. One of the reasons we got the nursing assistance that we did was because the hospital was interested in curtailing cost. There is an interesting study that I would refer you to in the Archives of Neurology in 1998, where the authors looked at academic medical centers and the factors that did not affect the results of carotid surgery. Interestingly enough, length of stay, decreasing use of angiography, or decreasing use of the ICU, were not associated with any change of the results. So holding down costs does not necessarily have a negative impact on results.
We also noticed that angiography was associated with a certain incidence of stroke. In our study it was 1.3%, which is comparable to the ACAS study.
There was a decrease in the number of cases performed and that probably represents better case selection. Whether we missed some people who should have been operated on but were not, we don't know.
There were no changes in personnel. The number of surgeons remained constant. These were all senior surgeons. None of the surgeons left or were replaced during the study. The results of this study are the result of the M&M conferences, the annual meeting, and the entire peer review process.
Thank you for submitting a comment on this article. It will be reviewed by JAMA Surgery editors. You will be notified when your comment has been published. Comments should not exceed 500 words of text and 10 references.
Do not submit personal medical questions or information that could identify a specific patient, questions about a particular case, or general inquiries to an author. Only content that has not been published, posted, or submitted elsewhere should be submitted. By submitting this Comment, you and any coauthors transfer copyright to the journal if your Comment is posted.
* = Required Field
Disclosure of Any Conflicts of Interest*
Indicate all relevant conflicts of interest of each author below, including all relevant financial interests, activities, and relationships within the past 3 years including, but not limited to, employment, affiliation, grants or funding, consultancies, honoraria or payment, speakers’ bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued. If all authors have none, check "No potential conflicts or relevant financial interests" in the box below. Please also indicate any funding received in support of this work. The information will be posted with your response.
Some tools below are only available to our subscribers or users with an online account.
Download citation file:
Web of Science® Times Cited: 13
Customize your page view by dragging & repositioning the boxes below.
The Rational Clinical Examination
Evidence Summary and Review 5
The Rational Clinical Examination
Updated Literature Search
All results at
Enter your username and email address. We'll send you a link to reset your password.
Enter your username and email address. We'll send instructions on how to reset your password to the email address we have on record.
Athens and Shibboleth are access management services that provide single sign-on to protected resources. They replace the multiple user names and passwords necessary to access subscription-based content with a single user name and password that can be entered once per session. It operates independently of a user's location or IP address. If your institution uses Athens or Shibboleth authentication, please contact your site administrator to receive your user name and password.