Assessment and Remedial Clinical Education of Surgeons in California FREE

The number of physicians and surgeons being disciplined by medical boards and health care organizations appears to be growing. In 2009 to 2010, the Medical Board of California (MBC) had 122 451 licensed physicians with MD degrees and received 6539 complaints, opened 1312 investigations, and referred 569 cases to the attorney general.¹ Cases with administrative outcomes pertained more to negligence and incompetence than any other category of inappropriate conduct. Such numbers are not available for health care organizations, but one may assume that the number of complaints is larger and that many investigations of possible incompetence occur, some resulting in disciplinary action.

Physician competence assessment is a nascent field in the United States. Institutions that have arisen in the United States in response to this challenge, initially to meet the needs of state medical boards, are brought together by the Coalition for Physician Enhancement, which has 5 institutional members.² Two of these perform more than 100 comprehensive clinical competence assessments per year; one is the Physician Assessment and Clinical Education (PACE) Program at the University of California, San Diego (UCSD), founded in 1996.³ Based on Coalition for Physician Enhancement membership, we estimate that fewer than 500 independent, comprehensive assessments of practicing physicians are done per year in the United States.² Two programs, the Center for Personalized Education for Physicians in Colorado and UCSD PACE, account for about half of this activity.

The PACE Program aims to assess competence and make a plan for remediating any deficiencies that are revealed. The assessment and remedial education of surgeons may present special challenges. About 5% of the physicians participating in the PACE Program have been general surgeons. We review 10 years of assessment and remedial clinical education of general, general/vascular, and general/colorectal surgeons in the UCSD PACE Program.

Forty-seven general surgeons participated in the UCSD PACE Program between May 2000 and August 2010. Seventeen surgeons (36%) were assessed in 2000 to 2005 and 30 (64%), in 2006 to 2010. Forty-six (98%) were male (mean [SD] age, 54 [11] years; range, 34-80 years). Twenty-nine (62%) were US medical graduates, and 18 (38%) had graduated from medical schools in Argentina, Armenia, Brazil, Canada, Chile, Egypt, India (2), Iran (3), Mexico (3), Pakistan, Philippines, South Africa, and Taiwan. Forty-four (94%) had allopathic (MD) and 3 (6%) had osteopathic (DO) degrees. One was a surgical resident. Four were general/vascular surgeons with active vascular practices, 2 were colon and rectal surgeons, 1 practiced urology and general surgery, 3 had stopped operating and were practicing aspects of primary care medicine, and 3 were in administration full-time.

Thirty-three (70%) came from state medical board actions: 25 from the MBC disciplinary division, 2 from the MBC licensing division, 3 from other state boards, and 3 self-referred during other state board actions. Fourteen (30%) came from health care organizations: 8 from California hospitals, 3 from hospitals in other states, 2 self-referred during hospital proceedings, and 1 self-referred during a medical group investigation. All referrals included challenges to the surgeon's professional competence, which PACE was asked to evaluate independently and, if necessary, to help remediate.

State board disciplinary actions that accompanied referral to PACE included public reprimand (or equivalent in other states), probationary status, public record of disciplinary action by hospital, and surrender of license. Of the 47 licenses, 16 physicians received 20 public reprimands, 15 were on probation, and 1 had public record of hospital discipline, 1 license was retired for disability, and 2 licenses were surrendered. The remaining 12 physicians had no public disciplinary record; these were surgeons referred by health care organizations or state board licensing divisions.

The PACE model consists of phase 1, 2 days of multilevel, multimodal testing, and phase 2, 5 days of formative assessment and remedial education in the residency program of the participant's specialty.¹

Specific components of phase 1 include:

The Table shows the components of the PACE phase 1 assessment and the core competency assessed by each component.

Phase 1 assessment also includes a complete H&P (excluding breast, genital, and rectal examinations) on the participating physician, looking for illnesses that might interfere with safe medical practice, and the MicroCog (Pearson Education, Upper Saddle River, New Jersey) computerized neurocognitive screening examination. This is not a diagnostic examination, but abnormal findings can suggest the need for formal neurocognitive assessment.

Phase 2 clinical education consists of at least 40 hours of supervised observation in the operating room, clinics, case discussions, and conferences, including a standardized Library Exercise that requires the use of medical literature to support surgical management decisions in 10 clinical scenarios. This is done in the context of the surgical residency program and supervised by individual PACE faculty in general and colorectal surgery (B.C.C.) and vascular surgery (E.L.O.). Although the participating surgeon performs no patient care, he or she is asked to comment on many cases over 5 days, giving PACE faculty a good sense of the participant's knowledge and clinical judgment.

Surgeons who had substantial vascular components in their practices had an additional oral examination in vascular surgery during the phase 1 assessment, and vascular clinic and observation of vascular operations were included in their phase 2 programs. Colon and rectal surgeons were evaluated with a colorectal-only phase 1 oral examination, and phase 2 was the same as for general surgeons. The urologist/general surgeon had phase 1 oral examinations in both fields.

Twenty-three (49%) underwent a 2-day phase 1 assessment, including a 1-hour mock oral board examination: 8 “passed” with no recommendations; 6, with minor recommendations; 6 had major recommendations; and 3 “failed.”

Of the 8 who passed with no recommendations, 7 were not asked by their board or referring entity to return for a phase 2 program, and 1 was required to attend (after the study period) a special, limited, 2-day phase 2 program with emphasis on patient safety, the area in which the board had concerns initially.

Of the 6 who passed with minor recommendations, 1 was referred to take a course in how to complete a medical record appropriately; 2 had psychiatric and drug/alcohol follow-up recommended; 1 was asked to write a plan for avoiding the clinical problems that had brought him to board attention (he did not); 1 was asked to develop with his hospital a plan for appropriate case load; and 1 had uneven performance on testing and was suggested to return for phase 2 (he did not). None of these surgeons was required by the referring entity to return for a phase 2 program.

Of the 6 who had major recommendations, 1 was asked to have a surgeon mentor/monitor at his home institution for 1 year; 1 had medical and psychiatric follow-up recommended as well as operating room proctoring for 6 months; 3 had uneven performance on testing and were asked to return for phase 2 (1 did, after the study period); and 1 was asked to undergo further cognitive testing and return for phase 2 (he did not).

Of the 3 who “failed” with adverse recommendations, 2 were the only surgeons in our series referred by the MBC Division of Licensing. One, who was working as an administrator, was found to have a “poor professional attitude” and was recommended to stay away from patient care. Another, a fourth-year resident, was found to be severely deficient in medical knowledge and was denied a California license; he did not complete surgical training. The third surgeon who “failed” displayed performance “not consistent with safe practice”; he was asked to undergo physical and psychiatric evaluation and to return for further competency evaluation and remedial education (he did not).

In sum, these 23 surgeons completed phase 1 only, with a variety of recommendations, many acted on and others not acted on. The latter may be considered failures of the program. As an evaluative and educational body, PACE only makes recommendations. Although referring authorities often enforce its recommendations, a participating physician can still circumvent them by reaching a new agreement with, or conversely by abandoning the dispute with, the referring agency.

Twenty-four surgeons (51%) had 26 five-day phase 2 clinical education programs, in each case following a phase 1 evaluation. Phase 2 programs were either recommended by PACE based on phase 1 performance or (more commonly) required by the referring agency. There were 20 “passes,” 1 minor recommendation, 3 major recommendations, and 2 “fails.”

Of the 20 “passes,” all were recommended for unrestricted return to practice. This included 2 “passes” for 1 surgeon referred twice by the MBC. The 1 surgeon with a minor recommendation was asked to complete a record-keeping course.

Of the 3 surgeons who had major recommendations, 1 surgeon, working as an administrator, had the stipulation that if he decided to return to practice, he should complete a formal reentry training program. One general/vascular surgeon was asked to complete a vascular review course and undergo proctoring for 6 major vascular cases. One surgeon, who had “failed” phase 2 but had remediated a professionalism problem, was asked to undergo proctoring in vascular surgery.

Of the 2 surgeons who “failed,” 1 “did not demonstrate competency in the areas of communication skills and professionalism” and was referred by his hospital to repeat phase 2. He then “passed” with a major recommendation (see earlier). The other “failure” was a surgeon who had a stroke, stopped operating, and was practicing limited primary care; he “failed” phase 1, with repeated observations of organic brain problems, and completed phase 2 in primary care rather than surgery. He was found to have “multiple serious deficiencies in competence,” and PACE faculty expressed “grave concern” about his ongoing practice. The MBC revoked his license.

Follow-up activities include giving the participating surgeon follow-up on the cases he or she has seen during the phase 2 program; this is done 2 to 4 weeks after completion. Also, we encourage past participants to contact us at any time about cases they encounter.

Participating surgeons often request letters in support of credentialing. Two surgeons who “passed” phase 2 requested and received supporting testimony in credentialing hearings. The surgeon who had his license revoked after “failing” phase 2 subpoenaed us for an adversarial hearing but then surrendered his license.

The UCSD PACE Program was initially designed to fill the evaluation and education needs of the MBC. However, surgeons evaluated in this program came from 7 other states also, suggesting something of a national scope for the activity. Other similar programs also report a scope of practice that extends beyond their state borders.^4- 5 This is not surprising, because surgical practice and standards do not vary across state lines, and evaluation and education needs are similar in all states. Likewise, the use of the allopathic-staffed PACE Program by osteopathic boards, hospitals, and physicians (1 each, in this series) is not surprising, because similar MD and DO surgical practices are held to the same standards.

With one-third of our evaluations occurring in the first half of the study period and two-thirds in the second half, PACE is becoming busier. This may reflect a more active MBC, but it also indicates use of this type of program by health care organizations, other state boards, and defense attorneys. In this series, only 27 cases (57%) came from the MBC, with 43% from other sources. A program designed to meet the needs of 1 state medical board is being used by nongovernmental organizations as well, and it seems to meet the needs of some individual surgeons: there were 6 self-referrals (13%), all occurring in 2008 to 2010. By submitting themselves to objective evaluation, surgeons in adversarial proceedings with their medical boards or health care organizations may gain independent validation of their knowledge and judgment, if they perform well in the PACE assessment. An impartial evaluation program that stands solidly behind its successful “graduates” may be of value to them.

Comprehensive competence assessment would be of little value without a plan for remediating the deficiencies discovered, so each PACE assessment report includes a remediation plan. When assessment suggests a physical, mental, or cognitive disorder, the physician must receive formal evaluation before participating in clinical education. The PACE Program has several specific programs to address commonly encountered deficiencies: Medical Record Keeping, Prescribing, Anger Management, Physician Communications, and Professional Boundaries. To avoid the conflict of interest inherent in self-referral, PACE recommendations for remedial education do not direct physicians specifically to PACE offerings. The PACE Program also offers a monitoring/mentoring program that the MBC has viewed as an acceptable alternative to its own monitoring procedures; this program has not, during the study period, included any general surgeons.

Potential conflicts of interest did arise in our series. During the study period, the PACE surgical consultant (B.C.C.) recused himself from 2 cases because of prior professional relationships. Also, we have not accepted tertiary referrals from participating surgeons we have supported, to avoid the appearance of a quid pro quo.

The PACE Program assessed surgeons in clinical practice and a few who wished to return to practice. An obvious limitation of the PACE Program for surgeons is that it does not address technical skill. This problem can be overcome by adding a simulation component to phase 1 and/or phase 2, especially now that standardized simulation tasks, eg, Fundamentals of Laparoscopic Surgery, are taught and tested in residency programs. However, many participating surgeons do little or no laparoscopy or flexible endoscopy, and the applicability of simulation to open surgery is not well established. An approach that more closely simulates a surgeon's operative practice would be to adapt a course such as Advanced Trauma Operative Management to model the operations the participating surgeon does. The most obvious way to evaluate technical skills would be to visit the participant’s practice and scrub with him or her in the operating room, a potentially expensive project that no stakeholder wants to take on at present. The PACE Program has developed a Physician Enhancement Program for on-site mentoring and monitoring of a participating physician's practice, but we are just now at the earliest stages of implementation for general surgeons.

Another limitation of PACE and all similar programs is the lack of standardization and validation of evaluation techniques, aside from the formal aspects of the phase 1 program. The informally standardized and nonvalidated evaluation criteria and techniques we use in our oral examinations and clinical education programs (mock oral board examinations, clinical observation, case presentation and discussion) are used to evaluate residents in training programs, and PACE faculty are residency program faculty who do that routinely. For now, with in-practice evaluation in its infancy, we rely on this experience to give the program a reasonable degree of validity in the eyes of participants and institutions. If current efforts to standardize and validate residency evaluation progress,^6- 7 then these processes may be transferrable to practicing surgeons, who are evaluated in much smaller numbers annually than residents. If comprehensive evaluation of practicing surgeons were ever to become routine, then standardization and validation would be necessary features of the program.

We have presented a 10-year experience performing comprehensive assessments and providing remedial clinical education to general, general/vascular, and colorectal surgeons. Organizations like the UCSD PACE Program have developed a role beyond making assessments for state medical boards; such assessment and the remedial clinical education program are used by hospitals, medical groups, and individual surgeons. We value the collegial aspects of this work, which at times recall the ideal of professional self-regulation.