From the Department of Surgery, University of California, San Francisco, East Bay, Oakland.
We hypothesized that physician assistants (PAs) will decrease surgery resident work hours and improve resident work outlook.
Surgical resident survey.
A county hospital in a university-based surgical residency program.
Surgery residents who switched (or "rotated") to the county hospital were polled monthly for 6 months after using PAs as team members on the surgical services.
Main Outcome Measures
Resident work hours and work outlook.
Surgery resident hours were significantly decreased by the fourth, fifth, and sixth months after PAs joined the surgical services. Despite what these data on resident hours suggest, 6 (60%) of 10 residents believed that the PAs had no influence on the amount of time the residents spend in the hospital. Six (60%) of 10 residents thought the PAs decreased stress levels and 6 (60%) of 10 residents thought the PAs helped to improve morale.
Physician assistants can have a positive influence on graduate surgical education programs. Physician assistants can help decrease surgery resident work hours and improve resident work outlook.
OVER THE PAST YEAR, many surgical educators have questioned whether the training of the general surgeon is optimal and many feel that graduate surgical education needs restructuring. The disappointing 2001 results resonated throughout surgical training programs across the country. After the 2001 match, there were initially 68 unfilled categorical first-level positions and 425 unfilled preliminary first-level positions. Senior medical student interest in general surgery hit an all-time low with only 6% of graduating medical students showing an interest in the discipline of surgery.1 In the past 9 years, the number of students applying to general surgery has decreased by 30%.2 The need for change is unquestioned and some have called for a "Halstedian 2" modification of graduate surgical education.3
The shift in perspective on surgical training is gaining widespread recognition. Graduate surgical education is a major topic of discussion at every surgical society meeting. Every month there seems to be another editorial addressing this issue and the ARCHIVES dedicated an issue to this subject. Many reasons have been given as to why a career in surgery has changed, including the significant debt incurred during medical school, the length of surgical training programs, poor work environments, decreasing reimbursement for surgical services, and lifestyle issues.4- 6
The Accreditation Council for Graduate Medical Education has released resident training guidelines that would modify many training programs and may or may not optimize the training of surgery residents. These include an 80-hour workweek and, maybe more importantly, a maximum work shift of 24 hours. The work that surgery residents perform should not be underestimated. If residency programs are to adhere to the Accreditation Council for Graduate Medical Education guidelines, then someone must make up the workload that would have been completed by the residents to ensure quality surgical care. Physician assistants (PAs) have been suggested as ideal candidates for the position.7- 9 Midlevel practitioners are able to reduce surgery house staff workload and improve patient care.10
Physician assistants have been in existence since the late 1960s and the idea of PAs working on a university teaching service is not new. Physician assistants have been used on trauma,11 orthopedic,12 pediatric,13 and thoracic and cardiovascular surgical services.14 Our department has spent several years working to acquire funding for PA positions on each surgical service at an urban county hospital. In light of recent events surrounding graduate surgical education, the approval for these positions could not have happened at a more opportune time. We hypothesized that PAs would decrease the work hours of surgery residents and improve the surgery residents' work environment by decreasing their workload and work-associated stress, and thereby improving morale.
A general surgery resident survey was conducted from January 1, 2002, through June 30, 2002, at the county hospital in a university-based surgical residency program. General surgery residents while on service at the county hospital were polled monthly for 6 months after using PAs on the surgical services. The main outcome measures of the survey were resident work hours and work outlook.
There are typically 10 to 12 general surgery residents assigned to the 4 surgical services each month. Surgical rotations last for 1 to 2 months and residents rotate to the county hospital a few times each year. During the period of the survey, chief residents took night call from home on alternating nights, junior house staff took in-house call every third to fourth night, and trauma team members took in-house call every other day.
In the first week of January, 1 PA was assigned to each surgical service. The PAs were fully incorporated into the surgical team and functioned at the level of a postgraduate year 1 or postgraduate year 2 resident. The PAs are under the direct supervision of the chief resident or attending staff. Each PA worked 4 ten-hour shifts per week, usually 7 AM to 5 PM on Mondays, Tuesdays, Wednesdays, and Fridays. Thursdays are set aside for teaching conferences. The PAs switch (or "rotate") between services every 3 months. Rotating the PAs provides a varied work experience and prevents the PAs from taking over a service. This rotation occurs on the 15th of each month to provide continuity of care because residents switch services on the first of the month.
The survey asked the following questions: (1) How many hours per week did you spend in the hospital? (2) With the PAs now on your service: (a) Is your workload decreased? (b) Do you feel less stressed at work? (c) Has your morale improved? (d) Do you spend less time in the hospital?
A total of 68 questionnaires were distributed to the surgery residents over the 6-month study period. Of these, 61 were completed and returned for a 91% response rate. The surgery residents' replies were completely anonymous.
Resident work hours are listed in the text as mean (SE). Work hours were analyzed using repeated-measures analysis of variance; the differences between time points were analyzed using mean contrasts. Statistical significance was set at P<.05.
Surgical residents' weekly work hours per month are shown in Figure 1. Prior to the arrival of PAs, general surgery residents worked a mean of 102.2 (3.2) hours per week. Residents' weekly work hours slowly decreased over time and were significantly decreased by the fourth, fifth, and sixth month after PAs joined the various services. Work hours decreased to a mean of 89.0 (5.4) hours per week in April, 88.1 (4.2) hours per week in May, and 87.3 (2.8) hours per week in June. In 6 months, the presence of PAs on the surgical services enabled each surgery resident on average to reduce his or her workload by 15 hours per week. This equates to about a 1:1 ratio of resident work hour decrease to PA work hour completed.
Surgical residents' weekly work hours per month. The mean (SE) baseline value of 102.2 hours per week (3.2) was obtained prior to the arrival of physician assistants. At the end of the study period, there was a decrease of 15 work hours per week per surgical resident. Asterisks indicate a significant decrease from the baseline value.
The influence of PAs on resident workload is shown in Figure 2. This graph represents the percentage of residents who felt that the PA on their service helped to decrease their own workload. In January, 6 (67%) of the 9 residents felt that the PAs decreased their own workload. This increased to 100% in both May (12/12) and June (10/10).
Influence of physician assistants on surgery residents' workload. This graph illustrates the surgery resident's perception of whether the physician assistant on his or her service helped to decrease his or her own workload. The y-axis represents the percentage of surgical residents who responded yes to the question.
The influence of PAs on resident work-associated stress is shown in Figure 3. This graph represents the percentage of residents who observed less stress at work since the PAs arrived on their services. In January and February, 4 (44%) of 9 residents experienced less stress since the PAs arrived on their services. This increased slightly to 58% (7/12) in May and 60% (6/10) in June.
Influence of physician assistants on surgery residents' work stress. This graph illustrates whether the surgery resident experienced less stress at work since the physician assistant started work. The y-axis represents the percentage of surgery residents who responded yes to the question.
The influence of the PAs on the residents' perception of how much time they spent in the hospital is shown in Figure 4. This graph represents the percentage of residents who believed they were spending less time in the hospital since the PAs arrived on their services. In January, 44% of the residents believed they were spending less time in the hospital. This percentage was basically unchanged throughout the study period and was recorded at 40% (4/10) in June.
Influence of the physician assistants on the surgery residents' perception of how much time the surgery resident spends in-house. This graph illustrates the surgery residents' perception of whether they were spending less time in the hospital since the physician assistant joined their service. The y-axis represents the percentage of surgery residents who responded yes to the question.
The influence of PAs on surgery resident morale is shown in Figure 5. This graph represents the percentage of residents who experienced an improvement in morale after the PAs joined their services. In January, 44% of the residents thought that the PAs improved morale at work. This percentage increased slightly over the survey period and was recorded at 60% in June.
Influence of physician assistants on surgery residents' morale. This graph illustrates if the surgery residents believed that having the physican assistant on their service improved their morale. The y-axis represents the percentage of surgery residents who responded yes to the question.
There has been significant discussion lately regarding the work environment of general surgery residents, resident work hours, and the need to reevaluate the training of general surgeons. Realizing the need to optimize the training of surgery residents, we petitioned our hospital administration for assistance in relieving some of the burden that surgery residents carry in caring for surgical patients at the county hospital. We proposed to the administration that PAs would be an efficient, cost-effective way to improve the surgical services. After years of dialogue with the hospital administration and numerous reports documenting the need for PAs, positions for surgical PAs were approved.
The first PA training program in this country was started at Duke University, Durham, NC, in 1965. Since then, PAs have been used on trauma,11 orthopedic,12 pediatric,13 and thoracic and cardiovascular surgical services.14 Most of these services incorporated PAs owing to the pressures of downsizing of surgical residencies that were prevalent a decade or two ago. The problems facing surgery departments today are different; however, by bolstering the surgical workforce, PAs may help solve the problem now just as they did then.
In 1979, a survey of chairmen of departments of surgery in hospitals with more than 400 beds revealed that PAs were working in one third of those departments. Two thirds of the chairmen felt the PAs improved surgical patient care and half thought that the PAs improved the quality of surgical residency training. They predicted an increase of 87% in the number of surgical PAs by 1985.15 Their prediction could not have had more foresight. The American Academy of Physician Assistants estimates there were 10 000 students enrolled in PA programs this past academic year and about 50 000 PAs in clinical practice at the beginning of 2002. The US Bureau of Labor Statistics projects that the number of PA jobs will increase by 53% between 2000 and 2010. This is more than 3 times the predicted increase in the total number of jobs available in the United States during the same period.
The purpose of this study was to define the influence PAs have on surgery residents. More specifically, we wanted to know if having PAs as team members on surgical services would decrease resident work hours and if the presence of the PAs would improve the work environment of surgery residents. We hypothesized that the addition of PAs to the surgical teams would (1) decrease resident work hours, (2) decrease resident workload, (3) decrease work-associated stress, (4) decrease the time residents spent in the hospital, and (5) improve resident morale.
The additional workforce provided by PAs enabled a significant decrease in the amount of hours the residents worked. This effect was not seen immediately since the decrease in work hours was not significant until the PAs had been on service for 4 months. The slow decrease in resident work hours over several months probably reflects the time necessary for a PA to learn the position and become an efficient member of the surgical team. Although the decrease from 102 to 87 hours per week represents an average decrease of 15 hours per week per resident, the 87 hours per week per resident does not adhere to the limits as set force by the Accreditation Council for Graduate Medical Education for resident work hours.
For every hour of work put in by a single PA, there was 1 hour less work completed by a surgery resident. If we were to comply with the 80-hour workweek guideline by means of decreasing resident work hours through the hiring of additional PAs, 2 additional PAs would be needed. The costs of hiring each PA including benefits approaches $100 000 per year. This is funded by the hospital administration. Budgetary limitations will not support the employment of additional PAs. To comply with the resident work hour guidelines, we have made additional changes in the surgical services including an every fourth night call schedule for all residents.
At the end of the study period, all residents noted that the PAs helped to decrease their workload. At the beginning of the study only 67% of the residents responded similarly. This may be owing to the fact that at the beginning of the study period PAs were new and residents spent time each day training the PAs. Although all residents at the end of the study period had a favorable reaction regarding the PAs' capacity to decrease their workload, this did not equate to a similar attitude toward the PAs' ability to decrease the time residents spent in the hospital. Sixty percent of the residents believed that PAs had no influence on their time spent in-house even after the PAs had been on the surgery service for 6 months. This opposes data we collected on resident hours showing a decrease of 15 hours per week per resident, an interesting finding that offers some insight on how surgery residents perceive the amount of time residents spend in the hospital. Residents may be unable to differentiate whether they are working 102 hours per week or 87 hours per week. What they do know is that they are working over twice a normal workweek and receiving no overtime pay.
Financial concerns may be just 1 factor affecting the resident's stress level and morale. After the PAs were on service for 6 months, 60% of the residents experienced less stress at work. This also means that 40% of residents believed that the PAs did not influence their stress levels. Similarly, 60% of the residents thought that the addition of the PAs improved morale, while 40% did not. The constant pressure of life-and-death situations, job responsibilities, career, family obligations, and financial concerns can all lead to increased stress in the workplace and eventually may progress into morale problems. It is obvious that PAs and the potential they bring to decrease the workload of surgery residents and subsequently decrease resident work hours is not the only answer. Other potential stressors in the workplace need to be evaluated and addressed to optimize the learning experience of surgery residents.
This study suffers from all the problems and biases inherent in surveys. The study period was short and included replies from residents switched to the county hospital for a short period. The decrease in resident work hours may also be attributed to increased resident experience and efficiency. Finally, data on stress and morale issues should be interpreted with caution owing to complexities involved with such issues. Although it is possible to apply statistical analysis to these data or stress and morale issues, we felt the complexities surrounding these issues would be less meaningful and, therefore, these analyses were not completed.
Although our experience with surgical PAs had been a tremendously positive experience, we did make one mistake. When we started the interview process for the PA positions, we looked for candidates much as we would search for a surgery resident position. We thought the best candidates were smart, aggressive, and enthusiastic about surgery. It turns out that these personality traits result in an aggressive and enthusiastic desire to be in the operating room. This created some conflict with the junior house staff who rightfully deserve to be in the operating room. After 6 months, we had 2 PAs leave for positions in private practice with job descriptions containing more operative responsibilities and more exposure to surgical procedures. When recruiting new PAs, we now emphasize a need for a strong desire for patient care responsibilities on the wards and in the clinics and not as much in the operating room.
Physician assistants can have a positive influence on surgical residency programs. Physician assistants can help decrease surgery resident work hours and improve resident work outlook. However, the increase in surgical workforce through the hiring of midlevel practictioners is not the sole answer to the problems that confronts the education of general surgeons. The American College of Surgeons Candidate and Associate Society stated that "it is critical that the [surgery resident] work environment be monitored, modified and optimized."16 This should be a continual process.
Corresponding author: Gregory P. Victorino, MD, Department of Surgery, University of California, San Francisco, East Bay, 1411 E 31st St, Oakland, CA 94602.
Accepted for publication April 27, 2003.
This paper was presented at the 74th Annual Meeting of the Pacific Coast Surgical Association; February 18, 2003; Monterey, Calif; and is published after peer review and revision. The discussions that follow this article are based on the originally submitted manuscript and not the revised manuscript.
Samuel E. Wilson, MD, Orange, Calif: Medical student women at the University of Florida entering lifestyle-friendly specialties such as dermatology, radiology, anesthesiology, pathology and psychiatry have a significantly higher overall grade point average compared with those students entering surgical training (data from the University of Florida; Am Surgeon. 2003;69:53-58). It is disappointing that the cleverest of students were less willing to sacrifice lifestyle for the demands of surgical residency. No message to surgical educators could be clearer.
Revision of the work hours as we all must do by June 30, 2003, is a start, which is long overdue. But it should not stop there. Residents have long subsidized health care budgets in hospitals by performing the work of ward clerks, nursing attendants, laboratory personnel, and stretcher bearers. Drs Victorino and Organ have shown us how the PA can not only decrease resident work hours but also improve morale and relieve stress, at least for the majority, which is not bad.
Some of us recall that the PA movement emerged during and after the Vietnam conflict as a way to use the skills of the technically proficient medics, usually upper level NCOs [noncommissioned officers], who were returning from military duty. Several formal training programs, usually lasting 1 to 2 years, were started, one of which was at the Drew-King Postgraduate Medical School, Los Angeles, Calif. These reeducated corpsmen, now termed "physician assistants," readily found employment with health maintenance organizations, large specialty groups, and in primary care practitioners' offices.
My first question to the authors is: what educational requirements today should a PA meet? Do you require an undergraduate degree and/or completion of a formal PA training program? Next, you state that PAs are cost-effective. I would like to think that is so, but at annual salaries up to $100 000, and even replacing residents' work hour for hour, this seems a difficult economic proposal to sell to my hospital administration. What arguments persuaded your hospital administrator?
Nurse practitioners are an increasingly important component of the health care workforce. How do their job descriptions differ from the PA, and how do they interface with PAs in the hospital? Do you see competition?
Finally, in our experience with PAs, we noted a high turnover rate, particularly for the best PAs, as they sought higher education and even medical school. Somewhat distressingly, as the PAs became more proficient and entrenched in their positions, the relationship with the frequently rotating interns became less that of an assistant and more of a master. Did you notice any trend toward this role reversal?
Karen Deveney, MD, Portland, Ore: I commend you on looking at a very important and timely issue. I have a few questions regarding the use of PAs. Who pays the PAs' salaries? Is it the department or is it the hospital? What hours do the PAs work? Do they do night shifts as well as day shifts? What is their job description? Is it essentially the very same as the residents, or are there certain things that they actually do not do as much as the residents do? I suspect that there are some stresses that the PAs may not relieve. Have you looked also at the environmental culture in your hospital, ie, the hassle factor of the 24/7 resident-intern victimization by the constant beeper and the hospital system always trickling everything down to the person lowest on the food chain and what role that may play in the resident's perception of stress?
William P. Schecter, MD, San Francisco, Calif: I believe your study began in January and ended in June. First, is it possible that the work hours decreased because the efficiency of the residents in each of their jobs improved as the year progressed. Second, having been a resident for many years, I can tell you that January is the lowest point of one's morale. One arrives at the hospital at 0 dark hundred and leaves the hospital when it is pitch black out, it is cold out, and you feel horrible. Spring is the time when young men's and young women's hearts turn to fancy, the birds start singing, and you feel better, and, for the interns, the end is in sight. The morale is going to be better no matter what. So my question is: have you taken these factors into account?
Ronald G. Latimer, MD, Santa Barbara, Calif: As I was standing in line, Dr Organ turned around and said, "Oh, Lord!" (laughter). This is a provocative paper. I would suggest other outcomes to look at. One, does the incidence of moonlighting increase directly related to the number of hours not worked? Another outcome measure would be improvement of in-service examinations. Did the less stressed residents do better on their inservice examinations? Finally, are we going to see the evolution of what has developed in the University of California teaching system with the PAs doing all of the patient care and not the residents?
Larry Danto, MD, Truckee, Calif: I also thought this was an excellent paper. Aside from the morale value and the ability to attract residents in surgery, there is also the issue of quality of practice related to PAs. Specifically, can you show that there is a decrease in medical errors with the use of PAs or in fact is it the other way around?
Unidentified Speaker: First, did you survey or is there some way that you can assess the influence of PAs on the quality of care of patients? Second, given that education is one of our primary goals in a surgery residency, how did this PA employment and use of PAs influence the education of the residents?
Theodore X. O'Connell, MD, Los Angeles, Calif: My question is a little bit different than how the patients were influenced, the economic consequences, etc, what we are really talking about is a quality-of-life issue for the residents. How do you get more people going into surgical residencies, lifestyle changes, etc? I know with my residents, human nature being what it is, if I do anything for them to make less call, less work, whatever, and I give them a survey, I would get 100% saying they approved of it because they were afraid if they do not say that, I will reverse the changes back. So my question is: why did only 50% of the residents say that their stress was reduced and the morale was improved? Were you able to get this information by this survey? Why didn't they feel that life was improved because they would just want to keep thing going, even if it was a modicum of decreased work.
Dr Organ: Dr Russell, Monsignor Wilson (laughter). When Dr Victorino and I initially discussed a method of assessing the effectiveness of PAs on the surgical services, I was immediately reminded of a presentation at one of the surgical meetings on this subject. Margaret [Kosiba] in our office went through the ARCHIVES' computer for published manuscripts thoroughly and could not find such a manuscript. We eventually found where the presentation was and it was presented at the Pacific Coast Surgical Association. I called the author to determine if we could obtain a copy of the manuscript and in an almost baptismal tone, Dr Wilson reminded me that we had rejected the manuscript. I explained to Dr Victorino that it was only fitting that we ask Dr Wilson to discuss this paper and explore his skills and extend his charity. Again, we thank him for his discussion and the other discussants as well.
To answer some of his questions specifically, most of our PAs were educated at Samuel Merritt College (Oakland, Calif) and received an adjunct degree. The training lasted 2 years. They passed the State of California examination. They have an orientation period and then are assigned to the different services. They have a credentialing process that Dr Victorino put together and they were proctored in a number of items like the staff does also.
Is it cost-effective? We do not have any direct data but I would refer you to the University of California study that came out in 1988 that was overlooked by many thinkers in medical education that dealt with the work hour issues. Orlo Clark was on that commission. One of the things they found out was that if you wanted to put the residents on a 72-hour week, it would cost $54 million dollars in additional residents. If you replaced them with midlevel practitioners, believe it or not, it would cost $160 million dollars (1988 dollars). If you replaced them with board-certified surgeons, it was well over $290 million dollars. We have relied on those figures.
How did you sell the program to your hospital administrator? By intimidation and some persuasion. However, the national furor over resident work hours was an important issue. In negotiating with administrators, it is important not to play the trump card, that is, that if you move the residents, it will shut the Trauma Unit down and the hospital will go caput. They must know this. The fear of that process is best if you do not use it and let it linger out there as a threat.
Did we experience the thing about them being on the service? Well, Eric, we went to school on you guys, and I picked up from reviewing your work that when these PAs stayed on the service for a year, they took over the service and they told the residents what to do. So we rotate them every 3 months.
Dr Deveney, the salaries are paid by the hospital. They work a 40-hour week. Dr Victorino determines their rotations. We are trying to extend the hours so that they can be paid for overtime. They can be assigned nights, days, and weekends and attend most of the conferences. The reason why we have had a 50% turnover is because all of them have seen television and think a surgical PA is one who just works in the operating room. There is a hassle factor. The senior residents benefit from this hassle factor because the postgraduate year 1 and postgraduate year 2 surgical residents are receiving all of the pain medication and enema calls and clarification of orders.
Dr Schecter, I do not know that PAs are going to change the basic hormone patterns of men and women in the spring. But I agree with you. July and August are more difficult times for house officers. I tell the senior and chief residents, "Trust your new interns but verify until you really know them." But I would disagree. January to June is a much better time for the junior-level residents, but it is a dangerous time for the chief residents because most of them make their mistakes in the last 6 months when their body is in the institution but their souls and minds are where they are going to go to in July.
Dr Latimer, moonlighting is something we do not allow. If you read the RRC guidelines carefully, the program director has that authority. But I would point out to you that the role of general surgery is accentuated, well, the quality of the life of our residents is harmed a great deal because if they go to the emergency room to take care of a perirectal abscess and the colorectal fellow is moonlighting in another hospital, that is downsizing for our programs. If they go to take care of a peritonsillar abscess and the otolaryngology resident is someplace else moonlighting, then he has to take care of it. This goes on with urology residents also. There are some circumstances under which we do let the residents moonlight if there is a severe financial problem. I do not know, Dr Latimer, what the influence of this is going to be on the ABSite examinations. I have predicted that it will not change very much.
Dr Danto, we evaluate PAs every quarter. We need more experience. We know this is not a new idea.
Finally, I just want to comment that as program directors we must find new ways, we have to think outside of the box to improve not only our lives but also the quality of the lives of our residents. I want to repeat again, the solution to this is professional. We must step up to the plate. It must not be political, legal, or legislative. I happen to believe, from an economic viewpoint, we ought to spend more money training midlevel practitioners who extend our life. We can train them for about $50 000 a year. To train the generalist, the internist, the primary care people, it is going to take about $300 000. So that is an easy decision to make. But we must improve the life of our residents and this is just one method of doing it. Certainly, we are going to have to download more of these responsibilities and if you look at it carefully, residents have been subsidizing hospital budgets for a long time. A lot of people have been talking about repatriation. I would hate for residents to get into the repatriation business because it would break our health care system.
I would like to conclude by quoting David Farley from the Mayo Clinic. He said, "While I like the thought of my pilot being rested and refreshed prior to takeoff, flying airplanes is easy compared to the art of surgery." He continued: "Thousands of surgeons fly planes in their spare time; but I don't know of a single pilot that does surgery on the side."
Thank you for submitting a comment on this article. It will be reviewed by JAMA Surgery editors. You will be notified when your comment has been published. Comments should not exceed 500 words of text and 10 references.
Do not submit personal medical questions or information that could identify a specific patient, questions about a particular case, or general inquiries to an author. Only content that has not been published, posted, or submitted elsewhere should be submitted. By submitting this Comment, you and any coauthors transfer copyright to the journal if your Comment is posted.
* = Required Field
Disclosure of Any Conflicts of Interest*
Indicate all relevant conflicts of interest of each author below, including all relevant financial interests, activities, and relationships within the past 3 years including, but not limited to, employment, affiliation, grants or funding, consultancies, honoraria or payment, speakers’ bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued. If all authors have none, check "No potential conflicts or relevant financial interests" in the box below. Please also indicate any funding received in support of this work. The information will be posted with your response.
Register and get free email Table of Contents alerts, saved searches, PowerPoint downloads, CME quizzes, and more
Subscribe for full-text access to content from 1998 forward and a host of useful features
Activate your current subscription (AMA members and current subscribers)
Purchase Online Access to this article for 24 hours
Some tools below are only available to our subscribers or users with an online account.
Download citation file:
Web of Science® Times Cited: 23
Customize your page view by dragging & repositioning the boxes below.
and access these and other features:
Enter your username and email address. We'll send you a link to reset your password.
Enter your username and email address. We'll send instructions on how to reset your password to the email address we have on record.
Athens and Shibboleth are access management services that provide single sign-on to protected resources. They replace the multiple user names and passwords necessary to access subscription-based content with a single user name and password that can be entered once per session. It operates independently of a user's location or IP address. If your institution uses Athens or Shibboleth authentication, please contact your site administrator to receive your user name and password.