Telemedicine:  The New Must for Surgery FREE

Zurich, Switzerland, 2032. After a hard 6-hour workday, Theodor Billroth VI (pseudonyms became very popular after the institution of the Worldwide Medical Court) orders the computer to close the microaccess ports of his third duodenopancreatectomy in a row and pulls his head out of the virtual operative environment. The first 2 cases went well, owing to the extensive virtual planning by his assistants; however, this last case. . . . The guy had such a bleed when he inexplicably lost contact with his second intra-abdominal microbot. Fortunately, Alan Whipple IV from New Sloan Kettering in New York, NY, was available to assist him for 30 minutes, but the fees would again wipe out any profit he had otherwise made this day. In a bad temper, he came down from his study to dine with his family, while the patient was cared for by the intensive care robot at his hospital in Sidney, Australia. . . .

Vision or nightmare, modern information technology renders this scenario possible within the not-too-distant future. Whether such development is profitable for the patient and society, what steps should be taken, and which ones should be avoided will be discussed here. Of course, not the least of the conjectures is whether there will still be any diseases in this future requiring radical surgical procedures. However, during this century, there has been little change in the need for surgery and the great breakthrough from "molecular" technologies is apparently not to occur immediately. Information technologies are actually revolutionizing society and some argue that the same technological development will revolutionize surgery to the same extent that antisepsis and general anesthesia did 100 years ago.^1- 2 However, 100 years ago, technological development was driven by specific demand. Today, technological development drives the applications and, therefore, great care must be taken not to succumb to technocratic excitation, but rather to carefully select for surgical application only those pieces of technological innovation that promise a clear benefit to patients. Considering telemedical applications, the entire surgical datastructure is to be questioned. Telesurgery, in the broadest sense, becomes possible only as surgery itself becomes digital.

The transmission of medical image information by electronic means (in the beginning not necessarily in a digital format) is about 40 years old.³ However, digital data formats are a prerequisite for image communication meeting medical quality requirements. As of today, most preoperative information about a given pathology is primarily digital (magnetic resonance imaging, computed tomographic scans, and sonographic imaging). To print this information onto plastic films constitutes a reduction of the informational content and availability. Only standards,⁴ time, and financial resources are necessary to use this information in its original format. It is then obvious that we need to develop tools of enhanced data viewing, ie, 3-dimensional reconstruction,^5- 6 volume rendering,^7- 8 virtual endoluminal views,^9- 10 and image fusion,^11- 12 to enable the best possible preoperative assessment of the available information. An improvement of surgical planning is evident, even if patient benefit can not be proven by proper clinical trial techniques. More difficulties may arise if the surgeon, during an operative procedure, wants to review the data or even wishes to superimpose digital information on his or her conventional view of the surgical site. The first problem may be solved by the development of surgery-adapted interfaces based on voice or motion control. The second would involve the construction of a virtual reality environment fusing real-time optical stereoscopic images with stored 3-dimensional data. For open-access general surgery, a multitude of problems arise from such a scenario, some of them purely technical, some of them rather human. Technologically, the first major difficulty arises from the structural and positional mobility of the involved soft tissues, which prevents a precise fusion of preoperative with intraoperative information. The immediate problem, however, lies in the quality of our eyes, their dynamic range, their high-resolution, large weighted field, and spatial optical data acquisition that is so far not equaled by any technical solution. However, the obvious benefits of minimal access surgery, at least for a limited number of diseases, have made surgeons all over the world accept a clearly inferior video image of their work field, even loss of the third dimension, when technological solutions for 3-dimensional viewing were available. Thus, if the benefit from virtual reality surgery is large enough, there is no fundamental obstacle to it. A good example for such benefit comes from cardiac surgery, where considerable progress has been made toward a virtual heart arrest that may enable the surgeon to work on a motionless image of the heart¹³ while the motion of the beating heart is removed electronically from the video image. Far more difficult, of course, is the then necessary addition of the cyclic spatial reconfiguration of the beating heart to a robotic interface between the surgeon working on the motionless image on the screen and the real moving surgical site.

Closer to reality are developments in digital data processing for surgical purpose. A project of 3-dimensional volume rendering for radiological data at our institution was baptized BERTA (bidirectional enhanced rendering for telemedical applications). The software performs real-time stereoscopic visualization of reconstructed radiological data by displaying off-line rendered views.¹⁴ On an imaginary sphere, enclosing the data cube (originated by computed tomography, magnetic resonance imaging, or other modalities), about 5000 views of the object are precalculated. The corresponding views can be created off-line by any method rendering volume data. As a result of the achievable high frame rate, the user is able to rotate the data cubes in real time (50 stereoscopic frames per second) by using different interactive devices (keyboard, mouse, joystick, or voice control). The user can create a region of interest that specifies a cut plane. This cut plane can be pushed through the voxel data set along the line of sight by the interactive device. The visible cut of the plane with the data cube is computed in real time as a gray-level image of the original voxel information (Figure 1). The display of the precalculated views can be used as a rough guide for prescanning the radiological data.

Once the tools are developed to integrate and handle the multitude of anatomical, physiological, and pathological body data within a digital environment, it becomes possible to manipulate this virtual patient by virtual therapies. The most obvious application may be seen in planning the plane of a surgical resection, for example, in parenchymatous organs, optimizing the relationship between radicality and organ damage. For this approach, the governing conditions for remaining organ function and postoperative morbidity must be known and translated into evaluating and self-optimizing algorithms. Again, based on such outcome prediction models, surgical training for the individual case, as well as general surgical training, becomes possible. The evaluating algorithms themselves may represent the individual medical intelligence and experience of an institution, which again can be communicated to others.

The actual technological development first integrates haptic interfaces into what at this point are still rather simple models of virtual interaction with existing morphological data. The Responsive Workbench shown in Figure 2 is a suitable presentation technology for medicine, especially for surgery (in cooperation with Virtual Media and System Design, and Gesellschaft für Mathematik und Daten Verarbeitung, St Augustin, Germany). In the Responsive Workbench concept, virtual objects and control tools are located on a real workbench, in our case a surgical table. To get this effect, stereoscopy is required. The objects, displayed as computer-generated stereoscopic images, are projected onto the surface of the table (in cooperation with Belgian-American Radio Corporation, Kortrijk, Belgium). This view corresponds to the conventional working situation in medical environments. The computer is invisibly integrated into the user's world. The idea behind it is to create a realistic virtual surgical environment where planning and training can be performed without a change in the surgical routine, with movements, gestures, and words being perceived by cameras and microphones, thus adapting the technology to the surgeon, not the other way around (Figure 3).

Teleradiology is developing as digital data communication acquires the necessary speed and quality to communicate radiological data. While at the beginning of this decade comparative trials of radiological diagnoses based on plain radiographs and digitized images displayed on a video monitor resulted in an inferior digitized image,^15- 16 reports today seem to prove equivalence.¹⁷ Especially for military applications, teleradiology systems have already proven their value.^18- 19 Once legal and financial issues are solved,^20- 21 the acceptance of this model of collegial reassurance and specialist consultation will certainly grow. In pathology, mailing some histology slides or an embedded specimen to a colleague for a second opinion is widely accepted and digital data communication will simply replace this already common practice.^22- 23 While some comparative trials yield a good diagnostic accuracy based on computerized still images^24- 25 the sampling error in static telepathological systems is a concern. In a survey of Austrian pathologists, most would require an interaction with the slide for image selection.²⁶ Such interaction is feasible by using telecommandable microscopes, based on 2-way communication.^27- 28 Thus, the idea of interaction enters teleconsultation. Teleconsulting in surgery is certainly more difficult, because a multitude of different data have to be communicated. For some kinds of data, even the sensor technology is not readily available (tactile sense, texture, elasticity, or degree of mobility), far less the representation algorithms on the recipient side. However, in those areas of surgery where the surgeon's perception is already limited to the videoscopic image or an optical microscope, teleconsultation will only depend on the availability of data communication technology at a sufficient quality level. The adaptation of high-resolution stereoscopic video imaging to surgical microscopes is one step in that direction (Figure 4), the integration of different data sources into a stereoscopic workstation another. We have developed the technological system architecture to display, communicate, and manipulate the complex surgical data set (Figure 5) required for surgical teleconsulting at our surgical research unit, OP 2000 (a project to design the operating room [OP] of the next century).²⁹

The OP 2000/SICONET (Stereoscopic Image Communication Network, in cooperation with Deutsche Telekom, Darmstadt, Germany) telecommunication module is suitable for the connection of 2 or more locations with stereoscopic broadcast video, audio, and data, as shown in several applications (Figure 6). This technique is able to transfer live video data for medical purposes. Figure 7 shows the components of this module, which is able to transmit 2 video channels (left and right) in digital serial component form (digital serial component, 2 × 270 megabits per second [MBit/s]), 4 audio channels, and additional data channels for computer networking, multiplexed to a 17-Mbit/s (satellite) or 34-Mbit/s (asynchronous transmission mode) data stream.

The network for the first field trials was implemented using the experimental 20/30-gigahertz (GHz) transponder of the German national satellite "DFS Kopernikus." This method of linking the codecs was chosen for the easy availability, great flexibility, and the possibility of making point to multipoint connections. Actually, a switch to the more commercially used 12/14-GHz transmission frequency band also exists. The operating rooms, the computer graphics system, endoscopic unit, and pathological laser laboratories at the Robert-Rössle-Klinik, Berlin, Germany, are connected by a central digital video router. The router's stereoscopic video outputs (DSC) are coded and transmitted by the OP 2000/SICONET telecommunication system (Figure 7).

In several demonstrations, different telemedical applications have been shown. The performance of the 17-MBit/s satellite link was successfully demonstrated by the first European stereoscopic telesurgery between Berlin and Paris, France (1996 Meeting of Computer Assisted Radiology and Surgery, Paris) and by the first transatlantic stereoscopic interactive telesurgery between Berlin and Chicago, Ill (50th Annual Meeting of the American Society of Surgical Oncology, 1997, Chicago).

In 1997, the OP2000/SICONET module was adapted to asynchronous transmission mode (ATM) channels. We have used an ATM fiberglass connection of about 80 km to connect different departments of the University Hospital Charité, which are distributed within the city of Berlin. At the 1997 Meeting of Computer Assisted Radiology and Surgery, Berlin, we demonstrated an intraoperative, multipoint consultation system via a 34-Mbit/s ATM for a patient with cancer that involved a pathological department (Charité Mitte) and a radiological department (Charité Wedding) assisting the surgeons at the Robert-Rössle-Klinik (Charité Buch) during the surgical intervention (Figure 8). Not only were video and audio connections provided, but interactive manipulations were performed remotely.

At the 1998 meeting of the European Society of Surgical Oncology, Lausanne, Switzerland, multiple operating theaters, 3 in Berlin and 2 in Lausanne, were interactively connected via satellite.

At the final demonstration of the European project PANORAMA (Package for New Operational Autostereoscopic Multiview systems and Applications; in cooperation with 12 European science centers, universities, and companies), an interactive telesurgical scenario using stereoscopic image sources and stereoscopic visualization was presented by OP2000/SICONET via ATM.

Telemedicine will help to focus on the advantages of expert knowledge by rendering access to all available data. These possibilities should be accessible during a consultation, an examination, or in the course of a surgical intervention. Real-time modalities, eventually including the possibility of direct interaction with the data, are referred to as telepresence and exceed by far an electronic version of the patient's medical folder.^4,30 Especially in oncology, interdisciplinary collaboration is immensely important for successful therapy. Preoperative diagnostic data still need to be evaluated according to the intraoperative findings. At this decisive moment, it is necessary to involve specialists of other oncological disciplines. Real-time communication devices have to be present to transfer image data and clinical observations and ensure the best possible transmission quality to resident and geographically distant experts.

Theoretically, there is only a small step from the virtual plan of a surgical procedure to its execution in the manner it was planned. However, taking the scalpel out of the surgeon's hand and putting it into the mechanical front end of a robot involves more than one might expect, and is not within the focus of this article. The creation of a robotic interface between the surgeon and the patient is a prerequisite for the scenario that was drawn up in the introduction. Increasing the distance between the surgeon guiding the procedure and the robotic execution of the orders from meters to kilometers turns computer aided surgery into telesurgery, with all of the implications of immediate teleconsulting and teleassistance. However, traditional medical ethics should limit the allowance of distance between surgeons and their patients to situations of war³¹ or to areas that would otherwise not receive medical care owing to limited resources.

Theodor Billroth VI may still stand at an operating table, to the right of his patient, performing open access surgery with a scalpel in his hand. However, he will certainly have vizualized the preoperative data through some kind of computed, digital data–rendering interface, eventually even prior to the patient consultation. Medical liability issues and ongoing subspecialization may have motivated him to consult with colleagues, all of them having access to the same data, and some of these consultations might take place during the surgical procedure. Also, at least some of the indications that today would be treated by open access surgery would in that future be treated by robots using minimal access procedures, and the surgeon might not touch the patient.

It is the task of our generation to preserve the traditional qualities of surgery while keeping an open mind to the benefits of a society-revolutionizing technology.

Corresponding author: Peter M. Schlag, MD, PhD, Direktor der Klinik für Chirurgie und Chirurgische Onkologie, Robert-Rössle-Klinik am Max-Delbrück-Centrum für Molekulare Medizin, Universitätsklinikum Charité, Medizinische Fakultät der Humboldt Universität zu Berlin, Lindenberger Weg 80, D-13125 Berlin, Germany (e-mail: schlag@rrk-berlin.de).