About RFS

The Effect of International Teleradiology Attending Radiologist
Coverage on Radiology Residents’ Perceptions of Night Call

Sandor A. Joffe, MDa,    Jarett S. Burak, MDa,    Marlene Rackson, MDa,    Devon A. Klein, MDb,    Marshall M. Joffe, MD, MPH, PhDc
The views that the author expresses in this article are strictly their own and should not be attributed to the American College of Radiology.
Reprinted with permission from JACR
Volume 3, Issue 11, Pages 872-878 (November 2006)
The Effect of International Teleradiology Attending Radiologist
Coverage on Radiology Residents’ Perceptions of Night Call
Joffe SA, Burak JS, Rackson M, Klein DA, Joffe MM
Want to discuss this Hot Topic? Check out the Hot Topic Forum in the Discussion area
Purpose: The purpose of this study was to determine the effects of international teleradiology attending radiologist coverage (ITARC) of emergency examinations on radiology residents’ perceptions of night call.

Methods: A survey was administered at 2 different radiology residency programs that have attending radiologists who cover the night shift via teleradiology from Israel 5 nights per week. The survey consisted of 12 questions concerning residents’ education and anxiety during on-call shifts and the effects of ITARC on these aspects of residency training. The questions were answered on a scale ranging from 1 to 5, with 3 being neutral.

Results: The radiology residents felt that ITARC improved the on-call learning experience (score 3.7; 1 much worse, 5 much improved). The residents felt neutral about the statements "Review of cases with the attending radiologist over the telephone is comparable educationally to having the attending radiologist in person at the workstation" (score 3.0) and "Having an attending radiologist easily available diminishes the need for me to commit to a diagnosis on my own and is therefore detrimental to my education" (score 2.9; 1 strongly disagree, 5 strongly agree). The residents’ stress levels on call were high without ITARC (score 1.8; 1 very high, 5 very low) and moderate with ITARC (score 2.7). The residents’ anxiety levels before a night on call were moderate without ITARC (score 2.9; 1 very high, 5 very low) and low with ITARC (score 3.7).

Conclusions: Radiology residents felt that ITARC improved their educational experience. International teleradiology attending radiologist coverage also decreased radiology residents’ stress and anxiety related to on-call shifts.

Key Words: Teleradiology, nighthawk, off-hours radiology coverage, resident education

J Am Coll Radiol 2006;3:872-878. Copyright© 2006 American College of Radiology

INTRODUCTION
Over the past 20 years, teleradiology has become a common and effective method of providing radiology night coverage at hospitals throughout the United States [1]. The volume of radiologic studies performed at night in emergency departments has rapidly increased, particularly more complicated studies such as abdominal computed tomography (CT) and CT pulmonary angiography [2], and there has been a demand for the contemporaneous interpretation of these examinations for the sake of patient care. In 2001, the ACR [3] issued a resolution that "all radiologic studies performed on emergency department patients should be promptly interpreted by radiologists." Outsourcing night coverage via teleradiology can improve the efficiency of coverage, with 1 radiologist covering multiple hospitals, and can improve work efficiency during the day by allowing staff radiologists to get adequate rest at night [4,5]. International teleradiology, with the transmission of images abroad, has been used for more than a decade [6].

International teleradiology has been used to cover routine, nonemergent examinations [7,8]. One major benefit of international teleradiology attending radiologist coverage (ITARC) is the time difference that permits a teleradiologist to be awake and alert by working normal daytime hours while covering the night shift in the United States [9,10]. International teleradiology attending radiologist coverage requires the security of a virtual private network and redundancy in Internet connections to eliminate downtime [9]. The rapid transmission of images is necessary to permit contemporaneous interpretations. An international teleradiologist should be licensed in all states and credentialed in all hospitals in which examinations are performed and preferably should be certified by the American Board of Radiology [11].

The majority of hospitals using teleradiology for night coverage are not academic hospitals; however, teleradiology, including ITARC, is also being used in academic hospitals with radiology residency training programs [10,12]. Although most interpretations by residents are accurate, interpretations by attending radiologists are more likely to be accurate [13]. Minor discrepancies between residents’ and attending radiologists’ interpretations of body CT scans have been demonstrated in 5.4% to 26.6% of cases, with major discrepancies in 1.0% to 2.3% [14,15]. In trauma CT cases, Velmahos et al [16] reported minor discrepancies in 11% of cases and major discrepancies in 5% and concluded that attending radiologist coverage is essential to avoid mistakes in these patients. Although Carney et al [15] found that these discrepancies did not adversely affect patient care, contemporaneous interpretations of emergency examinations by attending radiologists are preferable. On the other hand, the experience gained during on-call shifts by radiology residents is important in their assumption of clinical responsibility and their development of critical decision making [15].

At our institutions, ITARC interfaces with radiology residency training. The purpose of this study was to determine the effects of ITARC of emergency examinations on radiology residency training.

METHODS
A survey was administered in March 2004 at 2 different radiology residency programs that have the same attending radiologist providing coverage of the night shift via teleradiology from Israel. The night-shift coverage was provided 5 nights per week from midnight until 8 AM for both Beth Israel Medical Center (BIMC) and St. Luke’s- Roosevelt Hospital Center (SLR). The attending radiologist had subspecialty experience in abdominal imaging. The survey was administered by the program directors of the residency programs.

Both medical centers were digital departments, equipped with picture archiving and communication systems. The attending radiologist had a picture archiving and communication system station and Web browser software that was connected to both BIMC and SLR via a virtual private network. The virtual private network granted him access to all hospital-based imaging and computer systems, including the picture archiving and communication system, Talk Technology voice recognition software for dictation (Talk Technology, Brooklyn, NY), and the IDX radiology information system (IDX Systems Corporation, Burlington, Vt).

The attending radiologist had access to all emergency studies interpreted by residents and usually obtained the images 15 to 30 minutes after they were available to residents. The attending radiologist was accessible by telephone with a local telephone number using the voiceover- Internet protocol (VoIP). The attending radiologist typically interpreted all emergency body CT scans and ultrasound examinations from BIMC and was available to consult on other examinations, such as CT scans of the head and plain radiographs, at the request of an on-call radiology resident. The attending radiologist did not interpret any of the examinations from SLR unless specifically requested to by an on-call resident. For the most part, the examinations from both institutions were interpreted independently by radiology residents before the attending radiologist interpreted the studies. Occasionally, radiology residents contacted the attending radiologist for advice before issuing reports. Differences in the attending and resident radiologists’ interpretations were usually discussed by telephone at the time of the attending radiologist’s interpretation. The resident and attending radiologists were able to discuss cases directly in real time by referencing specific image numbers on a given study. In addition, the attending radiologist’s final reports were available to the radiology residents immediately after dictation.

Twenty first-year and second-year radiology residents participated in the survey, all 11 from BIMC and 9 of 12 from SLR. The residents were unaware of the purpose of the survey until after it was administered. The survey consisted of 12 questions concerning residents’ education and anxiety during on-call shifts and the effects of ITARC on these aspects of residency training. The questions were answered on a scale ranging from 1 to 5, with 5 being the most positive choice.

For each question, we calculated descriptive statistics, including means and standard deviations, and further calculated these statistics by year of residency and location of residency. When questions were asked separately about experiences with and without teleradiology coverage, we calculated differences in responses between the questions. We used the sign test (using exact binomial 2-sided P values, which are appropriate even with small numbers of subjects and data that are not normally distributed) to test whether the median responses were the same with and without teleradiology coverage. We also used the sign test to test whether median responses were more likely to be favorable ( 3) than unfavorable ( 3). We used Stata version 9.1 (StataCorp LP, College Station, Tex) for all calculations.

Table 1. Effects of ITARC on residents’ anxiety and stress levels
  Total P Value First-Year Residents Second-Year Residents BIMC SLR
What was your level of anxiety prior to a night shift during your first week of night float?
Without ITARC 1.5 (0.6)   1.5 1.5 1.5 1.4
With ITARC 2.8 (0.9)   2.5 N/Aa 2.6 3.1
Difference 1.3 (1.0) .0005        
What is your level of anxiety now prior to starting a night of night float?
Without ITARC 2.9 (0.8)   2.5 3.5 3.1 2.7
With ITARC 3.7 (0.8)   3.4 4.1 3.8 3.5
Difference 0.8 (0.6) .0002        
What is your stress level during night float?
Without ITARC 1.8 (0.7)   1.6 2.1 1.9 1.8
With ITARC 2.7 (0.8)   2.6 2.9 2.8 2.6
Difference 0.9 .0001        
Note: 1 very high, 2 high, 3 moderate, 4 low, 5 very low. Standard deviations are in parentheses. BIMC Beth Israel Medical Center; ITARC international teleradiology attending radiologist coverage; N/A not applicable; SLR St. Luke’s-Roosevelt Hospital Center.

aNote that ITARC began 6 months after the second-year residents began night float, and this question therefore did not apply to the second-year residents.


RESULTS
Resident Anxiety and Stress
International teleradiology attending radiologist coverage reduced residents’ anxiety during their first week of night float, with first-year residents demonstrating a reduction from a score of 1.5 (high to very high) to 2.8 (moderate) (P .0005 for testing whether more positive than negative or vice versa; Table 1). Note that ITARC began 6 months after the second-year residents began night float, and therefore, evaluation of the effects of ITARC on their first week of night float was not possible.

International teleradiology attending radiologist coverage also reduced radiology residents’ anxiety before starting shifts of night float at the time of the survey, 3 months after the first-year residents began doing nightfloat shifts and over 1 year after the second year residents began doing night-float shifts. There was an overall decrease in anxiety from a score of 2.9 (moderate) to 3.7 (low) (P .0002), with reductions in first-year residents’ anxiety from 2.5 (moderate to high) to 3.4 (low to moderate) and in second-year residents’ anxiety from 3.5 (low to moderate) to 4.1 (low). Note that the anxiety level for second-year residents was lower than for first-year residents, but ITARC decreased anxiety in both groups.

International teleradiology attending radiologist coverage reduced residents’ stress levels during night float, with overall reductions from 1.8 (high) to 2.7 (moderate) (P .0001) and reductions in first-year residents’ stress levels from 1.6 (high to very high) to 2.6 (moderate to high) and in second-year residents’ stress levels from 2.1 (high) to 2.9 (moderate). Although second-year residents experienced lower stress levels than first-year residents, ITARC still reduced stress in second-year residents. The residents felt, with a score of 4.3 (agree; P .0001), that the presence of an experienced radiologist to help with difficult cases contributed to decreased stress levels (Table 2). Overall, the residents felt neutral, with a score of 3.1 (neutral; P .3), that the presence of an attending radiologist decreased stress by reducing residents’ workloads, but there was a discrepancy at different hospitals, with residents at BIMC agreeing with this statement (score 3.9, P .004) and residents at SLR disagreeing (score 2.2, P .22).

Technical Aspects of ITARC
The residents felt neutral about the statement that reviewing cases with an attending radiologist over the telephone was comparable to reviewing cases with an attending radiologist present, with a score of 3.0 (neutral; P 1.0), with residents at BIMC feeling more positive (score 3.5, neutral to agree) and residents at SLR feeling more negative (score 2.3, disagree).

The residents at both training programs felt that meeting an attending radiologist in person and developing a relationship with the attending radiologist is important, with a score of 3.7 (agree; P .01). In addition, the residents found the use of the VoIP telephone to be a reliable method of communication with the attending radiologist, with a score of 4.4 (agree to strongly agree; P .0001).

Table 2. Effects of ITARC on residents’ education and technical aspects of ITARC
  Total P Value First-Year Residents Second-Year Residents BIMC SLR
Attending physician teleradiology coverage decreases my stress level on-call
Because I know that there is an experienced radiologist to help me with difficult cases. 4.3 (0.7) <.0001 4.3 4.1 4.3 4.2
Because the attending interprets some of the examinations without me when I am overwhelmed with a large volume of cases. 3.1 (1.2) .3 3.0 3.4 3.9 2.2
Review of cases with the attending radiologist over the telephone is comparable educationally to having the attending radiologist in person at the workstation. 3.0 (1.1) 1.0 2.9 3.0 3.5 2.3
Meeting the attending radiologist in person and developing a relationship is an important element of ITARC. 3.7 (0.9) .01 3.6 3.8 3.7 3.6
The telephone connection to the attending radiologist is clear and reliable. 4.4 (0.6) <.0001 4.2 4.8 4.5 4.3
Having an attending radiologist easily available diminishes the need for me to commit to a diagnosis on my own and is therefore detrimental to my education. 2.9 (1.2) 1.0 2.8 3.0 2.9 N/Aa
1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree. Standard deviations are in parentheses. Abbreviations as in Table 1. aNote that the this question was inadvertently omitted from the survey at SLR and was therefore part of the survey only at BIMC.


Educational Effects of ITARC
The residents at BIMC felt indifferent, with a score of 2.9 (neutral; P 1.0), that the availability of an attending radiologist was detrimental to their education by diminishing their need to commit to diagnoses on their own. Note that this question was inadvertently omitted from the survey at SLR and was therefore part of the survey only at BIMC.

The residents felt that ITARC improved their on-call learning experience, with a score of 3.7 (improved; P .004; Table 3). The residents found the learning experience on night float slightly better than other rotations, with a score of 3.6 (no change to improved; P .12) without ITARC and a score of 3.9 (improved; P .001) with ITARC.

Contemporaneous review of cases with the attending radiologist over the telephone was better for residents’ education than residents’ reviewing final reports the fol- lowing night, with a score of 4.0 (improved; P .04), and was not significantly different from review of cases with an attending radiologist in person in the morning, with a score of 3.2 (no change; P .45).

Table 3. Educational effects of ITARC
  Total P Value First-Year Residents Second-Year Residents BIMC SLR
How does attending physician teleradiology coverage affect the on-call learning experience? 3.7 (0.8) 0.004 3.8 3.7 3.6 4.0
Compared to other rotations, what is the quality of your learning experience during night float?
Without ITARC 3.6 (1.1) 0.12 3.4 3.9 4.1 2.9
With ITARC 3.9 (0.8) 0.001 3.9 3.9 4.1 3.7
Difference 0.3 (0.9) 0.29        
Educational value of reviewing cases with the attending radiologist over the telephone
Compared to reviewing the final reports the following night (BIMC only) 4.0 (1.0) 0.04 3.7 4.4 4.0 N/A
Compared to reviewing the studies with an attending radiologist in the morning (SLR only) 3.2 (1.1) 0.45 3.5 2.7 N/A 3.2
1 = much worse, 2 = worse, 3 = no change, 4 = improved, 5 = much improved. Standard deviations are in parentheses.
Abbreviations as in Table 1.


DISCUSSION
This study examined the effect of ITARC on the training of radiology residents. The survey was distributed to the first-year and second-year radiology residents at 2 training programs in New York City. These classes of residents were selected because they were the ones who regularly covered the night-float rotations and therefore the ones who interacted directly with the attending teleradiologist. Note that all the residents who participated in the survey had the opportunity to work with and without ITARC, because the attending teleradiologist worked with the residents 5 nights per week and the residents worked alone on the other 2 nights. Although the survey was prepared by the attending teleradiologist, it was administered by the program directors without notifying the residents that the attending teleradiologist was involved in the survey.

The first aspect of the survey concerned residents’ anxiety and stress. Anxiety was considered to be apprehension before an on-call shift, whereas stress was considered to be a state of worry or mental tension during an on-call shift. International teleradiology attending radiologist coverage reduced the radiology residents’ anxiety during their first week of night float and after multiple shifts of night float. Although anxiety levels were lower in residents with more experience, including first-year residents with 3 months of experience and second-year residents, all these groups demonstrated reductions in anxiety attributable to ITARC. In addition, stress levels, although higher in first-year residents than in second-year residents, were reduced in both classes by the presence of ITARC. These reductions in anxiety and stress levels were statistically significant.

In examining the reasons behind these reductions in stress with ITARC, the residents felt strongly that the presence of an experienced radiologist to help with difficult cases was a major reason for the decreased stress. Overall, the residents felt neutral that the presence of an attending radiologist decreased stress by reducing their workloads. However, at SLR, the radiology residents consulted the attending radiologist only for difficult cases, and the attending radiologist did not routinely interpret the cases from SLR and therefore did not typically reduce those residents’ workloads. At BIMC, where the attending radiologist routinely interpreted all of the emergent body CT cases, sometimes before the residents had the opportunity to interpret the cases, the residents did find that the reduction of their workloads by the attending radiologist reduced their stress levels at times of a high volume of cases.

The other major component of the survey concerned the effect of ITARC on residents’ education. Across the board, the residents felt that ITARC improved their oncall learning experience, and this finding was statistically significant. This positive opinion applied to both first- year and second-year residents and to both radiology residency training programs. The residents also felt that the learning experience on night float was slightly better than on other rotations and was slightly better with ITARC than without ITARC, although this finding was not statistically significant.

The residents at BIMC felt strongly that the contemporaneous review of cases with the attending radiologist over the telephone was clearly preferable to residents’ reviewing final reports on their own the following night. The residents at SLR demonstrated a slight preference for contemporaneous review with an attending radiologist by telephone over the review of cases in the morning with an attending radiologist in person. The residents felt neutral about the statement that reviewing cases with the attending radiologist over the telephone was comparable with reviewing cases with an attending radiologist present. This implies that the in-person review of cases is preferable but that review by telephone is an adequate method. Note that the BIMC residents felt more positive about this statement than the SLR residents. This difference may be due to the BIMC residents’ comparing telephone review with the review of reports on their own, although this was technically not part of the survey question. Although the SLR residents felt that the in-person review of cases was better than telephone review, the fact that they found contemporaneous review by telephone at least equal to later review in person implies that they had a preference for contemporaneous review, as would be expected intuitively. Therefore, the presence of an attending radiologist with whom residents can discuss cases contemporaneously, whether by telephone or in person, improves the educational experience of radiology residents.

One potential concern about an attending radiologist’s presence during on-call shifts is the perceived detrimental effect of the diminished need for residents to commit to diagnoses on their own. The survey demonstrated that the residents felt neutral that the availability of the attending radiologist would have a detrimental effect on their education. At our institutions, the attending radiologist usually allows radiology residents to interpret examinations first on their own. Even when radiology residents, before issuing preliminary reports, call the attending radiologist for advice on interpreting cases, the attending radiologist usually asks the residents for their opinions before offering advice. Therefore, if an attending radiologist is careful to permit radiology residents to act independently and accept responsibility for their interpretations to the extent possible without significantly affecting patient care, there will not be a significant detrimental effect on residents’ education and decision making.

The radiology residents at both training programs felt that meeting attending radiologists in person and developing relationships with them is important. At our institution, the attending teleradiologist typically comes to New York twice a year to work for a few days at the hospitals. He usually gives a few conferences to the residents from all the affiliated training programs. In particular, one of these visits is during the first 6 months of residency training, so that the attending radiologist can meet the first-year residents before they start the nightfloat rotation.

The residents found the use of the VoIP telephone to be a satisfactory method of communication with the attending radiologist. The VoIP telephone allows the attending radiologist to have a local New York telephone number while working from abroad. In addition, many VoIP setups have fixed monthly fees, independent of the number of telephone calls made or minutes used. This fixed fee removes the potential disincentive of cost from communication between the attending teleradiologist and health care workers, including residents, at hospitals. A VoIP telephone therefore allows an attending teleradiologist and radiology residents to have easy and unlimited communication that can enhance residents’ education.

A pitfall of this survey was the small number of residents surveyed. However, because ITARC is a relatively new phenomenon, only the first-year and second-year residents had an opportunity to work significantly with the attending teleradiologist. In addition, 3 of 6 secondyear residents at SLR were not available at the time of the survey and therefore did not participate. Despite the small numbers, the effects of ITARC were in many cases sufficiently large to be statistically significant and not plausibly due to random variability.

Another limitation of this study is that its results may not apply to ITARC at other institutions. The attending teleradiologist involved in this survey has a particular interest in teaching residents. The attending radiologist and the residents developed relationships, and the attending radiologist provided feedback to the residents on a regular basis while allowing them independence in formulating their own interpretations of examinations. These conditions may need to be met for ITARC to have similar results at other academic institutions.

CONCLUSIONS
Attending radiologist night coverage is becoming more common, particularly with ITARC. The obvious benefits include improved accuracy of readings after hours, availability of an attending radiologist to consult with referring physicians, and decreased burden of daytime attending radiologists. Academic radiology departments with residency training programs are beginning to use ITARC at night. Although ITARC improves patient care, it also affects the educational experiences of radiology residents. These effects may vary depending on the interest in teaching of the attending radiologist and the setup for both resident and attending radiologist on-call coverage. Our data show that radiology residents felt that ITARC improved their educational experiences. International teleradiology attending radiologist coverage also decreases radiology residents’ stress and anxiety related to on-call shifts. Instant feedback and the ability to troubleshoot difficult cases with an attending teleradiologist result in an enhanced night-float experience without depriving residents of the ability to make independent decisions.
Want to discuss this Hot Topic? Check out the Hot Topic Forum in the Discussion area
REFERENCES
aDepartment of Radiology, Beth Israel Medical Center, New York, NY.
bDepartment of Radiology, Lenox Hill Hospital, New York, NY.
cDepartment of Biostatistics and Epidemiology, University of Pennsylvania School of Medicine, Philadelphia, Pa.

Corresponding author and reprints: Sandor A. Joffe, MD, Department of Radiology, Beth Israel Medical Center, 1st Ave. at 16th St., New York, NY 10003. e-mail: sjoffe@chpnet.org.


  1. DeCorato DR, Kagetsu NJ, Ablow RC. Off-hours interpretation of radiologic images of patients admitted to the emergency department: efficacy of teleradiology. Am J Roentgenol 1995;165:1293-6.

  2. Rosen MP, Sands DZ III, Longmaid E, Reynolds KF, Wagner M, Raptopoulos V. Impact of abdominal CT on the management of patients presenting to the emergency department with acute abdominal pain.AmJ Roentgenol 2000;174:1391-6.

  3. American College of Radiology. Timely interpretation of emergency radiology studies. Reston, Va: American College or Radiology; 2002.

  4. Wong WS, Roubal I, Jackson DB, Paik WN, Wong VKJ. Outsourced teleradiology imaging services: an analysis of discordant interpretation in 124,870 cases. J Am Coll Radiol 2005;2:478-84.

  5. Wagner AL. After-hours coverage: problems and solutions. J Am Coll Radiol 2004;1:351-5.

  6. Goldberg MA, Sharif HS, Rosenthal DI, et al. Making global telemedicine practical and affordable: demonstrations from the Middle East. Am J Roentgenol 1994;163:1495-500.

  7. Kalyanpur A, Neklesa VP, Pham DT, Forman HP, Stein ST, Brink JA. Implementation of an international teleradiology staffing model. Radiology 2004;232:415-9.

  8. Boland G, Whelan P, Schultz T, Rincon S, Dreyer K, Mueller P. Teleradiology conflicts in an academic department: leveraging expertise remotely. Radiology 2001;221(P):370.

  9. Bradley WG. Offshore teleradiology. J Am Coll Radiol 2004;1:244-8.

  10. Kalyanpur A, Weinberg J, Neklesa VP, Brink JA, Forman HP. Emergency radiology coverage: technical and clinical feasibility of an international teleradiology model. Emerg Radiol 2003; 10:115-8.

  11. Moore AV, Bibb A, Campbell S, et al. Report of the ACR Task Force on International Teleradiology. Available at: http://www.acr.org/04meeting/ taskForceReports/finalWhitePaper.pdf.

  12. Larson DB, Cypel YS, Forman HP, Sunshine JH. A comprehensive portrait of teleradiology in radiology practices: results from the American College of Radiology’s 1999 survey. Am J Roentgenol 2005;185:24-35.

  13. Eng J, Mysko WK, Weller GE, et al. Interpretation of emergency department radiographs: a comparison of emergency medicine physicians with radiologists, residents with faculty, and film with digital display. Am J Roentgenol 2000;175:1233-8.

  14. Wechsler RJ, Spettell CS, Kurtz AB, et al. Effects of training and experience in interpretation of emergency body CT scans. Radiology 1996;199: 717-20.

  15. Carney E, Kempf J, DeCarvalho V, Yudd A, Nosher J. Preliminary interpretations of after-hours CT and sonography by radiology residents versus final interpretations by body imaging radiologists at a level I trauma center. Am J Roentgenol 2003;181:367-73.

  16. Velmahos GC, Fili C, Vassiliu P, Nicolaou N, Radin R, Wilcox A. Around-the-clock attending radiology coverage is essential to avoid mistakes in the care of trauma patients. Am Surg 2001;67:1175-7. 878 Journal of the American College of Radiology/ Vol. 3 No. 11 November 2006




    ACR Logo


ACR