Comparative Validation of Online Nomograms for Predicting Nonsentinel Lymph Node Status in Sentinel Lymph Node–Positive Breast Cancer FREE

Axillary lymph node status is the most clinically significant factor in the management of invasive breast cancer.^1- 3 In the 1990s, clinical guidelines replaced axillary lymph node dissection (ALND) with sentinel lymph node (SLN) biopsy for assessing regional lymph node involvement in patients with early-stage breast cancer and clinically negative nodes.^2,4 This practice reduces the morbidity associated with ALND in those patients who have a negative SLN and who are therefore at low risk for additional nodal disease.

Completion ALND is the current standard of care for patients with a positive SLN. However, there is ongoing scrutiny regarding the need for completion ALND in every patient with a positive SLN. Following ALND, only 35% to 50% of patients are found to have metastases within nonsentinel lymph nodes (NSLNs), and axillary recurrence following systemic treatment is low.^5- 8 This translates into additional surgery and increased risk of morbidity, possibly without additional therapeutic benefit, for a significant subset of patients. A recent randomized trial of SLN biopsy alone vs completion ALND for SLN-positive patients with early-stage breast cancer found no difference in 5-year survival.⁹ Indeed, the idea that ALND may not affect overall survival was first suggested by the National Surgical Adjuvant Breast and Bowel Project B-04 trial published nearly 35 years ago.¹⁰

Accordingly, several mathematical models have been developed to help predict whether tumor has metastasized to NSLNs when the SLN is positive. The most widely used models include 4 nomograms (Memorial Sloan-Kettering Cancer Center [MSKCC], Mayo, Cambridge, and Stanford) and 3 scoring systems (Tenon, M. D. Anderson, and Saidi).^{6- 7,11- 15} These models use an array of primary tumor characteristics (tumor size, histologic grade, lymphovascular involvement, and hormonal receptors) and nodal factors (number of positive SLNs, detection method, and size of metastases) to calculate the risk of NSLN metastasis. Only 2 of these models, the Stanford and MSKCC nomograms, are available as online calculators. Online availability allows easy, unrestricted access by physicians at any institution worldwide, and the calculator tool functions to generate results quickly, facilitating its use in patient counseling and tumor board discussions. The purpose of our study was to compare the accuracy and clinical utility of these 2 online calculators in predicting NSLN metastasis in SLN-positive breast cancer patients during a 10-year period at our institution.

We conducted a review of a prospectively maintained database of all patients who underwent SLN biopsy for early-stage breast cancer (defined as T1-T3 and clinically node negative) at Oregon Health & Science University in Portland, Oregon, from October 1, 1999, through January 31, 2008. Approval from the institutional internal review board was obtained before data collection. Our inclusion criteria were as follows: presence of an invasive breast cancer (T1-T3), a clinically negative axilla, SLN biopsy with tumor metastasis in at least 1 node, performance of a completion ALND, and documentation of all variables as required by each nomogram. Patients who did not meet these criteria were excluded from the study. Risk of NSLN metastasis was calculated for every patient using the online calculator for each nomogram.

Our methods for SLN biopsy and pathology review have previously been described in detail.¹⁶ In brief, all SLNs were identified using a combination of technetium Tc 99 sulfur colloid tracer and isosulfan or methylene blue dye. Sentinel lymph nodes were immediately examined by frozen section and interpreted by a surgical pathologist. If the frozen section was positive, an axillary dissection was completed in the same procedure.

We re-reviewed the pathologic characteristics for all positive SLNs to confirm size of tumor and method of detection. The size of metastatic tumor deposits within all nodes was categorized according to the American Joint Committee on Cancer StagingManual (7th ed): isolated tumor cells (ITC) were defined as tumor deposit of 0.2 mm or less (pN0i+), micrometastases (MI) were defined as tumor deposit of more than 0.2 mm but not more than 2 mm (pN1mi), and macrometastases were defined as tumor deposit of more than 2 mm (pN1).¹⁷ Maximal size of SLN tumor deposit was used for nomogram calculations.

Calculations were made using the MSKCC “Additional Nodal Metastasis” online calculator (http://www.mskcc.org/applications/nomograms/breast). The nomogram provides a risk estimate of NSLN metastasis based on 9 histopathologic variables: frozen section analysis (yes/no), primary tumor size (in centimeters), primary tumor type (ductal/lobular) and nuclear grade (1-3), number of positive SLNs, method of SLN detection (routine hematoxylin and eosin, serial hematoxylin and eosin, or immunohistochemistry), number of negative SLNs, lymphovascular invasion (yes/no), multifocality (yes/no), and estrogen receptor positivity (yes/no).⁷ A tumor is considered estrogen-receptor positive if at least 10% of cells stain positive within the specimen. In our study, the Nottingham Histologic Score was used in place of nuclear grade, and lymphovascular invasion was scored as negative when not stated in the pathology report.

Calculations were also made using the “Stanford Online Calculator” for NSLN metastasis (https://www3-hrpdcc.stanford.edu/nsln-calculator/). This nomogram requires 3 histopathologic variables: primary tumor size (in centimeters), size of SLN metastasis (ITC, MI, and macrometastases), and angiolymphatic invasion (yes/no).⁶ Similar to MSKCC, the Stanford nomogram provides a predicted probability of having positive NSLNs.

The performance of each nomogram was evaluated by drawing a receiver operating characteristic (ROC) curve and calculating the area under the curve (AUC). The AUC reflects a test's ability to discriminate between diseased and nondiseased patients.⁵ An AUC of 1 indicates perfect concordance (100% sensitivity and 100% specificity), whereas an AUC of 0.5 indicates a result equal to chance. The 95% confidence interval (CI) for each AUC was determined using a 2000-replicate bootstrap procedure.

The nomograms were further assessed using false-negative rate, positive predictive value, and calibration plots. The false-negative rate (ie, the probability that the nomogram predicts there is no disease when there actually is disease) for nomogram values of 10% or less was used to assess the ability of each nomogram to accurately identify patients at very low risk for NSLN metastasis. The positive predictive value (ie, the probability that the patient has disease given a positive test result) for nomogram values of 80% or greater was used to assess the ability of each nomogram to accurately identify patients at very high risk for NSLN metastasis. The accuracy of each nomogram was determined by calibration plot, which compares predicted (nomogram) probabilities to the observed incidence of NSLN metastasis. Predicted probabilities were represented in quintiles to increase homogeneity within groups while minimizing the size of the 95% CIs. Finally, we tested both nomograms in a subset of patients who had only ITC or MI disease in the SLN. Previous validation studies have shown that nomograms often underestimate or overestimate NSLN metastasis in this low-risk population.^5,18- 19 Statistical analysis was performed using R, version 2.12.1 (R Foundation for Statistical Computing, Vienna, Austria).²⁰

From October 1, 1999, through January 31, 2008, there were 579 SLN biopsies for invasive breast cancer performed at our institution, of which 179 (30.9%) had at least 1 positive SLN. Of those, 123 patients met our inclusion criteria. Fifty-six patients were excluded: 26 did not undergo completion ALND and 30 were missing a required variable for the nomograms. The mean number of SLNs identified was 2.3 (range, 1-6), and the mean number of nodes removed during axillary dissection was 15 (range, 8-40). Most women who underwent axillary dissection had invasive ductal carcinoma (87.8%) and an SLN with macrometastatic disease (73.2%). At least 1 positive NSLN was found in 52 patients (42.2%), and ALND resulted in upstaging of 25 patients (20.3%). The descriptive tumor and nodal characteristics of our population as used in the MSKCC and Stanford nomograms are listed in the Table.

The ROC curves for the MSKCC and Stanford nomograms are shown in Figure 1. The AUC values were 0.72 and 0.70, respectively. There were no statistically significant differences between the curves given that the 95% CIs overlap. False-negative rates for nomogram values of 10% or less were low for each nomogram: 4.1% for the MSKCC (95% CI, 0%-9.2%) and 7.8% for the Stanford (95% CI, 1.7%-14.4%). The positive predictive value for nomogram probabilities of 80% or greater was higher for MSKCC than Stanford: 90.9% for the MSKCC (95% CI, 70.0%-100.0%) and 61.8% for the Stanford (95% CI, 45.7%-73.6%). On a calibration plot (Figure 2), predicted rates of NSLN metastasis were closer to observed rates for MSKCC than for Stanford.

We also tested the performance of each nomogram in a subset of low-risk patients with SLN metastasis size of 2.0 mm or less (ITC and MI). Thirty-three patients (26.8%) were identified from our original study group as having ITC or MI as the largest tumor deposit within the SLN. Of these patients, 11 (33.3%) had NSLN metastases (3 patients with ITC disease and 8 patients with MI disease). The ROC curve for this subanalysis is shown in Figure 3. The MSKCC nomogram produced an AUC of 0.75 and the Stanford nomogram produced an AUC of 0.70. The difference was not statistically significant. On calibration plot, the Stanford nomogram was closer to the ideal predictor, indicating better accuracy in this subgroup.

Adequately assessing nodal status is essential to accurate staging and treatment planning in early-stage breast cancer.^1,3,8,21 Completion ALND, which has been the standard of care for SLN-positive patients, has the advantage of more accurate staging, focused treatment planning, and possibly a lower rate of recurrence.⁸ However, NSLNs are often negative for tumor, and the risk of morbidity associated with a completion ALND is not insignificant. Following a completion ALND, 10% to 15% of patients will develop permanent lymphedema in the adjacent arm.^22- 23 Additional complications include wound infection, seroma, neurologic damage, arm weakness, and decreased upper extremity range of motion.²² Recently, Giuliano et al⁹ reported results from the American College of Surgeons Oncology Group Z0011 study, a randomized, multicenter trial that showed that ALND did not significantly affect overall or disease-free survival at 5 years for SLN-positive patients with T1 or T2 breast cancer when treated with lumpectomy, adjuvant systemic therapy, and tangential-field whole-breast radiation. Thus, physicians must balance understaging with overtreating the clinically negative axilla.

Accordingly, several studies have attempted to identify tumor and nodal characteristics that can predict the presence of positive NSLNs. These studies have shown primary tumor size and SLN metastasis size to be most closely associated with NSLN involvement.^7,11,24- 25 Compared with other nomograms that have been developed to predict NSLN metastasis, the online availability of the MSKCC and Stanford nomograms makes them widely accessible and clinically applicable. For this reason, we focused our validation on these 2 nomograms.

The MSKCC nomogram, published in 2003 by Van Zee and colleagues,⁷ was the first nomogram of its kind to predict NSLN metastasis following a positive SLN biopsy. When applied prospectively to a group of 373 patients, the model produced an AUC of 0.77. Subsequent validation studies have shown an AUC ranging from 0.58 to 0.84.^{5,14,18- 19,25- 26} In 2008, a modified nomogram using only 3 histopathologic variables was developed at Stanford by Kohrt and colleagues.⁶ The nomogram was developed using a data set of 171 patients, in which most (96.7%) had an SLN with ITC or MI disease. The nomogram was validated against the MSKCC nomogram using an independent data set of 77 patients with predominantly macrometastases disease. The AUC was 0.74 and 0.62 for the Stanford and MSKCC nomograms, respectively. Subsequent validation studies of the Stanford nomogram have demonstrated an AUC of 0.64 to 0.73.^5,25- 26

In our population of 123 SLN-positive patients, the nomograms had similar AUCs (0.72 for the MSKCC and 0.70 for the Stanford) with no statistically significant difference. Previous studies have relied on the AUC alone to validate nomograms; however, we further assessed their performance using the false-negative rate, the positive predictive value, and calibration plots. The nomograms performed similarly when identifying patients at low risk for NSLN metastases, but the MSKCC nomogram demonstrated an increased ability to correctly predict patients at high risk for NSLN disease. This difference in performance was more clearly seen in the calibration plots (Figure 2), which showed that across all nomogram values, the MSKCC-predicted probabilities were closer to the observed rate of NSLN metastasis.

Identification of ITC and MI disease in the SLN has become increasingly common with the use of serial sectioning and immunohistochemistry on pathologic evaluation. The clinical significance and management of these findings remain unclear. The reported incidence of NSLN metastasis in patients with ITC ranges from 4.7% to 16%^6,27- 28 and in patients with MI disease from 12% to 42%.^6,28 As mentioned previously, the Stanford nomogram was developed around a population of patients with mostly ITC and MI disease, and unlike the MSKCC nomogram, the size of the SLN metastasis factors into its calculations. When we tested the accuracy of the nomograms in a subset of patients with only ITC or MI disease, the Stanford nomogram closely matched the observed incidence of NSLN involvement (Figure 4).

In light of the American College of Surgeons Oncology Group Z0011 trial, some physicians have questioned the continued need for nomograms in clinical decision making. On the contrary, we believe that need for nomograms will increase in the future because there will be a growing cohort of SLN-positive early-stage breast cancer patients with undissected axillae. Among these patients, nomograms can be used to provide an estimated risk of residual axillary disease and, potentially, of axillary relapse. Such a cohort has already been the subject of a preliminary report from MSKCC published in 2004 by one of our coauthors (A.M.N.). In fact, this analysis⁸ suggested a higher axillary relapse rate among patients not undergoing ALND.

Furthermore, the findings from the American College of Surgeons Oncology Group Z0011 trial are only applicable to a limited number of breast cancer patients (patients with T1-T2 primary tumors and a positive SLN who undergo lumpectomy, adjuvant systemic therapy, and tangential-field whole-breast radiation). The trial did not address patients with early-stage disease who elect mastectomy, patients undergoing neoadjuvant chemotherapy who have clinically negative axillae but a positive SLN, or patients in whom axillary radiation is being considered. For these patients, nomograms are still applicable because axillary dissection remains the standard of care.

There were some limitations to our study. First, because a nomogram could not be applied unless all histopathologic variables were known, 30 patients were excluded from the study. Most commonly missing was SLN metastasis size, which is a variable required by the Stanford nomogram. Although the MSKCC nomogram requires more variables, this was not a limiting factor in our study population. Another limitation was our relatively small population size, particularly affecting the subanalysis of patients with ITC and MI disease. Furthermore, as the use of nomograms in our multidisciplinary breast cancer conference increased, some patients with very small nodal tumor deposits did not undergo ALND dissection and thus were not included in the study. In fact, use of nomograms has been shown to reduce the number of ALND.²⁹ An interesting future study would be the correlation of nomogram probabilities with clinical outcomes in patients treated with SLN biopsy alone.

In conclusion, our study is unique in that it offers a more comprehensive validation of the MSKCC and Stanford nomograms, thereby elucidating the clinical utility of each nomogram. Despite similar AUC values between the nomograms, the MSKCC model more accurately identified patients at high risk for NSLN metastasis; therefore, it may be more useful when considering a completion ALND. Conversely, the Stanford nomogram more accurately predicted NSLN involvement in a subset of patients with ITC and MI disease and should be considered for use in these patients. However, when calibrated across all nomogram values, the MSKCC nomogram was consistently more reliable in predicting actual NSLN pathologic outcomes in our study population.