Risk Analysis Index and Its Recalibrated Version Predict Postoperative Outcomes Better Than 5-Factor Modified Frailty Index in Traumatic Spinal Injury
Article information
Abstract
Objective
To assess the discriminative ability of the Risk Analysis Index-administrative (RAI-A) and its recalibrated version (RAI-Rev), compared to the 5-factor modified frailty index (mFI-5), in predicting postoperative outcomes in patients undergoing surgical intervention for traumatic spine injuries (TSIs).
Methods
The Current Procedural Terminology (CPT) and International Classification of Disease-9 (ICD-9) and ICD-10 codes were used to identify patients ≥ 18 years who underwent surgical intervention for TSI from National Surgical Quality Improvement Program (ACS-NSQIP) database 2015–2019 (n = 6,571). Multivariate analysis and receiver operating characteristic (ROC) curve analysis were conducted to evaluate the comparative discriminative ability of RAI-Rev, RAI-A, and mFI-5 for 30-day postoperative outcomes.
Results
Multivariate regression analysis showed that with all 3 frailty scores, increasing frailty tiers resulted in worse postoperative outcomes, and patients identified as frail and severely frail using RAI-Rev and RAI-A had the highest odds of poor outcomes. In the ROC curve/C-statistics analysis for prediction of 30-day mortality and morbidity, both RAI-Rev and RAI-A outperformed mFI-5, and for many outcomes, RAI-Rev showed better discriminative performance compared to RAI-A, including mortality (p = 0.0043, DeLong test), extended length of stay (p = 0.0042), readmission (p < 0.0001), reoperation (p = 0.0175), and nonhome discharge (p < 0.0001).
Conclusion
Both RAI-Rev and RAI-A performed better than mFI-5, and RAI-Rev was superior to RAI-A in predicting postoperative mortality and morbidity in TSI patients. RAI-based frailty indices can be used in preoperative risk assessment of spinal trauma patients.
INTRODUCTION
Traumatic spinal injury (TSI) includes injuries to the spinal cord, nerve roots, osseous structures, and discoligamentous components of the spinal column [1-3]. The main cause for spinal injury is blunt trauma, most commonly caused by motor vehicle accidents, followed by falls and sport injuries [1-3]. Spinal column injury can cause mechanical instability, impaired movements, and damage to neural structures can lead to partial or complete paralysis [1-3]. Among TSI, traumatic spinal cord injury (tSCI) is a subset, which leads to neurologic deficits secondary to traumatic injury [4]. The yearly incidence of TSI has been estimated to be 54 cases per 1 million people in the United States or about 17,810 new TSI cases each year [5]. Irrespective of TSI type, these injuries are a major cause of disability, with significant socioeconomic consequences [1-3].
While historically, TSI has been associated with average age in 40s and predominantly males, there has been a change in the epidemiological trends in recent years with an aging population, and the average of TSI continues to increase [1,5 6]. It has been predicted that major proportion of new TSI will occur in patients above 70 years of age [7], with similar projections for overall TSI [1,3]. It is imperative to perform risk stratification and to identify prognostic factors for TSI patients [8].
Frailty, a measure of physiological reserve, broadly defined as the cumulative burden of baseline comorbid conditions and functional status impairment, has been associated with worse postoperative outcomes across surgical subspecialties [9-13]. In recent years, several studies have assessed the impact of baseline frailty status on postoperative outcomes in patients undergoing spine surgery [14], though its application to spine trauma and TSI has been limited to 2 studies [15,16]. The lack of large scale, high quality, clinical studies on frailty and spine trauma has been emphasized before [17]. Furthermore, majority of previous studies on frailty and spine surgery outcomes have employed the modified frailty index-11 (mFI-11), or its abridged version, modified frailty index-5 (mFI-5) for preoperative frailty status assessment [14]. While the mFI indices have been classically used for frailty assessment, however mFI are more a measure of comorbidity rather than frailty. Though functional status has been commonly associated with the definition of frailty, the variables of diabetes, chronic obstructive pulmonary disease (COPD), hypertension, and congestive heart failure (CHF) are more commonly defined as comorbidities [18,19].
To more precisely incorporate the multifactorial nature of frailty, Hall et al. [20,21] developed the Risk Analysis Index (RAI) in 2017 in an effort to provide an effective screening tool to assess the frailty of surgical patients. This frailty index including both the prospective clinical RAI and the retrospective administrative RAI (RAI-A) could be easily calculated from variables captured by the Veterans Affairs Surgical Quality Improvement Program and the American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP) databases includes both comorbidities and functional status variables that robustly measures baseline frailty status, and has recently been validated in a prospective, single-center spine surgery study [22], and also by us in brain tumor resection patients [23]. Since its development the RAI has demonstrated significant utility in predicting 30-, 180-, and 365-day postoperative outcomes across multiple surgical subspecialties [20,21,24-28]. Recently, Arya et al. [24] have recalibrated and improved the original RAI (RAI-Rev) in order to better discriminate 30-day mortality and morbidity. Presently, there is a gap within the literature demonstrating the predictive utility of RAI-A and RAI-Rev within spine surgery, and, more specifically, in patients undergoing surgery for spinal trauma. The present study was performed to assess the predictive capability of RAI-A and RAI-Rev on 30-day postoperative outcomes in traumatic spine surgery patients utilizing large scale data from NSQIP. Based on the reported robustness of RAI-A and RAI-Rev in predicting postoperative outcomes in other surgical specialties, we hypothesized that RAI-A and RAI-Rev would be superior to the mFI-5 in predicting postoperative 30-day mortality and morbidity in patients undergoing surgery for spinal trauma.
MATERIALS AND METHODS
1. Data Source
The NSQIP is a nationally validated, multi-institutional database of over 700 participating hospitals with > 200 variables collected for pre-, intra-, and postoperative outcomes. Data are entered from each institution by ACS-trained surgical clinical reviewers to ensure consistency and reliability [29,30]. Our data were extracted from the NSQIP database for the years 2015 to 2019. The present study was performed under the data user agreement of the ACS with the University of New Mexico (UNM) and was approved by the Institutional Review Board of UNM School of Medicine (Study ID 21-315).
2. Patient Population and Baseline Characteristics
Current Procedural Terminology codes (Supplementary Table 1) were used to identify all patients over 18 years old that had undergone spine surgery in the NSQIP dataset. International Classification of Diseases (ICD)-9 and ICD-10 diagnostic codes were then used to identify spinal trauma patients (Supplementary Table 2). Forty-six patients were removed due to primary diagnosis codes unrelated to spinal trauma, and another 77 were excluded due to missing length of stay (LOS) duration. The final sample size of patients who underwent surgery for spinal trauma was 6,571. The baseline study variables included age, sex, body mass index (BMI), elective versus nonelective surgery type, LOS, transfer status, and operative time. The analyzed medical comorbidities included diabetes mellitus, COPD, CHF, dyspnea, hypertension, disseminated cancer (defined as multiple metastases by NSQIP), open wound, steroid use, weight loss (substantial unintentional weight loss > 10%), bleeding disorders, preoperative transfusion, transfer status, and preoperative sepsis/septic shock/systemic inflammatory response (SIRS). Preoperative SIRS criteria are defined by NSQIP as the presence of at least 2 of the following criteria: temperature > 38°C or < 36°C, heart rate > 90 beats per minute, respiratory rate > 20 breaths per minute or PaCO2 < 32 mmHg, leukocytosis or leukopenia (white blood cell count > 12,000/mm3 and < 4,000/mm3, respectively) or > 10% immature (band) forms, or anion gap acidosis [23,29]. Additional preoperative comorbidities extracted included functional dependence (both complete and partial dependence) and smoking status.
3. Retrospective Risk Analysis Index Scoring (RAI-A and RAI-Rev)
Retrospective RAI-A and the recalibrated RAI-Rev scoring, adapted from Hall et al. and Arya et al. are shown in Supplementary Table 3. RAI scoring system is based on 11 variables: sex, age, cancer diagnosis (excluding melanoma), weight loss defined as unintentional weight loss of 4.5 kg over 3 months, renal failure, CHF, poor appetite, shortness of breath at rest, residence defined as not independent living, cognitive deterioration, and activities of daily living (ADL) defined as functional status. Age scoring is related to having a cancer diagnosis, which can be seen in Supplementary Tables 4 and 5 for RAI-A and RAI-Rev, respectively. Cognitive deterioration over the past 3 months was originally included in Hall’s scoring method for RAI-A to be included with ADL scoring, however preoperative cognitive decline is not recorded in NSQIP and therefore was excluded from this study’s adaptation of Hall’s RAI-A scoring. Score stratification of RAI-A and RAI-Rev as they relate to frailty tiers is demonstrated in Supplementary Table 6 (nonfrail ≤ 10, prefrail 11–20, frail 21–30, and severely frail ≥ 31 RAI-A and RAI-Rev scores).
4. Modified Frailty Index-5 (mFI-5)
Although the modified frailty index was initially developed with 11 variables (mFI-11), in 2014, the NSQIP stopped mandating the reporting of some of the mFI-11 preoperative variables and as such the mFI-5 was adapted based on these 5 remaining variables: diabetes mellitus, hypertension, dependent functional status, COPD, and CHF (Supplementary Table 7). The presence of each mFI-5 variable receives one point. Thus, the mFI-5 scores range from 0 to 5, where a score of 0 is “nonfrail,” 1 is defined as “prefrail,” 2 as “frail,” and a score of 3 or more as “severely frail.”
5. Outcome Measures
The outcome measures included 30-day mortality, major complications, Clavien-Dindo physical status (PS) classification grade IV complications, 30-day unplanned readmission, 30-day unplanned reoperation, extended LOS (ELOS), and discharge destination. Major complications consisted of presence of one of the following: prolonged intubation exceeding 48 hours, unplanned reintubation, sepsis, septic shock, pneumonia, deep vein thrombosis/thrombophlebitis, pulmonary embolism (PE), acute cerebrovascular accident or stroke with neurological deficit, acute renal failure, myocardial infarction (MI), cardiac arrest requiring cardiopulmonary resuscitation, superficial surgical site infection (SSI), deep incisional SSI, organ space SSI, or wound disruption. Minor complication was defined as intra-/postoperative blood transfusion, renal insufficiency, or urinary tract infection. Clavien-Dindo PS classification grade IV complications were designated by the presence of life-threatening complications defined by single or multiple organ dysfunction requiring intensive care unit management. Clavien-Dindo PS classification grade IV complications included: sepsis or septic shock, acute renal failure, PE, MI, cardiac arrest requiring cardiopulmonary resuscitation, ventilation >48 hours, and unplanned reintubation [31]. ELOS was defined as > 75th percentile LOS of study population. The “nonhome discharge” location encompassed all patients discharged to any rehabilitation facility, skilled nursing facility, hospice care, and patients leaving against medical advice. Home or facility that is home are included in the “home destination.” Patients that expired were not included in either of these groups and only in mortality rate for the study.
6. Statistical Analysis
We conducted statistical analyses using IBM SPSS Statistics ver. 27.0 (IBM Co., Armonk, NY, USA), GraphPad Prism v 9.0 (GraphPad Software Inc., La Jolla, CA, USA), MedCalc for Windows, version 19.4 (MedCalc Software, Ostend, Belgium), and R statistical software version 3.5.3 (R Foundation for Statistical Computing, Vienna, Austria). Continuous variables with skewed distribution are reported as median with interquartile range (IQR). The normality of the data was determined by employing the D’Agostino-Pearson, Shapiro-Wilk, and Kolmogorov-Smirnov tests. We performed univariate and multivariate analyses (employing logistic regression) for mFI-5, RAI-A, and RAI-Rev for the following outcomes: 30-day mortality, major complications, Clavien-Dindo PS classification IV complications, 30-day unplanned reoperation, 30-day unplanned readmission, ELOS, and discharge to a nonhome destination. The effect sizes for dichotomous outcomes were summarized by odds ratio (OR) and associated 95% confidence intervals (CIs). We performed the receiver operating characteristic (ROC) curve analysis including the area under the curve (AUC)/C-statistics calculations to discern the predictive ability of mFI-5, RAI-A, and RAI-Rev for outcomes after spine surgery. We used the DeLong test to compute the significance of C-statistic comparison between mFI-5, RAI-A, and RAI-Rev [32]. For all purposes, p < 0.05 was considered statistically significant.
RESULTS
1. Study Population Characteristics
We identified 6,571 patients that underwent surgery for TSI and met our inclusion criteria. The median age of our cohort was 64 years (IQR, 52–75 years), 57.3% patients were male, and the median BMI was 27.7 kg/m2 (IQR, 24.2–32.2 kg/m2). Majority of the cases were nonelective (54.1%). Within the cohort the median LOS was 6 days (IQR, 3–10 days), and the median operation time was 160 minutes (IQR, 110–230 minutes). One point seven percent of the patient cohort expired, 5.5% were returned to the operating room, and 8% were readmitted to the hospital, all within 30 days from their respective TSI surgery. The distribution for frailty tiers based on mFI-5 was as follows: not frail 40.7%, prefrail 36.3%, frail 19.3%, and severely frail 4.2%. The distribution for frailty tiers based on RAI-A was as follows: not frail 69.6%, prefrail 25.1%, frail 4.5%, and severely frail 0.9%. The distribution for frailty tiers based on RAI-Rev was as follows: not frail 10.5%, prefrail 32.4%, frail 44.4%, and severely frail 13.1%. The most common comorbidities within our cohort were hypertension (52.4%), current smoking status (21.7%), and diabetes mellitus (20.2%). Thirteen point four percent of the patient cohort experienced at least one or more major postoperative complications and the most common were postoperative pneumonia (4.8%), prolonged intubation (3.7%), unplanned reintubation (2.4%), and sepsis (1.8%). Clavien-Dindo PS classification grade IV complications occurred in 8.1% of the patient cohort and 44.2% of the cohort was transferred to a nonhome destination following surgery. All patient demographics and clinical characteristics are summarized in Table 1.
Baseline demographic and clinical characteristics, 30-day mortality, readmission, reoperation, major and minor complications, and discharge disposition of patients with spine trauma from NSQIP database 2015–2019 (n=6,571)
2. Univariate Analysis of mFI-5, RAI-A, and RAI-Rev on Surgical Outcomes
While univariate analysis of frailty tiers by all 3 scoring systems showed that increasing frailty tiers were associated with poor postoperative outcomes, patients identified as frail and severely frail using RAI-A and RAI-Rev had the highest odds of worse outcomes (Table 2). In the RAI-A and RAI-Rev univariate analysis, each frailty index was the most predictive of 30-day mortality and discharge to a nonhome destination. For 30-day postoperative mortality, the severely frail group for RAI-A demonstrated OR of 20.78 (95% CI, 8.73–49.49; p< 0.001) and the severely frail group for RAI-Rev showed OR of 48.12 (95% CI, 6.64–348.52; p < 0.001). For discharge to a nonhome destination the severely frail group for RAI-A had OR of 9.27 (95% CI, 4.68–18.37; p< 0.001) and RAI-Rev had OR of 18.08 (95% CI, 13.70–23.84; p< 0.001). The mFI-5-based severely frail group demonstrated OR of 4.79 (95% CI, 2.50–9.17; p< 0.001) for 30-day mortality and 6.83 (95% CI, 5.18–9.02; p< 0.001) for nonhome discharge.
Univariate analysis of mFI-5, RAI-A, and RAI-Rev for 30-day postoperative outcomes: mortality, major complication, Clavien-Dindo physical status classification grade IV, reoperation, readmission, ELOS, and nonroutine discharge in TSI data from NSQIP 2015–2019
3. Multivariate Analysis of mFI-5, RAI-A, and RAI-Rev on Surgical Outcomes
Multivariate regression analysis of frailty scores for patients undergoing surgery for spinal trauma (adjusting for BMI, emergent surgery status, operative time, race, and ethnicity) demonstrated results similar to univariate analysis with all 3 frailty scores showing that increasing frailty tiers result in poor postoperative outcomes, and patients identified as frail and severely frail using RAI-A and RAI-Rev had the highest odds of poor outcomes (Table 3). The likelihood of an adverse event was markedly high in severely frail cohorts of both RAI-A and RAI-Rev scoring systems, however OR of mortality, major complication, Clavien-Dindo PS classification grade IV complication, readmission, ELOS, and nonroutine discharge show RAI-Rev’s superiority in its ability to predict adverse outcomes. RAI-Rev frail populations have an OR of mortality of 10.35 OR (95% CI, 1.27–78.03; p= 0.02) and severely frail had OR 28.77 (95% CI, 3.69–224.54; p= 0.001), whereas RAI-A frail had OR 2.22 (95% CI, 1.02–4.84; p=0.04) and severely frail had OR 4.64 (95% CI, 1.59–13.59; p = 0.005) for mortality (Table 3). Surprisingly, mFI-5 frailty tiers demonstrated decreased risk of mortality across all frailty types which might indicate conservative management of comorbid individuals or the inability of mFI-5 to truly capture frailty phenotype.
4. ROC Curve Analysis and C-Statistics
The ROC analysis was conducted to assess the comparative predictive value of the mFI-5, RAI-A, and RAI-Rev for postoperative morbidity and mortality (Fig. 1). In the analysis for prediction of 30-day mortality, C-statistics indicated significantly better performance of RAI-Rev compared to mFI-5 (C-statistic = 0.819, 95% CI, 0.810–0.829 for RAI-Rev vs. C-statistic = 0.622, 95% CI, 0.610–0.634 for mFI-5, p< 0.0001, DeLong test). Similarly, for mortality, RAI-A performed better compared to mFI-5 (C-statistic= 0.768, 95% CI, 0.758–0.779 for RAI-A vs. C-statistic = 0.622, 95% CI, 0.610–0.634 for mFI-5, p < 0.001, DeLong test). Interestingly, RAI-Rev performed better than RAI-A in predicting postoperative mortality (C-statistic= 0.819, 95% CI, 0.810–0.829 for RAI-Rev vs. C-statistic=0.768, 95% CI, 0.758–0.779 for RAI-A, p = 0.0043, DeLong test). For most of other outcome variables, both RAI-Rev and RAI-A outperformed mFI-5, and for many outcomes, RAI-Rev showed better discriminative performance compared to RAI-A, including ELOS (p= 0.0042), readmission (p< 0.0001), reoperation (p= 0.0175), and nonhome discharge (p< 0.0001).
Receiver operating characteristic (ROC)/area under the curve (A-G) and ROC/C-statistics analysis (H) for the relative predictive abilities of the mFI-5 and RAI on mortality (A), major complication (B), Clavien-Dindo physical status classification IV complication (C), ELOS (D), nonhome discharge (E), readmission (F), and reoperation (G) in patients who underwent surgical intervention for traumatic spinal injury from NSQIP database 2015–2019. mFI-5, 5-factor modified frailty index; RAI-A, Risk Analysis Index-administrative; RAI-Rev, Risk Analysis Index recalibrated version; ELOS, extended length of stay; NSQIP, National Surgical Quality Improvement Program.
DISCUSSION
The goal of this study was to do a comparative analysis of discriminative ability of mFI-5, RAI-A, and RAI-Rev on postoperative outcomes of TSI based on large scale multicenter data from NSQIP. To the best of our knowledge, this is the first study evaluating RAI in spine trauma patients. Based on AUC/C-statistics analyses, both RAI-Rev and RAI-A outperformed mFI-5 in predicting worse postoperative outcomes, and among the 2 versions, the recalibrated RAI-Rev performed better than RAI-A for majority of outcomes. While RAI-Rev was a statistically significant predictor of worse postoperative outcomes, it performed the best for mortality, which is expected based on the fact that RAI-based frailty scales were originally developed using variables in the instrument that correlate the best with mortality [20,33]. The present study data provide evidence for the usage of RAI-based frailty scales in preoperative risk stratification of this patient population.
Frailty in spine trauma outcomes is a new topic, with only 2 previously published studies on the topic, utilizing mFI scores [15,16]. RAI-A, a recently developed frailty index, and its recalibrated version, RAI-Rev, have not previously been evaluated in TSI patients, however RAI has recently been reported to possess superior predictive ability in spine procedures and brain tumor resection patients [22,23]. RAI-based frailty scores comprise a 14-item scoring system as compared to mFI-5 which is based on 5 items, and additionally the 14 items in RAI are more relevant to functional status of the patient as compared to mFI scores, and are thus both conceptually and mathematically superior to mFI [20,33]. The present study data validates this in spine trauma patients. Previous studies have also demonstrated that how mFI-5 acts less as a metric of frailty and more as a comorbidity score [20,34-37]. This is corroborated by the data from the present study where in multivariate analysis for mortality, mFI-5 yielded unexpectedly low OR with decreased risk across all frailty tiers.
As individuals age, frailty becomes more prevalent, and with an increase in frailty comes an increased risk for falls and traumatic injury to the spine and spinal cord [38,39]. Not only does increased frailty predispose patients to the risk of TSI, but also individuals injured at older ages have an increased risk for mortality and morbidity [40]. The management and clinical decision making involving these patients provides a significant challenge to surgeons when considering surgical intervention. Because of this, it is important to have robust frailty tools for preoperative risk stratification of these patients. Both RAI-Rev and RAI-A come out as robust predictors of postoperative outcomes of TSI, and the present study advocates for their usage in clinical practice for prognostication of these patients, and to counsel them and their families regarding the expected outcomes.
The primary limitations of this study are those inherent to performing analysis using a national multicenter large database, understanding the limits of the recorded variables and ability to interpret results of their analysis. NSQIP variable limitations required modification of the RAI-A and RAI-Rev scores to exclude preoperative cognitive status. Additionally, weight loss and poor appetite are both captured in NSQIP under the same weight loss variable, “WTLOSS,” and therefore they cannot be differentiated within our adapted scoring. Despite these limitations, the NSQIP variables-based modifications of RAI scoring were in line with previous studies which demonstrated discriminative ability of this frailty measure [20,24,25]. In addition to NSQIP’s limitations impacting our RAI scoring, another limitation, particularly with reference to studying TSI, NSQIP also fails to provide an injury severity variable. Without an injury severity marker, we feel this study should be validated using a database such as the Trauma Quality Improvement Program which includes such variables. Lastly, as this study was only able to evaluate 30-day outcomes recorded in NSQIP, further analysis in long-term prospective studies is needed to further evaluate RAI’s ability to predict 90- and 180-day or further long-term outcomes.
CONCLUSION
Both RAI-A and RAI-Rev outperform mFI-5 in predicting postoperative worse outcomes following surgical intervention for TSI. Increasing RAI-Rev score was more discriminative than its former iteration, RAI-A, and mFI-5, in predicting likelihood of mortality, ELOS, readmission, reoperation, and nonhome discharge. Our analysis demonstrates that RAI-Rev may provide better preoperative risk assessment than these prior indices and should be included in preoperative risk stratification of TSI patients.
SUPPLEMENTARY MATERIALS
Supplementary Tables 1-7 can be found via https://doi.org/10.14245/ns.2244326.163.
Current Procedural Terminology (CPT) codes used to identify patients undergoing spine surgery from National Surgical Quality Improvement Program database 2015–2019
Most abundant International Classification of Diseases (ICD)-9 and ICD-10 codes among the spine trauma cohort, n=6,571, from National Surgical Quality Improvement Program 2015–2019
RAI-A and RAI-Rev scoring adapted from Hall and colleagues, 2017 and Arya and colleagues, 2020 respectively
Age and cancer diagnosis table scoring adapted from Hall and colleagues, 2017 for RAI-A scoring
Age and cancer diagnosis table of scoring, adapted from Arya and colleagues, 2020, for RAI-Rev
RAI-A and RAI-Rev scores stratified into frailty group status
NSQIP clinical variables matched to mFI-5
Notes
Conflict of Interest
The authors have nothing to disclose.
Funding/Support
This study received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Author Contribution
Conceptualization: MC, RT, SFK, AJD, MHS, RGM, CAB; Data curation: MC, RT, SFK, AJD, RGM, CAB; Formal analysis: MC, RT, SFK, AJD, RGM, CAB; Methodology: MC, RT, SFK, AJD, MHS, RGM, CAB; Project administration: M Schmidt, RGM, CAB; Visualization: M MC, RT, SFK, AJD, MHS, C Bowers; Writing - original draft: M MC, RT, SFK, AJD, MHS, C Bowers; Writing - review & editing: MC, RT, SFK, AJD, MHS, RGM, CAB.