From the Archives: Understanding the CIBMTR Outcomes Reports
The publication of the 2022 CIBMTR Transplant Center-Specific Survival Report is expected in December 2022. To help Clinical Program Directors prepare to evaluate their programs’ results, this volume’s From the Archives feature is, “Understanding the CIBMTR Outcomes Report,” published in October 2016.
Many of the concerns outlined in this article are still presented to FACT today, and we added more information and updated references to address recent questions. Our hope is that increased understanding of the CIBMTR report will transform its reputation as an anxiety-inducing evaluation into a reputation for what it is meant to be: one of many resources available to transplantation programs to understand and improve one-year survival.
There are many great Just the FACTs newsletter articles related to clinical outcomes, and it was difficult to choose! To see more helpful articles about clinical outcomes, click here.
Happy reading!
Understanding the CIBMTR Outcomes Reports
Allogeneic transplant programs in the U.S. are required to submit corrective action plans to FACT when they do not meet the expected range of one-year survival in the annual CIBMTR Transplant Center-Specific Survival Report. To adequately assess lower-than-expected one-year survival, it is necessary to fully understand how the report is generated.
CIBMTR reviews its methodology for the Center-Specific Survival Report every other year at the Center Outcomes Forum. This forum includes a variety of stakeholders, including transplant physicians, and is intended to also increase transparency and understanding. The agendas and summaries of each meeting can be found on the CIBMTR website. CIBMTR also explains the report methodology in the document titled, Methodology Employed for Annual Report on Hematopoietic Cell Transplant Center-Specific Survival Rates, which was last updated in December 2021. The document provides technical information on statistics, but offers the following summary:
“A fixed effects censored data logistic regression model is fitted to survival data for first unrelated and related donor hematopoietic cell transplants at U.S. centers. The model is adjusted for recipient age, recipient race, Karnofsky/Lansky score, Sorror HCT-CI, adult BMI group, pediatric BMI group, recipient CMV status, history of mechanical ventilation, history of invasive fungal infection, prior autologous transplant, disease/stage, AML ELN risk group, AML transformed from MDS or MPN, AML therapy related, number of induction cycles for AML in CR1, interval from diagnosis to transplant in ALL and AML in CR2 and CR3+/relapse, ALL cytogenetic risk group, Philadelphia positive-status in ALL, number of induction cycles for ALL in CR1, MDS IPSS-R risk score at HCT, MDS predisposing condition, CLL and other chronic leukemia disease status, NHL subtype, sensitivity to chemotherapy in NHL and HL, MM ISS stage at diagnosis, plasma cell disorder disease status, year of transplant, donor type/graft type/HLA matching, BM or PBSC donor/recipient sex match, unrelated BM or PBSC donor age at transplant, and recipient median household income. The report on transplant center-specific survival rates helps to identify centers that may have under-performed or over-performed compared to the overall network of transplant centers during this specified time period.”
Each year, CIBMTR provides a comprehensive version of the Center-Specific Survival Report directly to programs prior to publishing the results on the Be the Match website. The report is usually provided to programs annually in mid-December. In each of these reports, CIBMTR outlines in detail the risk factors that are considered and ultimately included in the final multivariate model used to produce the report that year. Each report includes sections on methods, statistical analysis, and results that describe how the model is formulated. Not only does this provide context for individual programs’ results, but it also provides information that any program can use regarding factors that have had a statistically significant impact on one-year survival.
CIBMTR also explains the report, and its tools and resources to help Clinical Programs use their data, during a 2016 FACT webinar recording titled, Using CIBMTR Data to Determine and Evaluate Clinical Outcomes, presented by Stephen Spellman, MBS. This recording gives an overview of how CIBMTR data is used to determine outcomes and how Clinical Programs can use additional data to further evaluate outcomes and improve.
The following are common points Clinical Programs question about the CIBMTR report, and how they can be evaluated in the context of creating corrective action plans to address lower-than-expected one-year survival:
- High-risk patients: Some corrective action plans state the root cause of death to be transplants for high-risk patients. As outlined above, the CIBMTR report is risk-adjusted. High-risk patients should be accounted for within the report. The FACT committee expects corrective actions that specifically address the causes of death. Broad refusal to transplant patients with high risk is not the intent of FACT requirements. As transplants are often the last hope for patients, careful attention to trends in causes of death is particularly important for these patients to improve their outcomes. For example, some programs have determined myeloablative therapy was not necessary or beneficial for a group of frail patients; others adjusted their protocol for preparative regimens.
- Socioeconomic factors: Some programs are located in regions in which their patients have a low socioeconomic status. Social factors do have an impact on a patient’s survival, and FACT understands that these factors are difficult to include in a risk-adjusted methodology. In these cases, FACT would expect to see a corrective action plan that targets social issues identified to be a root cause of patient deaths. Programs have implemented methods to successfully address low resources and lack of caregivers, such as pre-discharge dispensing of prescription medications, bus passes, and expanded clinic hours.
- Small programs: It is difficult to identify trends among a small number of transplants, but FACT will look for a good-faith effort of the program to review data and determine if a trend can be found. One small program found that its patients had a high rate of CNS disease, and educated its network of referring physicians.
- Confidence interval: A common worry is that Clinical Programs will have one-year survival lower than the expected range, through no fault of their own, because of the 95% confidence interval. It is important to realize that each program has its own confidence interval. A defined number does not have to drop out of the curve. Therefore, it is possible for each program to meet expected one-year survival. Small programs typically have a wider expected survival range.
- Delay in reporting: Due to the inherent timeframe of “one-year” survival, the CIBMTR report is delayed by two years because the transplant has to have occurred a year prior, and an additional year is needed to analyze the data. Furthermore, the report uses three years’ worth of data. For example, data analysis may show that a Clinical Program had a particularly bad year in 2019 that resulted in lower-than-expected one-year survival in the 2021 report. That year will affect one-year survival in the next two reports for 2022 and 2023. However, it is still necessary to review the causes of death and their root causes for the timeframe of the report. Programs must submit current internal survival data as part of their corrective action plans. Upward trajectories of internal data are taken into account.
- Overall one-year survival: The CIBMTR report only provides overall one-year survival; however, drilling down into specific diseases will help Clinical Programs determine root causes and which corrective actions may help. This is the same for treatment-related mortality or disease relapse. This type of drill-down has helped programs identify root causes.
- Data errors: Some Clinical Programs have noted that errors in the data submitted to the CIBMTR were the true root of lower than expected one-year survival. Indeed, this can affect results of the algorithm. If data errors are a problem, FACT will want to see corrective actions related to accurate data management and evidence that these corrections have had an impact on the risk-adjusted survival.
