Machine learning reveals why cancer clinical trials fall short in real-world patients

TrialTranslator uncovers survival gaps for high-risk patients and provides a path to better cancer research.

study: Assessing the generalizability of oncology test results to real patients using machine learning-based test emulation. Image credit: Komsan Loonprom/Shutterstock.com

Many cancer trial results do not generalize well to real-world patients. The research team investigated this question using TrialTranslator, a machine learning framework that systematically tests the generalizability of cancer RCT results. Survey results announced at natural medicine.

Generalizability of RCT results is low

Randomized controlled trials (RCTs) are considered the gold standard for evaluating cancer treatments. However, their findings often fail to translate to real-world situations, and patients, physicians, and drug regulators are concerned about the limited generalizability of these results.

In oncology, actual survival and treatment efficacy are often significantly lower than those reported in RCTs, sometimes reducing median overall survival (mOS) by as much as 6 months. New anticancer drugs, such as checkpoint inhibitors, also perform poorly when applied to diverse patient populations observed outside of clinical trials.

Reason for the difference

The main reason for this gap is the restrictive eligibility criteria often used in RCTs that create study populations that do not reflect real-world patient diversity. Trial participants are often young, healthy, and less likely to suffer complications.

Informal biases, such as preferential selection based on race or socio-economic status, can also affect hiring. These limitations cannot account for real-world patient heterogeneity, where outcomes can vary widely even with the same treatment protocol.

In the current study, we sought to address this issue by improving the prediction of actual outcomes of cancer treatments evaluated in phase 3 RCTs. To do this, the researchers developed TrialTranslator, a machine learning (ML) framework designed to systematically assess the generalizability of RCT results.

By leveraging electronic health records (EHRs) and advanced ML algorithms, this framework identifies patterns and phenotypes that can influence treatment outcomes and provides more nuanced insight into survival effects across diverse patient groups. enables accurate evaluation.

About research

Using Flatiron Health's comprehensive national EHR database, researchers applied TrialTranslator to evaluate 11 landmark RCTs. These trials looked at four of the most common advanced solid tumors.metastatic breast cancer (mBC), metastatic prostate cancer (mPC), metastatic colorectal cancer (mCRC), and advanced non-small cell lung cancer (aNSCLC).

Each RCT was emulated by identifying real-world patients with matching cancer types. biomarker Profiles, treatment plans, etc.

Patients were stratified into three prognostic phenotypes (low risk, intermediate risk, and high risk) based on the mortality risk score obtained from the ML model. This framework then assessed survival outcomes, including mOS and restricted mean survival time (RMST), and compared treatment effects across these phenotypes with those reported in the original RCTs.

Key findings: Risk-based outcome gap

This study revealed significant differences between RCT results and real-world results.

• Low- and intermediate-risk patients: These phenotypes showed survival and treatment effects that closely matched RCT results. For example, low-risk patients often experienced survival benefits similar to those reported in clinical trials, but with only modest reductions in mOS (approximately 2 months).
• High-risk patients: In contrast, high-risk phenotypes had significantly worse outcomes. Survival benefits were significantly reduced (62% lower than RCT estimates) and in many cases fell outside the 95% confidence intervals reported in the original trials. Seven of the 11 emulated trials did not show clinically meaningful survival improvements (>3 months) in high-risk patients.

Overall, emulated trials consistently estimated survival outcomes that were on average 35% lower than those reported in RCTs. This discrepancy highlights the challenges in extrapolating test results to more heterogeneous real-world populations.

Robust validation of results

The robustness of these findings was confirmed through extensive validation. Subgroup analysis, semi-synthetic data simulation, and alternative eligibility criteria demonstrated consistent results and strengthened the reliability of TrialTranslator. Sensitivity analysis also showed that more stringent eligibility criteria had little effect on the observed differences, suggesting that patient prognosis plays a more important role than inclusion criteria in determining treatment outcome. has been.

Impact on oncology

These findings highlight the need for a paradigm shift in the design and interpretation of clinical trials. Current RCTs often overlook the heterogeneity of real-world patient prognosis, which contributes to limitations in generalizability. High-risk patients, in particular, are underserved in existing trials because their outcomes deviate the most from RCT results.

Tools like TrialTranslator offer a promising solution. Integrating data from EHRs with ML-based phenotyping can provide personalized predictions of treatment efficacy at the individual patient level. This enables more informed clinical decision-making and allows patients and clinicians to set realistic expectations about treatment outcomes.

Additionally, these tools have the potential to revolutionize trial design by prioritizing patient prognosis over traditional eligibility criteria. By stratifying patients based on risk phenotype, future trials may be able to better represent the full spectrum of cancer patients and provide more accurate treatment estimates. efficacy.

conclusion

“This study highlights the important role that prognostic heterogeneity plays in the limited generalizability of RCT results,” the authors conclude. While low- and intermediate-risk patients may receive expected benefits from cancer treatment, high-risk patients often experience reduced survival.

ML-based frameworks like TrialTranslator can help close this gap, potentially enabling more comprehensive trials and better real-world results. Using such tools, oncology can move closer to a truly individualized treatment approach that takes into account the diverse needs of real-world patients.