Dynamic Infinity Mapping Framework (DIMF): An Adaptive Oncology Dosing Reinforcement Learning Approach

Author- Shrishti Rastogi

Abstract

Dynamic Infinity Mapping Framework (DIMF) is a new computational paradigm to manage the stochastic evolution of subpopulations of cancer cells under therapeutic pressure. DIMF combines Markov Decision Processes (MDP), Dynamic Graph Neural Networks (GNNs), and Reinforcement Learning (RL) to offer a powerful framework for optimising the dosage of multiple drugs adaptively. This report outlines the mathematical modelling of state transitions, algorithmic implementation of the RL agent and an empirical calibration using quantitative interaction and toxicity data from the large clinical trials. Our framework shows better ability to cross the balance line between therapeutic effect and total toxicity than static modelling methods.

Introduction

New challenges for modern oncology are the emergence of acquired resistance and complexity of multi-drug interactions in combination therapies. Tumor subpopulation dynamic mutational drift is not considered by static dosing models [149]. Based on this idea, “Dynamic infinity” states that the evolution of cancer is the continuous movement through a fitness landscape; treatment is an external perturbation of the system [56]. The Dynamic Infinity Mapping Framework (DIMF) has been created to map these trajectories and optimize interventions on the fly. DIMF hopes to harness the synergy of combinations like Nivolumab + Ipilimumab [55], Dabrafenib + Trametinib [130] and Osimertinib + Savolitinib [72] to achieve the highest possible PFS while staying in clinical safety margins [145]. In this section, Mathematical Formalization of State Transitions is performed.Mathematical Formalization of State Transitions is carried out in this section. DIMF is a stochastic model of the tumour microenvironment, where the tumour mutational state changes both due to intrinsic drift and extrinsic therapeutic influence. For Mutation Drift, we have a Markov Decision Process (MDP). The mutation trajectory is represented as an MDP tuple \( (\mathcal{S}, \mathcal{A}, \mathcal{P}, \mathcal{R}, \gamma)\):

State Space (\(\mathcal{S}\)):

The state \(s_t\) is described by a feature matrix \(\mathbf{X}_t \in \mathbb{R}^{N \times D}\), where \(N\) represents the number of cell subpopulations and \(D\) is the number of genomic features. The latent state is obtained by the GNN: \[ s_t = f_{GNN}(\mathcal{G}_t, \mathbf{X}_t) \]

Action Space \(\mathcal{A}\):

The action \(a_t\) is a vector of dosage amounts of \(k\) drugs: \[ a_t = [d_1, d_2, \dots, d_k], \quad d_i \in [0, 1] \]

Transition Probability \(\mathcal{P}\):

The state transition includes a calibrated drift function and the cross-resistance probabilities: \[ \frac{d\mathbf{x}_i}{dt} = \left( \mu_i - \sum_{k} \phi_{k,i}(C_k) \cdot \mathbf{I}_{DD}(C_k, \dots) \right) \mathbf{x}_i + \sum_{j} P(R_j | R_i) \cdot \mathcal{J}_{ji} \] where \(\mu_i\) is the growth rate, \(\mathbf{I}_{DD}\) is the interaction index, and \(P(R_j | R_i)\) is the probability of transition to a secondary resistance state (e.g., MET amplification after Osimertinib [65]).

Dynamic GNN Representation

The architecture of the tumour is represented as a dynamic graph \(\mathcal{G}_t = (\mathcal{V}_t, \mathcal{E}_t)\). The hidden representation \(\mathbf{h}_i^{(l+1)}\) at layer \(l+1\) is: \[ \mathbf{h}_i^{(l+1)} = \sigma \left( \mathbf{W}_c^{(l)} \mathbf{h}_i^{(l)} + \sum_{j \in \mathcal{N}(i)} \frac{1}{\sqrt{\text{deg}(i)\text{deg}(j)}} \mathbf{W}_n^{(l)} \mathbf{h}_j^{(l)} \right) \] A Recurrent GNN (Gated Graph Neural Network) is used for temporal aggregation: \[ \mathbf{H}_t = \text{GRU}(\mathbf{H}_{t-1}, \text{GNN}(\mathcal{G}_t, \mathbf{X}_t)) \] In this section, the concept of a RL-based adaptive dosing algorithm is introduced. The DIMF agent is based on a Deep Reinforcement Learning (DRL) architecture to maximize reward function that considers clinical effectiveness and toxicity.

Reward Function Construction

Reward \(R(s_t, a_t)\) is defined as: \[ R(s_t, a_t) = -\text{Burden}(s_t) - \sum_{i=1}^k \beta_i d_{i,t} - \eta \cdot \mathbb{I}(\sum d_{i,t} > \tau)\] \(\beta_i\) are toxicity coefficients and \(\eta\) is a penalty for synergistic toxicity when it surpasses threshold \(\tau\) [13]. An algorithm to optimize the dosage according to adaptation is given. In Algorithm 1, the main logic for dose adjustment is explained. Initialize the parameters of the GNN \(\theta_G\) and of the Actor \(\theta_\pi\) and Critic \(\theta_Q\). For each episode: Observe the initial genomic profile \(\mathbf{X}_0\) and plot the graph \(\mathcal{G}_0\). For every time step \(t\) : Calculate state embedding \(s_t = \text{GRU}(\mathbf{H}_{t-1}, \text{GNN}(\mathcal{G}_t, \mathbf{X}_t))\). Select action \(a_t = \pi(s_t | \theta_\pi) + \mathcal{N}(0, \sigma)\). Observe \(a_t\) and \(R_t\) and \(s_{t+1}\) using the transition model. Add transition \((s_t, a_t, R_t, s_{t+1})\) to Replay Buffer \(\mathcal{D}\). Select a mini-batch of data samples from \(\mathcal{D}\), and update \(\theta_Q\) so as to minimize Bellman error. Update \(\theta_\pi\) with Policy Gradient: \(\nabla_{\theta_\pi} J \approx \mathbb{E}[\nabla_a Q(s,a) \nabla_{\theta_\pi} \pi]\).

Empirical Parameter Calibration

The DIMF parameters are set by taking everything from clinical literature and empirical databases.
The metrics of interaction and synergy are presented.Interaction and synergy metrics are given. Modeling of synergy values is done with the Combination Index (CI) or Bliss Independence. The synergy values are summarized in Table 1.

Calibrated Synergy and Interaction Metrics

CI: Regimen	Interaction Metric	Synergy Value (CI)	Clinical Context
Dabrafenib + Trametinib	Combination Index (CI)	Strong synergy in BRAF V600E melanoma [3]	--
Osimertinib + Savolitinib	Loewe Excess	0.20 – 0.35	Rescues MET-driven resistance [72]
Nivolumab + Ipilimumab	Bliss Independence	0.15 – 0.25	Potentiates T-cell activation [13]

Toxicity and dose-limiting factors. Toxicity matrices \(\mathbf{T}\) represent the prevalence of acute toxicity. The RL agent is used to change the penalty weight \(\eta\) based these frequencies.

Toxicity Parameters for RL Calibration

Situation	Organ Toxicities (%)	Key Toxicities	Grade 3/4 Freq.	RL Penalty (η)
Nivo + Ipi	55.0% [2, 11]	Colitis (8%), Hepatitis (6%)	High (1.5)	--
Dab + Tram	26.0% [44]	Pyrexia (5%), Hypertension (4%)	Medium (0.6)	--
Osimertinib	12.0% [45]	Rash (1%), Diarrhea (2%)	Low (0.2)	--

Cross-Resistance Probabilities

The framework explicitly considers the transition probabilities to resistance (e.g., EGFR C797S or MET amplification) under the pressure of Osimertinib [113, 141].

The mutation transition probabilities P(R_B | R_A)

Initial Treatment (RA)	Secondary Mutation (RB)	Probability P(RB | RA)	Clinical Evidence
Osimertinib	EGFR C797S	0.07 – 0.26	Acquired resistance [82]
Osimertinib	MET Amplification	0.07 – 0.25	Bypass signaling [113]
Dabrafenib	NRAS Mutation	0.10 – 0.20	MAPK reactivation [117]

Experimental Benchmark Design

We want to set benchmark to evaluate DIMF: Conventional static dosing protocol.

Comparative Models

Static Fixed-Dose (SFD): Fixed maximum tolerated dose (MTD) as per clinical guidelines [13].

Heuristic Adaptive (HA): Dose reduction as per observed Grade 2+ toxicities (standard clinical practice) [102].

Proposed: Continuous RL-based adjustment based on Molecular GNN embeddings (DIMF).

Benchmarking Criteria

The performance is evaluated for three main-streams:
1. Progression-Free Survival (PFS): Modeled with the integrated transition function.
2. Toxicity Adjusted Return (TAR): Sum of the reward over the horizon of the simulation.
3. Resistance Latency: Time until the subpopulation $\mathbf{x}_{res} > \theta_{critical}$.

Limitations and Gaps

The present embodiment of DIMF is based on retrospective clinical data for parameterisation. One of the practical challenges is the high fidelity real-time genomic monitoring (daily ctDNA) [73]. Moreover, the model to represent the subpopulation is based on GNN, and it is subject to the assumption of high accuracy in the identification process, which is not guaranteed in clinical settings due to sampling noise [110].

Conclusion

The Dynamic Infinity Mapping Framework is a mathematically sound basis for adaptive oncology. The complex empirical parameters used in DIMF, such as irAE rate at the Grade 3/4 level for Nivo+Ipi [2] and the synergistic CI values of targeted doublets [3] are successfully modeled within a single optimization engine. The next step in this research will be to combine longitudinal single-cell sequencing data to further improve predictions of mutation drift.

[1] Shen, J., Wang, Z. Frontiers in Oncology 2022, “Recent advances in the progress of immune checkpoint inhibitors in the treatment of advanced gastric cancer: A review”.
[2] Achkar, T., Tarhini, A. A. The application of immunotherapy in the treatment of melanoma, in the Journal of Hematology & Oncology, 2017.
[3] Uitdehaag, J. C. M. et al. "Combinations of existing drugs selectively target the growth of cells driven by CTNNB1, KRAS or MYC,", \textit{PLOS ONE}, 2015.
[11] Zhou, S., Khanal, S., Zhang, H. Risk of immune-related adverse events associated with ipilimumab-plus-nivolumab and nivolumab therapy in cancer patients, \textit{Therapeutics and Clinical Risk Management}, 2019.
Callahan, M. K. et al. "CTLA-4 and PD-1 Pathway Blockade: Combinations in the Clinic," Frontiers in Oncology, 2015.
[44] Abraham, J. Trametinib, dabrafenib combination for unresectable or metastatic melanoma for treatment with BRAF V600E or V600K mutations, 2014.
[45] Sequist, L. V. et al. "A Phase 1b Study of Osimertinib Plus Savolitinib in Patients with EGFR Mutation-Positive, MET-Amplified, Non-Small Cell Lung Cancer after Progression on EGFR-Tyrosine Kinase Inhibitors," 2019.
Wolchok et al, Overall Survival with Combined Nivolumab and Ipilimumab in Advanced Melanoma, \textit{New England Journal of Medicine}, 2017.
Griffin, M., et al., “BRAF inhibitors: resistance and the promise of combination treatments for melanoma”, Therapeutic Advances in Medical Oncology, 2017.
Osimertinib + Savolitinib to Overcome Acquired MET-Mediated Resistance in Epidermal Growth Factor Receptor–Mutated, MET-Amplified Non–Small Cell Lung Cancer: TATTON, 2022.
72) Sequist, L. V. et al. Osimertinib in combination with savolitinib in patients with EGFR mutation-positive, MET-amplified, non-small-cell lung cancer after progression on EGFR tyrosine kinase inhibitors: Interim results from a multicentre, open-label, phase 1b study, The Lancet Oncology, 2020.
Monitoring of plasma concentrations of dabrafenib and trametinib in advanced BRAFV600 melanoma patients, 2022.
Lim, S. M. et al. "Brief Report: Heterogeneity of Acquired Resistance Mechanisms to Osimertinib and Savolitinib," 2021.
[102] Teimouri, A., et al. (2022). "Real-World Adherence to Toxicity Management Guidelines for Immune-Related Adverse Events.
Tang, S. Q. et al., "The Pattern of Time to Onset and Resolution of Immune-Related Adverse Events Caused by Immune Checkpoint Inhibitors in Cancer: A Pooled Analysis of 23 Clinical Trials and 8,436 Patients (2021)".
[113] Piper-Vallillo, A. J. et al. "Acquired resistance to osimertinib plus savolitinib is mediated by MET-D1228 and MET-Y1230 mutations in EGFR-mutated MET-amplified lung cancer," 2020.
Savoia, P. et al. (2020) Clinical implications of acquired BRAF inhibitors resistance in melanoma.
Long et al., "Final Results for Adjuvant Dabrafenib plus Trametinib in Stage III Melanoma," New England Journal of Medicine, 2024.
Nicos, M. et al., "Possibilities of overcoming resistance to Osimertinib in NSCLC patients with mutations in the EGFR gene.
McGregor, B. A. et al., "Grade 3/4 Adverse Event Costs of Immuno-oncology Combination Therapies for Previously Untreated Advanced Renal Cell Carcinoma", 2024.
Loria, R., et al. 2022. "Cross-resistance among sequential cancer therapeutics: an emerging issue.