Back to Journals » Clinical Pharmacology: Advances and Applications » Volume 14

Neural Net Modeling of Checkpoint Inhibitor Related Myocarditis and Steroid Response

Authors Stefanovic F, Gomez-Caminero A, Jacobs DM, Subramanian P , Puzanov I, Chilbert MR , Feuerstein SG, Yatsynovich Y, Switzer B, Schentag JJ

Received 13 April 2022

Accepted for publication 2 August 2022

Published 10 August 2022 Volume 2022:14 Pages 69—90


Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 3

Editor who approved publication: Professor Arthur Frankel

Filip Stefanovic,1,2 Andres Gomez-Caminero,3 David M Jacobs,2,4 Poornima Subramanian,2 Igor Puzanov,5,6 Maya R Chilbert,4 Steven G Feuerstein,4 Yan Yatsynovich,6,7 Benjamin Switzer,5 Jerome J Schentag2,4,5

1Department of Biomedical Engineering, University at Buffalo School of Engineering and Applied Sciences, Buffalo, NY, USA; 2CPL Associates LLC, Buffalo, NY, USA; 3Worldwide Health Economic and Outcomes Research, Bristol Myers Squibb, Princeton, NJ, USA; 4Department of Pharmacy Practice, University at Buffalo School of Pharmacy and Pharmaceutical Sciences, Buffalo, NY, USA; 5Department of Medicine, Roswell Park Comprehensive Cancer Center, Buffalo, NY, USA; 6Department of Medicine, University at Buffalo Jacobs School of Medicine, Buffalo, NY, USA; 7Kettering Medical Center, Kettering, OH, USA

Correspondence: Jerome J Schentag, CPL Associates LLC, 73 High St. Suite 310, Buffalo, NY, 14203, USA, Tel +1 716-867-0550, Fax +1 716-633-3331, Email [email protected]

Background: Serious but rare side effects associated with immunotherapy pose a difficult problem for regulators and practitioners. Immune checkpoint inhibitors (ICIs) have come into widespread use in oncology in recent years and are associated with rare cardiotoxicity, including potentially fatal myocarditis. To date, no comprehensive model of myocarditis progression and outcomes integrating time-series based laboratory and clinical signals has been constructed. In this paper, we describe a time-series neural net (NN) model of ICI-related myocarditis derived using supervised machine learning.
Methods: We extracted and modeled data from electronic medical records of ICI-treated patients who had an elevation in their troponin. All data collection was performed using an electronic case report form, with approximately 300 variables collected on as many occasions as available, yielding 6000 data elements per patient over their clinical course. Key variables were scored 0– 5 and sequential assessments were used to construct the model. The NN model was developed in MatLab and applied to analyze the time course and outcomes of treatments.
Results: We identified 23 patients who had troponin elevations related to their ICI therapy, 15 of whom had ICI-related myocarditis, while the remaining 8 patients on ICIs had other causes for troponin elevation, such as myocardial infarction. Our model showed that troponin was the most predictive biomarker of myocarditis, in line with prior studies. Our model also identified early and aggressive use of steroid treatment as a major determinant of survival for cases of grade 3 or 4 ICI-related myocarditis.
Conclusion: Our study shows that a supervised learning NN can be used to model rare events such as ICI-related myocarditis and thus provide clinical insight into drivers of progression and treatment outcomes. These findings direct attention to early detection biomarkers and clinical symptoms as the best means of implementing early and potentially life-saving steroid treatment.

Keywords: NN modeling, myocarditis, checkpoint inhibitor, steroid response, dose and timing


Immune checkpoint-inhibitor (ICI)-associated myocarditis is a rare but serious side effect of treatment with ICI, with a fatality rate of around 50%.1 Clinical trials failed to detect this low frequency adverse event and almost all detailed laboratory and clinical information comes from cases reported during clinical use. Over 150 cases of ICI-related myocarditis have been reported in the post-marking period for the PD-1 inhibitors. There are cases for all of the widely used ICIs, including pembrolizumab2–11 nivolumab,12–31 the CTLA-4 inhibitor ipilimumab, either alone or in combination with nivolumab,14,16,18,31–34 and the PD-L1 inhibitors atezolizumab35 and durvalumab.36,37 A multicenter observational cohort study estimated the prevalence of ICI-related myocarditis to be approximately 1.14%.1 Additionally, numerous cases also appear under related headings such as cardiotoxicity, cardiac toxicity, myositis, and immune-related cardiovascular events. A recent retrospective study estimated the absolute 1-year risk of all cardiotoxicities including arrhythmias, pericarditis, myocarditis and heart failure to be 9.7%. Collectively, these cases have prompted guidelines for diagnosis and management.38,39

Patients with the more severe manifestations of ICI-related myocarditis have varying degrees of heart block upon EKG, reduced ejection fraction, elevated plasma troponin, CPK, AST and LDH. Endomyocardial biopsy and autopsy studies reveal massive infiltration of CD8+ lymphocytes in the hearts of most of these severe and fatal events.40 Most patients also present with concomitant immune system related adverse events (irAEs) in addition to myocarditis, such as fatigue, rash, endocrine disorders, hepatitis,41 neurological disorders,42 colitis,43 and pneumonitis.44 In a previous study in 403 patients hospitalized between 2003 and 2013, where the goal was to establish risk factors for mortality from known or suspected viral myocarditis, the main predictors of mortality were patient age >50 years, creatinine clearance <60 mL/min, ventricular tachycardia, NYHA classification >3, and elevated troponin.45

Although correlations to ICI-related myocarditis and myocarditis generally have been identified, no report has attempted a clinical disease model integrating time-related changes in laboratory findings, clinical histories, and treatment responses with clinical outcomes. One barrier to the development of such a clinical algorithm is the paucity of cases available for analysis due to the low frequency of ICI-related myocarditis. A second challenge is the relatively little data collected on each case, and lack of time-related clinical signals in particular. Machine learning methods conventionally require large datasets to make correlations between variables and an outcome of interest, but thus far have not been applied to time and magnitude in individuals. In most cases, these artificial intelligence algorithms source case report forms or adverse reaction reports, and thus rely on sparse data collected at indistinct time points designated by events, such as initiation of a drug therapy or beginning of an adverse event.

An alternative machine learning approach applies Neural Network (NN) Models to the patient medical records of target cases that are enriched for the event in question, such as we previously used for understanding bleeding events from warfarin.46 Having access to the medical records of well-defined irAE cases from a survey paper recently published by our clinical group47 we then applied the NN modeling approach to better understand time and magnitude and treatment response in ICI-related myocarditis. This model leverages known clinical covariates in a structured scoring system in order to boost model performance, generalizability, and interpretability.48 Furthermore, this approach allows us to integrate patient-level information regarding the presentation of ICI-related myocarditis such as the co-occurrence of other irAEs which may be infrequent but are highly correlated with ICI-related myocarditis in an individual patient’s clinical context.

Previously, we examined 15 cases of ICI-related myocarditis retrospectively for determinants of severe disease and mortality using conventional statistical hazard modeling methods.47 In this study, we utilize the rich time-series clinical data available from patient electronic health records to construct a NN-based predictive model of determinants of ICI-related myocarditis development, severity, response to treatments and resolution.


Study Design and Modeling

We developed a NN model of a longitudinal cohort of patients treated with ICIs at Roswell Park Comprehensive Cancer center and developed elevated serum troponin concentrations of >0.06 ng/mL - upper limit of normal for the laboratory test.47 Data were collected from electronic health records and included outpatient and inpatient health-care claims, clinical monitoring, laboratory and pharmacy data. The laboratory data include all tests performed and results. Pharmacy data include all patient demographics and medication dosing.

Patient Sample

Twenty-three cases were identified based on elevated troponin levels as demonstrating likely ICI-cardiotoxicity as determined by clinical assessment using Naranjo Score. Eleven of these patients had severe myocarditis, 4 had mild myocarditis, and in the remaining 8, the elevated troponin was ascribed to other causes such as acute coronary syndromes or exacerbations of heart failure. All patients had troponins and ECGs available before and after ICI initiation, as well as after cardiotoxicity development. Ten patients in the severe myocarditis and 3 patients in the subclinical myocarditis cohorts had echocardiograms with estimated ejection fractions. Gadolinium enhanced cardiac MRI (CMR) was performed in 3 patients with severe myocarditis. Ischemic evaluation with coronary angiography was performed in 3 patients with severe myocarditis.

Neural Net Model Inputs, Outputs, Biomarkers

Myocarditis severity was used as the primary output parameter since the goal was to define the drivers of myocarditis onset and severity. Myocarditis was expressed as a score 0–5, using criteria adapted from CTCAE guidelines (Supplementary Table S1, see Online Supplement for detailed scoring criteria). Troponin levels were also modeled as an output. We also modeled composite parameters chosen to elucidate the dependent relationships, including tumor diameter as a measure of tumor burden, combination irAEs, combination lab signals, and combination myocarditis signs and symptoms metric. See Online Supplement for more information on inputs and modeling methods.

We created several combination signals using various normalized factors listed in Table 1. Each included factor was weighted on clinical importance and the individual weighted components were averaged (if present) into the combination signals. If specific parameters were not available for a patient, they were omitted so that the average would not be affected.

Table 1 Baseline Clinical, and Laboratory Characteristics of 23 Patients Prior to ICI Administration

The following combination signals and constituents were developed for entry into the model:

  • Combination Adverse Event Signal (“CombAdvEvt”): myocarditis, pruritis, myalgia, dyspnea, chest pain, bradycardia, edema, diplopia, ptosis, pancreatitis, thyroiditis, hepatitis, colitis, myocardial infarction, optic nerve disorders, myasthenia gravis, rash, encephalitis, increased creatinine or BUN, fatigue, diarrhea, pneumonitis, and polyneuropathy.
  • Combination Myocarditis Metric (“CombLoadMyoMetric”): respiratory rate, body temperature, QRS, troponin (not normalized), use of antiPD1, use of antiCTLA4, and WBC.
  • “Organ Metric”: total bilirubin, serum creatinine, oxygenation, platelets, hypotension, and SOFA Score.
  • Combination Lab Signal (“CombLabSignal”): troponin Score, BNP Score, AST Score, LDH Score, QRS Score, Lactate Score, CPK Score and CPK-MB Score.

Neural Net Modeling

Modeling for the effects of ICIs was done in MatLab by loading all patient data and their corresponding values as part of a structured time-based array. As the data were loaded, input data were verified using the unique patient identifiers and stored in a sequential manner based on their time stamps. Patients were assigned identification numbers so that duplicate entries do not occur. Minimum mean square error (MMSE) was calculated to rank order the model outputs. For detailed description of the neural net model structure, see the Online Supplement.


Sample Characteristics

We identified 23 cases of troponin elevation during ICI therapy as defined by troponin >0.06 ng/mL. Of these 23, 15 had ICI-related myocarditis as defined by the criteria, while the remaining 8 cases had other cardiotoxicities during ICI treatment. All available data was collected on each case. In the final model, there were 140,916 data elements collected in the 23 study patients. While this averages to 6126 data points per patient, the Electronic Health Record derived datasets were more concentrated in the severe cases who were hospitalized and survived longer. Table 1 shows little difference at baseline in clinical or demographic characteristics upon comparison of the 15 ICI-related myocarditis and 8 other cardiotoxicity cases. The patients were of similar age and underlying disease profile, received similar treatments, and in most cases had similar clinical monitoring. Patients who presented with ICI-related myocarditis had similar clinical laboratory biomarkers to patients who presented with other cardiotoxicities. CPK and AST trended higher in patients who had ICI-related myocarditis, although these associations were not statistically significant.

The baseline characteristics of the 15 cases of ICI associated myocarditis are presented in Table 2. There were 11 patients who reached a score of 3 or 4, values that are associated with hospital care and in the case of score of 4, ICU care. These patients demonstrated multi-organ involvement by multiple lab tests of organ function, and usually more than one other immune-related adverse event. In general, the myocarditis score was a marker of systemic involvement in the inflammatory process as evidenced by the elevation in LFTs. Specifically, LDH was markedly elevated at baseline in patients who died of ICI-related myocarditis in comparison to survivors and positive controls in the same population who experienced non-ICI-related troponin elevations (such as NSTEMI, heart failure exacerbations) (Table 2). As shown in Table 2, there were no other major differences between baseline pre-ICI therapy conditions and biomarkers at presentation in the 4 cases of fatal ICI-related myocarditis vs the 11 patients with myocarditis who survived.

Table 2 Baseline Clinical and Laboratory Characteristics Between Patients with Fatal and Non-Fatal ICI-Related Myocarditis, vs Other Cardiac Conditions on ICI Treatment

Time Course of Clinical Progression

Four of the 15 patients who developed ICI-related myocarditis had fulminant myocarditis resulting in death. The time course of clinical progression of the 11 non-fatal cases and 4 fatal cases of ICI-related myocarditis are shown in Figures 1 and 2, respectively.

Figure 1 Biomarker time course in 11 patients with ICI myocarditis and lived. Time course of selected biomarkers in 11 patients with elevated troponin caused by ICI associated myocarditis. Time zero is onset of myocarditis, shown by change in myocarditis severity score from zero. Myocarditis was assigned a clinical severity score from 0–5 with 1 corresponding to symptomatic disease, 2 to symptomatic disease + abnormal biomarkers, 3 to myocarditis-related hospitalization, 4 to ICU care, and 5 assigned at the time of death. All 11 patients survived the ICI myocarditis episode displayed in this figure, although a score of 4 was reached by patient 1, 4, 8 and 9. Notable is the close association between prednisone equivalent dose score and the decline of myocarditis score as well as both laboratory and clinical indices of myocarditis. Among the important biomarkers that track with use of steroids, the QRS rose before prednisone score and declined rapidly afterwards. Lymphocyte count declined with myocarditis severity score and rose after it resolved.

Figure 2 Biomarker time course in 4 patients with fatal ICI myocarditis. Time course of selected biomarkers in 4 patients with elevated troponin caused by ICI-related myocarditis. Time zero is onset of myocarditis, shown by change in myocarditis severity score from zero. Myocarditis was assigned a clinical severity score from 0–5 with 1 corresponding to symptomatic disease, 2 to symptomatic disease + abnormal biomarkers, 3 to myocarditis-related hospitalization, 4 to ICU care, and 5 assigned at the time of death. These 4 patients died primarily as a result of ICI myocarditis or immediate sequelae. Notable is the close association between steroid use and dose (represented in prednisone equivalents by “PrednisoneEq_Score”) and the decline of myocarditis score as well as both laboratory and clinical indices of myocarditis. However, in most cases prednisone dose was too low and started too late to be lifesaving. QRS rose before prednisone score but did not decline with inadequate prednisone dose. Lymphocyte count declined with myocarditis and did not recover with inadequate prednisone dose.

As shown in Figures 1 and 2, lymphocyte count was an important predictor of the onset severity and duration of ICI-related myocarditis. Greater decline in lymphocyte counts preceded severe cases of ICI-related myocarditis, and lower lymphocyte counts were correlated with the development of severe ICI-related myocarditis. After high dose steroids, the lymphocyte counts rose in most of these severe cases. Cardiac imaging exhibited inflammatory changes such as cardiac edema and increased wall thickness, as well as onset of heart failure in settings where there was either heart block or extensive myocardial damage from inflammation.

In Figure 1, the individual time plots of biomarkers are shown for 11 patients with ICI-related myocarditis who survived. In these data plots, time zero is onset of myocarditis, shown by change in myocarditis score. Notable is the close association between steroid use and the decline of myocarditis score as well normalization of both laboratory and clinical indices of myocarditis. Treatment with sufficient doses of steroids was associated with resolution of QRS widening and reduced risk of death. Early and adequate steroid use is lifesaving in these cases. Consistent with the effects of myocarditis on lymphocyte trafficking, the lymphocyte count declined with myocarditis and rose after it resolved from use of high dose corticosteroids.

Figure 2 shows the time course of NN model-selected clinical covariates in 4 patients with fatal outcome (myocarditis score 5) caused by ICI-related myocarditis. These 4 patients died primarily because of ICI-myocarditis or immediate sequelae. In these cases, there was insufficient decrease in myocarditis score following the use of immunosuppressive therapy, which often was initiated later in the time course of their clinical progression.

The time course of clinical progression of the 8 patients who experienced troponin elevations for other causes while on ICI is shown in Figure 3. Parameters chosen for display are highly correlated with ICI-related myocarditis; patients without ICI-related cardiotoxicities therefore show little change in these biomarkers over time. Although three of these cases did receive corticosteroids due to initial suspicion of myocarditis, there was seldom any evident change in the biomarkers of myocarditis, identifying these cases as not having an inflammatory cause in the heart. As shown in Figure 3, some of these cases were so similar upon clinical presentation, that they were treated with steroids. In each of these, the steroids were stopped after the actual cause was determined, and it was concluded that ICIs were not the cause of their cardiotoxicity. Further discussion of the precise clinical diagnostics used to define myocarditis cases is found in our clinical paper (47).

Figure 3 Biomarker time course in 8 ICI treated patients with other cardiotoxicity. Time course of selected biomarkers in 8 patients with elevated troponin that was determined not to be caused by ICI associated myocarditis. Two patients had a rise in troponin associated with worsening of CHF, 5 patients had coronary artery events primarily ischemic, and one patient had tumor hyper-progression with obstructing cardiopulmonary disease. Time zero here is the onset of troponin rise, shown by the change in troponin score. Although 3 of these cases did receive prednisone, there was seldom any evident change in the biomarkers of myocarditis, identifying these cases as not having an inflammatory cause in the heart.

Neural Net Modeling and MMSE Analyses

An MMSE analysis was performed for predictors of ICI-related myocarditis by comparing the clinical predictors of myocarditis score and troponin elevation between patients who developed ICI-myocarditis and those who developed troponin elevations of other cause (Table 3). The patients who developed troponin elevations of other causes were used as a positive control to analyze the determinants of ICI-related myocarditis vs other cardiac presentations in the same patient population. The top predictors of clinical myocarditis severity score in patients with confirmed ICI-related myocarditis were the combined myocarditis metric (“CombLoadMyoMetric”), the combined lab signal (“CombLabSignal”), CPK-MB, ALT, troponin and QRS interval. The top predictors of troponin in positive control patients who did not have myocarditis were myocarditis metric (“CombLoadMyoMetric”), the combined lab signal (“CombLabSignal”) and lactate. The modeling outputs presented some similar trends to the clinical time progression presented by the ICI-related myocarditis patients (Table 3). EKG changes were largely non-specific for ICI-related myocarditis, as they were also closely associated with the presentation of other cardiac conditions but were very predictive of myocarditis score in patients who did have ICI-related myocarditis (Table 3). Markers of organ failure and life-threatening conditions, such as shock, use of vasopressors, blood pressure and oxygenation were only found in the most severe subset, so while they were not as strongly associated with clinical outcomes when examining all 15 cases, they did strongly associate with clinical outcomes when examining the differences between the 11 non-fatal vs the 4 fatal cases (as shown in Table 4). The biomarker comparison between the positive controls and the 15 cases of ICI-related myocarditis reveal how similar non-ICI related cardiac presentations in ICI treated patients are to ICI-related myocarditis (Table 3). Once the diagnostics are performed, most patients differentiate from a lack of inflammatory involvement in the heart, different conduction defects, and the cardiotoxicity does not improve with steroid therapy, and may worsen if the cause is myocardial infarction.

Table 3 All 23 Patients with Elevated Troponin: Comparison of Troponin and Myocarditis Score in 15 Patients with ICI-Related Myocarditis vs 8 Patients with Other Cardiotoxicity Causes for Elevated Troponin

Table 4 Model MMSE Outputs of 15 ICI-Related Myocarditis Cases, Comparing 11 Non-Fatal and 4 Fatal Cases

Analysis of 4 Cases of Fatal ICI-Related Myocarditis

A separate MMSE analysis was performed for predictors of the output of ICI-related myocarditis fatality by investigating the association between clinical myocarditis score (and troponin) in patients who survived and in patients who died. All patients died of ICI-related myocarditis within 30 days of the onset of symptoms.

Model errors represent the strength of the relationship between the input and the output, ie, the lower the model error, the stronger the relationship between the predictor and the outcome. For example, changes in the time and magnitude of CPK-MB are associated with a 0.2% model error in patients with fatal myocarditis, indicating that only 0.2% of CPK-MB data points would be expected to not correspond to the biomarker profile of fatal ICI-related myocarditis. Our results showed that CPK-MB, troponin score, QRS interval and ALT score were the most closely correlated inputs with the outcome of a fatal myocarditis score of 5 (Table 4). These inputs were associated with much lower model errors for the outcome of myocarditis score when myocarditis was fatal compared to when it was not, indicating that they were both highly predictive and specific to the outcome of severe myocarditis. For example, CPK-MB was associated with a 0.2% error in the patients with fatal myocarditis whereas it was associated with a 35.7% error in patients with non-fatal myocarditis. Interestingly, ALT score was associated with a 0.7% error for patients with fatal myocarditis whereas it was associated with 31% error for non-fatal myocarditis. Troponin score was associated with a 1.4% error in fatal myocarditis whereas it was associated with a 26% error in non-fatal myocarditis. QRS interval was associated with a 2% error in fatal myocarditis whereas it was associated with a 34.4% error in non-fatal myocarditis. Physiologic organ failure markers such as hypotension, lactate, and presence of third-degree heart block were also associated with the development of a fatal myocarditis score.

Steroid Effect on ICI-Related Myocarditis Cases

Most patients received high dose steroids in the treatment of ICI-related myocarditis. In the clinical data, early and aggressive use of high dose corticosteroids was often the key finding in those with severe ICI-related myocarditis who survived. We examined the cases where optimal steroids resulted in a positive outcome, so we selected these 6 cases as enriched for steroid effect on myocarditis score. Figure 4 shows the myocarditis score and the prednisone biomarkers vs the time of myocarditis onset. Patients who responded received treatment on average 2-days earlier than patients who did not respond, and received higher doses earlier and longer. With rapid initiation of high dose steroids, responders typically experienced myocarditis resolving to grade 2 or less toxicity (correlating to symptomatic disease but not hospitalized), within 10–15 days of symptom onset. Patients who responded to steroids also often received prolonged courses of high dose steroids after achieving a myocarditis severity score of 3 or higher. Patients who did not respond often had steroid doses reduced while their myocarditis severity scores remained high, indicating inadequate treatment.

Figure 4 Prednisone vs myocarditis score. The averaged prednisone equivalent dose and myocarditis score are shown for patients with a response (left) and patients who did respond to treatment (right). In patients who responded, they received treatment on average 2-days earlier than patients who did not respond, and received higher doses earlier. Patients recovered from myocarditis often required prolonged high dose steroids before myocarditis scores returned to baseline.

Leave-One-Out (LOO) Model Validation

We then tested the robustness of the MMSE rankings of these 6 patients by leave-one-out (LOO) iterations, running the NN model for each group of 5 where one case was left out (Table 5). We also analyzed the subgroup of patients who responded to prednisone compared to prednisone non-responders for the outcome of myocarditis score, in order to determine clinical predictors of clinical decompensation in patients with steroid non-response. Rankings of clinical predictors were similar between iterations where one case was left out, indicating that the model was robust. For example, the combination parameters “CombLoadMyoMetric” and “CombAdvEvt” were the top two ranking parameters in 5 out of 7 iterations of the MMSE analysis. Troponin and troponin score were also highly correlated with the outcome for myocarditis score in every iteration of the analysis. Thus, model predictability is not driven by a single subject and is representative across the steroid treated population. Additionally, we analyzed the outcome of myocarditis score for patients that were prednisone non-responders. These patients displayed a markedly altered biomarker profile, with the top predictors of myocarditis score being CPK-MB, lactate, and CPK.

Table 5 Model MMSE Outputs for LOO Analysis of ICI-Related Myocarditis Cases, Comparing 6 Patients Who Had Clinical Resolution of Myocarditis in Response to Prednisone Treatment with 4 Non-Responder


Neural Net models based on electronic health records data yield large amounts of time and magnitude data elements. We leveraged the robust monitoring practices of critical care units to assemble a comprehensive model of each patients’ ICI-myocarditis onset, treatments, responses and outcomes. Additionally, we were able to circumvent challenges associated with modeling binary outcomes using convolutional NNs by converting our variables to clinical severity scoring systems, as applied to lab tests, all symptoms and concurrent irAEs. The model permitted complex investigations regarding the interrelationships between inputs and outputs. For example, we used troponin score to investigate the drivers of changes in troponin concentration, demonstrating the impact of immune activation on cardiomyocytes. Here, our analysis showed that across outcomes, troponin and myocarditis severity score were highly correlated with one another and with critical outcomes such as death.

Compared to conventional case report collection, we were able to utilize medical records data to construct a time and magnitude-related data set driven by the clinical time course of myocarditis. Over 300 different parameters were collected per patient and follow-up evaluation continued for outcomes for up to 6 months. Large Phase 3 studies typically collect about 500 data elements per patient, and a clinical study of 350 patients may yield a similar amount of data as we were able to collect on 23 patients. In the case of ICI-related myocarditis, most phase 3 case report forms did not report serial cardiac biomarkers and cardiac imaging results. Having many datapoints focused on a rare event is an advantage, in that fewer, more highly sampled patients can yield more insight into time and magnitude-related drivers of rare events in comparison to many sparsely sampled patients. The collection of large amounts of data on informative patients and analysis using a longitudinal NN modeling approach enables more precise analysis of important co-variates that may otherwise be overlooked. Specifically, the NN modeling approach is able to model non-linear correlations between time-related data arrays and clinical outcomes, which is an extraordinarily difficult task for conventional statistical subgroup analyses. By quantifying the time-related changes in the magnitude of the variables, we were able to model ICI-related myocarditis, including the response to steroid treatment, and the progressive changes in biomarkers of organ physiology as the condition worsened or abated.

We constructed models of clinical outcomes using both scored and unscored data points, and then algorithmically determined the most related input-output relationship for each variable. Here, scoring patients on a 0–5 scale simplified illustration, but did not appear to significantly influence the associations between variables and outputs of interest. Thus, the scoring of laboratory biomarkers and clinical disease severity on a 5-point scale allows a robust, yet convenient means of grouping data along a common time and magnitude axis. Scoring is particularly advantageous when variables are distributed non-linearly, such as troponin elevation ranging from 0.06 to >65 ng/mL. Scoring thus allows us to leverage our existing clinical knowledge to assist in constructing better, and more interpretable models, even though the model reaches the same conclusion without the graphical output-friendly transformation step.

Using this approach, any input of interest could also be used as an output and a model could be constructed to evaluate drivers within the same dataset. For example, each organ system that is affected by drug-related injury has a global marker, which could be used to build clinical models by designating these effects as outputs. Importantly, we severity scored each of these drug-related injury endpoints when collected. Thus, using a similar method, this same database could be used to further investigate colitis, neurological sequelae, thyroiditis, ophthalmological effects, hepatitis and pneumonitis, all of which were observed and scored in these patients. A potential next step in this work is co-modeling of clinical trial data with real-world data from electronic health records. The approach shown here models all the affected organ systems separately and together in the same time frame.

The time and magnitude modeling applied to these patients suggests that there may be a prodromal immune activation as an early signal of immune-related adverse events (irAEs). In these patients, prodromal immune activation leads to systemic organ involvement, and among a tiny fraction of activated patients, can precede the development of severe myocarditis upon progression. While the prediction of ICI-related myocarditis based on clinical risk factors has not been successful, our modeling demonstrated a means to determine factors associated with the early development of myocarditis. This method could potentially be applied to facilitate earlier detection and earlier treatment when biomarkers reveal off target organ involvement as the result of ICI treatment.

In general, the predominant absent parameter in those who died was early and aggressive use of steroid therapy, which differentiates the 11 cases who survived from the 4 cases who did not. In patients who died of fatal ICI-related myocarditis, steroids were used after troponin was very elevated and the myocarditis was likely extensive and ongoing in the heart. These cases also had conduction defects on EKG, and evidence of major secondary organ compromise such as hypotension, tissue hypoxia, compromised cardiac output and cardiogenic shock, and end organ damage.

Severe ICI-related myocarditis is rare and there have not been any baseline pre-treatment variables that identify those who will develop this rare complication, although higher LDH at baseline appears to be associated with this risk. A striking observation in severe cases was the rapid onset of ICI myocarditis in those afflicted, usually apparent in the second week after the first dose. This observation justifies the use of early detection strategies, such as weekly troponin monitoring. Early assessment of ICI-treated patients for symptoms of clinically significant inflammatory activation, such as fever, rash, fatigue, and lack of appetite, justifies an immediate measurement of troponin in these patients. The reason for early detection is clear – patients with rapid onset who receive high doses of steroids before they begin to show heart block, have a better chance of survival.47


Our model development was limited in scope by a relatively small number of cases. Hence, the focus of our analysis was to use NN modeling to investigate complex and non-linear associations between time- and magnitude-related changes between hundreds of inputs and only a few outcomes of interest. Although the LOO modeling showed these predictions to be quite stable among the remaining populations, the generalizability of the associations could be further strengthened with a larger sample size and more heterogeneity in the population. Future work in this area will endeavor to develop a predictive NN model for the early detection of ICI-related myocarditis with training and validation. Further investigations should also focus on determining optimal treatments and the ideal timing and dose of those treatments.


Our study shows that a supervised learning NN based on intensive sampling before, during and after irAEs can be used to model rare events such as ICI-related myocarditis and thus provide clinical insight into time and magnitude drivers of progression and treatment outcomes. This approach has the potential to distinguish between likely ICI-related myocarditis and other potential explanations for elevated troponin. These findings direct attention to early detection biomarkers and clinical symptoms as the best means of implementing early and potentially life-saving high dose steroid treatment.


MMSE, minimum mean squared error; NN, neural net; irAE, immune system related adverse events; ICI, immune checkpoint inhibitors; CTCAE, common terminology criteria for adverse events; CTLA-4, anti-cytotoxic T lymphocyte-associated antigen; PD-L1, anti-programmed cell death ligand-1; PD-1, anti-programmed cell death 1; NYHA, New York Heart Association; LOO, leave one out; ALT, alanine aminotransferase; AST, aspartate transaminase; LDH, lactate dehydrogenase; CPK, creatine phosphokinase; BNP, B-type natriuretic peptide; CPKMB, creatine kinase-MB; EKG, electrocardiogram.

Data Sharing Statement

All other relevant data are available from the corresponding author upon reasonable request.

Ethics Approval

All clinical procedures and protocols conformed to institutional guidelines and were approved by the institutional review board at the Roswell Park Comprehensive Cancer Center and at University at Buffalo. Due to the retrospective nature of the research and high mortality associated with the condition, a consent waiver was obtained, as well as a HIPPA waiver for individually identifiable information that could not be anonymized such as lab dates.

Consent for Publication

All authors have reviewed this manuscript and have agreed to its content for publication.

Author Contributions

All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.


The NN modeling and analysis of data was funded by Bristol Myers Squibb, and all authors acknowledge that funding supported this study. The funding entity did not provide cases, direct the work or influence the work product in any way.


Dr Andres Gomez-Caminero is an employee of Bristol Myers Squibb. Dr Igor Puzanov reports he has received consulting fees from Iovance, Nektar, Oncorus, Merck in past 2 years. Dr Jerome J Schentag reports grants from Bristol Myers Squibb, during the conduct of the study. The authors acknowledge Bristol Myers Squibb funded this work but have no other competing interests.


1. Mahmood SS, Fradley MG, Cohen JV, et al. Myocarditis in patients treated with immune checkpoint inhibitors. J Am Coll Cardiol. 2018;71(16):1755–1764. doi:10.1016/j.jacc.2018.02.037

2. Takai M, Kato D, Iinuma K, et al. Simultaneous pembrolizumab-induced myasthenia gravis and myocarditis in a patient with metastatic bladder cancer: a case report. Urol Case Rep. 2020;31:101145. doi:10.1016/j.eucr.2020.101145

3. Nierstedt RT, Yeahia R, Barnett KM. Unanticipated myocarditis in a surgical patient treated with pembrolizumab: a case report. A A Pract. 2020;14(6):e01177. doi:10.1213/XAA.0000000000001177

4. Lee DH, Armanious M, Huang J, Jeong D, Druta M, Fradley MG. Case of pembrolizumab-induced myocarditis presenting as torsades de pointes with safe re-challenge. J Oncol Pharm Pract. 2020;1078155220904152. doi:10.1177/1078155220904152

5. Wang Q, Hu B. Successful therapy for autoimmune myocarditis with pembrolizumab treatment for nasopharyngeal carcinoma. Ann Transl Med. 2019;7(11):247. doi:10.21037/atm.2019.04.73

6. Hellman JB, Traynis I, Lin LK. Pembrolizumab and epacadostat induced fatal myocarditis and myositis presenting as a case of ptosis and ophthalmoplegia. Orbit. 2019;38(3):244–247. doi:10.1080/01676830.2018.1490439

7. Katsume Y, Isawa T, Toi Y, et al. Complete atrioventricular block associated with pembrolizumab-induced acute myocarditis: the need for close cardiac monitoring. Internal Med. 2018;57(21):3157–3162. doi:10.2169/internalmedicine.0255-17

8. Martinez-Calle N, Rodriguez-Otero P, Villar S, et al. Anti-PD1 associated fulminant myocarditis after a single pembrolizumab dose: the role of occult pre-existing autoimmunity. Haematologica. 2018;103(7):e318–e21. doi:10.3324/haematol.2017.185777

9. Inayat F, Masab M, Gupta S, Ullah W. New drugs and new toxicities: pembrolizumab-induced myocarditis. BMJ Case Rep. 2018;2018. doi:10.1136/bcr-2017-223252

10. Nasr F, El Rassy E, Maalouf G, et al. Severe ophthalmoplegia and myocarditis following the administration of pembrolizumab. Eur J Cancer. 2018;91:171–173. doi:10.1016/j.ejca.2017.11.026

11. Laubli H, Balmelli C, Bossard M, Pfister O, Glatz K, Zippelius A. Acute heart failure due to autoimmune myocarditis under pembrolizumab treatment for metastatic melanoma. J Immunother Cancer. 2015;3(1):11. doi:10.1186/s40425-015-0057-1

12. Lie G, Weickhardt A, Kearney L, et al. Nivolumab resulting in persistently elevated troponin levels despite clinical remission of myocarditis and myositis in a patient with malignant pleural mesothelioma: case report. Transl Lung Cancer Res. 2020;9(2):360–365. doi:10.21037/tlcr.2020.02.05

13. Edahiro R, Shiroyama T, Hijiki S, et al. Severe myocarditis with slight lymphocytic infiltration after nivolumab treatment. Lung Cancer. 2020;140:116–117. doi:10.1016/j.lungcan.2019.12.004

14. Valenti-Azcarate R, Esparragosa Vazquez I, Toledano Illan C, Idoate Gastearena MA, Gallego Perez-Larraya J. Nivolumab and Ipilimumab-induced myositis and myocarditis mimicking a myasthenia gravis presentation. Neuromuscul Disord. 2020;30(1):67–69. doi:10.1016/j.nmd.2019.10.006

15. Tan JL, Mugwagwa AN, Cieslik L, Joshi R. Nivolumab-induced myocarditis complicated by complete atrioventricular block in a patient with metastatic non-small cell lung cancer. BMJ Case Rep. 2019;12(7):e229963. doi:10.1136/bcr-2019-229963

16. Saibil SD, Bonilla L, Majeed H, et al. Fatal myocarditis and rhabdomyositis in a patient with stage IV melanoma treated with combined ipilimumab and nivolumab. Curr Oncol. 2019;26(3):e418–e21. doi:10.3747/co.26.4381

17. Martin Huertas R, Saavedra Serrano C, Perna C, Ferrer Gomez A, Alonso Gordoa T. Cardiac toxicity of immune-checkpoint inhibitors: a clinical case of nivolumab-induced myocarditis and review of the evidence and new challenges. Cancer Manag Res. 2019;11:4541–4548. doi:10.2147/CMAR.S185202

18. Fazel M, Jedlowski PM. Severe myositis, myocarditis, and myasthenia gravis with elevated anti-striated muscle antibody following single dose of ipilimumab-nivolumab therapy in a patient with metastatic melanoma. Case Reports in Immunology. 2019;2019:2539493. doi:10.1155/2019/2539493

19. Matsuo K, Ishiguro T, Najama T, Shimizu Y, Kobayashi Y, Mutou M. Nivolumab-induced myocarditis successfully treated with corticosteroid therapy: a case report and review of the literature. Internal Med. 2019;58(16):2367–2372. doi:10.2169/internalmedicine.2596-18

20. Monge C, Maeng H, Brofferio A, et al. Myocarditis in a patient treated with Nivolumab and PROSTVAC: a case report. J Immunother Cancer. 2018;6(1):150. doi:10.1186/s40425-018-0473-0

21. Yamaguchi S, Morimoto R, Okumura T, et al. Late-onset fulminant myocarditis with immune checkpoint inhibitor nivolumab. Can J Cardiol. 2018;34(6):812e1- e3. doi:10.1016/j.cjca.2018.03.007

22. Chen Q, Huang DS, Zhang LW, Li YQ, Wang HW, Liu HB. Fatal myocarditis and rhabdomyolysis induced by nivolumab during the treatment of type B3 thymoma. Clin Toxicol. 2018;56(7):667–671. doi:10.1080/15563650.2017.1401079

23. Fukasawa Y, Sasaki K, Natsume M, et al. Nivolumab-induced myocarditis concomitant with myasthenia gravis. Case Rep Oncol. 2017;10(3):809–812. doi:10.1159/000479958

24. Matson DR, Accola MA, Rehrauer WM, Corliss RF. Fatal myocarditis following treatment with the PD-1 inhibitor nivolumab. J Forensic Sci. 2018;63(3):954–957. doi:10.1111/1556-4029.13633

25. Suzuki S, Ishikawa N, Konoeda F, et al. Nivolumab-related myasthenia gravis with myositis and myocarditis in Japan. Neurology. 2017;89(11):1127–1134. doi:10.1212/WNL.0000000000004359

26. Sauer R, Kiewe P, Desole M, et al. Lymphozytäre Myokarditis unter Nivolumabtherapie bei metastasiertem klarzelligen Nierenzellkarzinom. Pathologe. 2017;38(6):535–539. German. doi:10.1007/s00292-017-0349-y

27. Tay RY, Blackley E, McLean C, et al. Successful use of equine anti-thymocyte globulin (ATGAM) for fulminant myocarditis secondary to nivolumab therapy. Br J Cancer. 2017;117(7):921–924. doi:10.1038/bjc.2017.253

28. Tadokoro T, Keshino E, Makiyama A, et al. Acute lymphocytic myocarditis with anti-PD-1 antibody nivolumab. Circ Heart Fail. 2016;9(10). doi:10.1161/CIRCHEARTFAILURE.116.003514

29. Semper H, Muehlberg F, Schulz-Menger J, Allewelt M, Grohe C. Drug-induced myocarditis after nivolumab treatment in a patient with PDL1- negative squamous cell carcinoma of the lung. Lung Cancer. 2016;99:117–119. doi:10.1016/j.lungcan.2016.06.025

30. Gibson R, Delaune J, Szady A, Markham M. Suspected autoimmune myocarditis and cardiac conduction abnormalities with nivolumab therapy for non-small cell lung cancer. BMJ Case Rep. 2016;2016. doi:10.1136/bcr-2016-216228

31. Mehta A, Gupta A, Hannallah F, Koshy T, Reimold S. Myocarditis as an immune-related adverse event with ipilimumab/nivolumab combination therapy for metastatic melanoma. Melanoma Res. 2016;26(3):319–320. doi:10.1097/CMR.0000000000000251

32. Samara Y, Yu CL, Dasanu CA. Acute autoimmune myocarditis and hepatitis due to ipilimumab monotherapy for malignant melanoma. J Oncol Pharm Pract. 2019;25(4):966–968. doi:10.1177/1078155218755868

33. Reuben A, Petaccia de Macedo M, McQuade J, et al. Comparative immunologic characterization of autoimmune giant cell myocarditis with ipilimumab. Oncoimmunology. 2017;6(12):e1361097. doi:10.1080/2162402X.2017.1361097

34. Berg DD, Vaduganathan M, Nohria A, et al. Immune-related fulminant myocarditis in a patient receiving ipilimumab therapy for relapsed chronic myelomonocytic leukaemia. Eur J Heart Fail. 2017;19(5):682–685. doi:10.1002/ejhf.806

35. Gupta R, Zaid S, Sayed A, et al. Atezolizumab induced myocarditis on a background of cardiac amyloidosis. Am J Ther. 2019;26(6):e795–e7. doi:10.1097/MJT.0000000000000968

36. Mahmood SS, Chen CL, Shapnik N, Krishnan U, Singh HS, Makker V. Myocarditis with tremelimumab plus durvalumab combination therapy for endometrial cancer: a case report. Gynecol Oncol Rep. 2018;25:74–77. doi:10.1016/j.gore.2018.05.014

37. Touat M, Maisonobe T, Knauss S, et al. Immune checkpoint inhibitor-related myositis and myocarditis in patients with cancer. Neurology. 2018;91(10):e985–e94. doi:10.1212/WNL.0000000000006124

38. Puzanov I, Diab A, Abdallah K, et al.; Society for Immunotherapy of Cancer Toxicity Management Working Group. Managing toxicities associated with immune checkpoint inhibitors: consensus recommendations from the Society for Immunotherapy of Cancer (SITC) Toxicity Management Working Group. J Immunother Cancer. 2017;5(1):95. doi:10.1186/s40425-017-0300-z

39. Neilan TG, Rothenberg ML, Amiri-Kordestani L, et al.; Checkpoint Inhibitor Safety Working Group. Myocarditis associated with immune checkpoint inhibitors: an expert consensus on data gaps and a call to action. Oncologist. 2018;23(8):874–878. doi:10.1634/theoncologist.2018-0157

40. Johnson DB, Balko JM, Compton ML, et al. Fulminant myocarditis with combination immune checkpoint blockade. N Engl J Med. 2016;375(18):1749–1755. doi:10.1056/NEJMoa1609214

41. Patrinely JR, McGuigan B, Chandra S, et al. A multicenter characterization of hepatitis associated with immune checkpoint inhibitors. Oncoimmunology. 2021;10(1):1875639. doi:10.1080/2162402X.2021.1875639

42. Guidon AC, Burton LB, Chwalisz BK, et al. Consensus disease definitions for neurologic immune-related adverse events of immune checkpoint inhibitors. J Immunother Cancer. 2021;9(7):e002890. doi:10.1136/jitc-2021-002890

43. Wang DY, Mooradian MJ, Kim D, et al. Clinical characterization of colitis arising from anti-PD-1 based therapy. Oncoimmunology. 2019;8(1):e1524695. doi:10.1080/2162402X.2018.1524695

44. Johnson DB, Taylor KB, Cohen JV, et al. Anti-PD-1-induced pneumonitis is associated with persistent imaging abnormalities in melanoma patients. Cancer Immunol Res. 2019;7(11):1755–1759. doi:10.1158/2326-6066.CIR-18-0717

45. Xu D, Zhao RC, Gao WH, Cui HB. A risk prediction model for in-hospital mortality in patients with suspected myocarditis. Chin Med J. 2017;130(7):782–790. doi:10.4103/0366-6999.202747

46. Jacobs DM, Stefanovic F, Wilton G, Gomez-Caminero A, Schentag JJ. An integrated epidemiological and neural net model of the warfarin effect in managed care patients. Clin Pharmacol. 2017;9:55–64. doi:10.2147/CPAA.S136243

47. Puzanov I, Subramanian P, Yatsynovich YV, et al. Clinical characteristics, time course, treatment and outcomes of patients with immune checkpoint inhibitor-associated myocarditis. J Immunother Cancer. 2021;9(6):e002553. doi:10.1136/jitc-2021-002553

48. Hwang M, Leem CH, Shim EB. Toward a grey box approach for cardiovascular physiome. Korean J Physiol Pharmacol. 2019;23(5):305–310. doi:10.4196/kjpp.2019.23.5.305

Creative Commons License This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.