Rates of laboratory adverse events by course in paediatric leukaemia ascertained with automated electronic health record extraction: a retrospective cohort study from the Children's Oncology Group.

2022

e678-e688

07/2022

2352-3026

BACKGROUND: Adverse events are often misreported in clinical trials, leading to an incomplete understanding of toxicities. We aimed to test automated laboratory adverse event ascertainment and grading (via the ExtractEHR automated package) to assess its scalability and define adverse event rates for children with acute myeloid leukaemia and acute lymphoblastic leukaemia.

METHODS: For this retrospective cohort study from the Children's Oncology Group (COG), we included patients aged 0-22 years treated for acute myeloid leukaemia or acute lymphoblastic leukaemia at Children's Healthcare of Atlanta (Atlanta, GA, USA) from Jan 1, 2010, to Nov 1, 2018, at the Children's Hospital of Philadelphia (Philadelphia, PA, USA) from Jan 1, 2011, to Dec 31, 2014, and at the Texas Children's Hospital (Houston, TX, USA) from Jan 1, 2011, to Dec 31, 2014. The ExtractEHR automated package acquired, cleaned, and graded laboratory data as per Common Terminology Criteria for Adverse Events (CTCAE) version 5 for 22 commonly evaluated grade 3-4 adverse events (fatal events were not evaluated) with numerically based CTCAE definitions. Descriptive statistics tabulated adverse event frequencies. Adverse events ascertained by ExtractEHR were compared to manually reported adverse events for patients enrolled in two COG trials (AAML1031, NCT01371981; AALL0932, NCT02883049). Analyses were restricted to protocol-defined chemotherapy courses (induction I, induction II, intensification I, intensification II, and intensification III for acute myeloid leukaemia; induction, consolidation, interim maintenance, delayed intensification, and maintenance for acute lymphoblastic leukaemia).

FINDINGS: Laboratory adverse event data from 1077 patients (583 from Children's Healthcare of Atlanta, 200 from the Children's Hospital of Philadelphia, and 294 from the Texas Children's Hospital) who underwent 4611 courses (549 for acute myeloid leukaemia and 4062 for acute lymphoblastic leukaemia) were extracted, processed, and graded. Of the 166 patients with acute myeloid leukaemia, 86 (52%) were female, 80 (48%) were male, 96 (58%) were White, and 132 (80%) were non-Hispanic. Of the 911 patients with acute lymphoblastic leukaemia, 406 (45%) were female, 505 (55%) were male, 596 (65%) were White, and 641 (70%) were non-Hispanic. Patients with acute myeloid leukaemia had the most adverse events during induction I and intensification II. Hypokalaemia (one [17%] of six to 75 [48%] of 156 courses) and alanine aminotransferase (ALT) increased (13 [10%] of 134 to 27 [17%] of 156 courses) were the most prevalent non-haematological adverse events in patients with acute myeloid leukaemia, as identified by ExtractEHR. Patients with acute lymphoblastic leukaemia had the greatest number of adverse events during induction and maintenance (eight adverse events with prevalence ≥10%; induction and maintenance: anaemia, platelet count decreased, white blood cell count decreased, neutrophil count decreased, lymphocyte count decreased, ALT increased, and hypocalcaemia; induction: hypokalaemia; maintenance: aspartate aminotransferase [AST] increased and blood bilirubin increased), as identified by ExtractEHR. 187 (85%) of 220 total comparisons in 22 adverse events in four AAML1031 and six AALL0923 courses were substantially higher with ExtractEHR than COG-reported adverse event rates for adverse events with a prevalence of at least 2%.

INTERPRETATION: ExtractEHR is scalable and accurately defines laboratory adverse event rates for paediatric acute leukaemia; moreover, ExtractEHR seems to detect higher rates of laboratory adverse events than those reported in COG trials. These rates can be used for comparisons between therapies and to counsel patients treated on or off trials about the risks of chemotherapy. ExtractEHR-based adverse event ascertainment can improve reporting of laboratory adverse events in clinical trials.

FUNDING: US National Institutes of Health, St Baldrick's Foundation, and Alex's Lemonade Stand Foundation.

10.1016/S2352-3026(22)00168-5

Lancet Haematol

35870472

Rates of laboratory adverse events by course in paediatric leukaemia ascertained with automated electronic health record extraction: a retrospective cohort study from the Children's Oncology Group.

2022

e678-e688

07/2022

2352-3026

BACKGROUND: Adverse events are often misreported in clinical trials, leading to an incomplete understanding of toxicities. We aimed to test automated laboratory adverse event ascertainment and grading (via the ExtractEHR automated package) to assess its scalability and define adverse event rates for children with acute myeloid leukaemia and acute lymphoblastic leukaemia.

METHODS: For this retrospective cohort study from the Children's Oncology Group (COG), we included patients aged 0-22 years treated for acute myeloid leukaemia or acute lymphoblastic leukaemia at Children's Healthcare of Atlanta (Atlanta, GA, USA) from Jan 1, 2010, to Nov 1, 2018, at the Children's Hospital of Philadelphia (Philadelphia, PA, USA) from Jan 1, 2011, to Dec 31, 2014, and at the Texas Children's Hospital (Houston, TX, USA) from Jan 1, 2011, to Dec 31, 2014. The ExtractEHR automated package acquired, cleaned, and graded laboratory data as per Common Terminology Criteria for Adverse Events (CTCAE) version 5 for 22 commonly evaluated grade 3-4 adverse events (fatal events were not evaluated) with numerically based CTCAE definitions. Descriptive statistics tabulated adverse event frequencies. Adverse events ascertained by ExtractEHR were compared to manually reported adverse events for patients enrolled in two COG trials (AAML1031, NCT01371981; AALL0932, NCT02883049). Analyses were restricted to protocol-defined chemotherapy courses (induction I, induction II, intensification I, intensification II, and intensification III for acute myeloid leukaemia; induction, consolidation, interim maintenance, delayed intensification, and maintenance for acute lymphoblastic leukaemia).

FINDINGS: Laboratory adverse event data from 1077 patients (583 from Children's Healthcare of Atlanta, 200 from the Children's Hospital of Philadelphia, and 294 from the Texas Children's Hospital) who underwent 4611 courses (549 for acute myeloid leukaemia and 4062 for acute lymphoblastic leukaemia) were extracted, processed, and graded. Of the 166 patients with acute myeloid leukaemia, 86 (52%) were female, 80 (48%) were male, 96 (58%) were White, and 132 (80%) were non-Hispanic. Of the 911 patients with acute lymphoblastic leukaemia, 406 (45%) were female, 505 (55%) were male, 596 (65%) were White, and 641 (70%) were non-Hispanic. Patients with acute myeloid leukaemia had the most adverse events during induction I and intensification II. Hypokalaemia (one [17%] of six to 75 [48%] of 156 courses) and alanine aminotransferase (ALT) increased (13 [10%] of 134 to 27 [17%] of 156 courses) were the most prevalent non-haematological adverse events in patients with acute myeloid leukaemia, as identified by ExtractEHR. Patients with acute lymphoblastic leukaemia had the greatest number of adverse events during induction and maintenance (eight adverse events with prevalence ≥10%; induction and maintenance: anaemia, platelet count decreased, white blood cell count decreased, neutrophil count decreased, lymphocyte count decreased, ALT increased, and hypocalcaemia; induction: hypokalaemia; maintenance: aspartate aminotransferase [AST] increased and blood bilirubin increased), as identified by ExtractEHR. 187 (85%) of 220 total comparisons in 22 adverse events in four AAML1031 and six AALL0923 courses were substantially higher with ExtractEHR than COG-reported adverse event rates for adverse events with a prevalence of at least 2%.

INTERPRETATION: ExtractEHR is scalable and accurately defines laboratory adverse event rates for paediatric acute leukaemia; moreover, ExtractEHR seems to detect higher rates of laboratory adverse events than those reported in COG trials. These rates can be used for comparisons between therapies and to counsel patients treated on or off trials about the risks of chemotherapy. ExtractEHR-based adverse event ascertainment can improve reporting of laboratory adverse events in clinical trials.

FUNDING: US National Institutes of Health, St Baldrick's Foundation, and Alex's Lemonade Stand Foundation.

10.1016/S2352-3026(22)00168-5

Lancet Haematol

35870472

Unintended consequences of evolution of the Common Terminology Criteria for Adverse Events.

2019

e27747

2019 Apr 09

1545-5017

<p><strong>BACKGROUND: </strong>Adverse events (AEs) on Children's Oncology Group (COG) trials are reported manually by clinical research assistants (CRAs). The Common Terminology Criteria for Adverse Events (CTCAE) was developed to provide standardized definitions for identifying and grading AEs. The CTCAE has expanded significantly over its five versions, but the impact of CTCAE definitional changes has not been examined.</p>

<p><strong>PROCEDURE: </strong>This study compared AE number and ascertainment among the first four CTCAE versions using a case vignette. Each CTCAE version was used to create a list of AEs and grades by two separate CRAs.</p>

<p><strong>RESULTS: </strong>The CTCAE expanded from 9 categories and 49 AEs in v1.0 to 26 categories and 790 AEs in v4.0. CRAs independently selected different approaches to AE ascertainment-comprehensive and parsimonious. The number of AEs identified in the parsimonious approach was stable with 10-14 in each CTC version. The comprehensive approach identified 9, 20, 29, and 37 AEs in CTC versions 1.0, 2.0, 3.0, and 4.0, respectively. Only approximately 65% of AEs were conclusively graded in versions 2.0 to 4.0 using the comprehensive approach.</p>

<p><strong>CONCLUSIONS: </strong>CTCAE has increased in complexity. Although this increased complexity allows for more granular AE reporting, these data demonstrate potential unintended negative consequences of increasing CTC AE complexity, including the risk of varying approaches to AE capture. A comprehensive evaluation of CTC AE definitions and CRA reporting practices across COG institutions and AEs are needed to improve the accuracy and efficiency of AE reporting.</p>

10.1002/pbc.27747

Pediatr Blood Cancer

30968531

Center-level variation in accuracy of adverse event reporting in a clinical trial for pediatric acute myeloid leukemia: A report from the Children's Oncology Group.

2017

2017 Jun 22

1592-8721

10.3324/haematol.2017.168815

Haematologica

28642300

Merging Children's Oncology Group Data with an External Administrative Database Using Indirect Patient Identifiers: A Report from the Children's Oncology Group.

2015

e0143480

2015

1932-6203

<p><strong>PURPOSE: </strong>Clinical trials data from National Cancer Institute (NCI)-funded cooperative oncology group trials could be enhanced by merging with external data sources. Merging without direct patient identifiers would provide additional patient privacy protections. We sought to develop and validate a matching algorithm that uses only indirect patient identifiers.</p>

<p><strong>METHODS: </strong>We merged the data from two Phase III Children's Oncology Group (COG) trials for de novo acute myeloid leukemia (AML) with the Pediatric Health Information Systems (PHIS). We developed a stepwise matching algorithm that used indirect identifiers including treatment site, gender, birth year, birth month, enrollment year and enrollment month. Results from the stepwise algorithm were compared against the direct merge method that used date of birth, treatment site, and gender. The indirect merge algorithm was developed on AAML0531 and validated on AAML1031.</p>

<p><strong>RESULTS: </strong>Of 415 patients enrolled on the AAML0531 trial at PHIS centers, we successfully matched 378 (91.1%) patients using the indirect stepwise algorithm. Comparison to the direct merge result suggested that 362 (95.7%) matches identified by the indirect merge algorithm were concordant with the direct merge result. When validating the indirect stepwise algorithm using the AAML1031 trial, we successfully matched 157 out of 165 patients (95.2%) and 150 (95.5%) of the indirectly merged matches were concordant with the directly merged matches.</p>

<p><strong>CONCLUSIONS: </strong>These data demonstrate that patients enrolled on COG clinical trials can be successfully merged with PHIS administrative data using a stepwise algorithm based on indirect patient identifiers. The merged data sets can be used as a platform for comparative effectiveness and cost effectiveness studies.</p>

10.1371/journal.pone.0143480

PLoS ONE

26606521

Accuracy of Adverse Event Ascertainment in Clinical Trials for Pediatric Acute Myeloid Leukemia.

2016

2016 Feb 16

1527-7755

<p><strong>PURPOSE: </strong>Reporting of adverse events (AEs) in clinical trials is critical to understanding treatment safety, but data on AE accuracy are limited. This study sought to determine the accuracy of AE reporting for pediatric acute myeloid leukemia clinical trials and to test whether an external electronic data source can improve reporting.</p>

<p><strong>METHODS: </strong>Reported AEs were evaluated on two trials, Children's Oncology Group AAML03P1 and AAML0531 arm B, with identical chemotherapy regimens but with different toxicity reporting requirements. Chart review for 12 AEs for patients enrolled in AAML0531 at 14 hospitals was the gold standard. The sensitivity and positive predictive values (PPV) of the AAML0531 AE report and AEs detected by review of Pediatric Health Information System (PHIS) billing and microbiology data were compared with chart data.</p>

<p><strong>RESULTS: </strong>Select AE rates from AAML03P1 and AAML0531 arm B differed significantly and correlated with the targeted toxicities of each trial. Chart abstraction was performed on 204 patients (758 courses) on AAML0531. AE report sensitivity was &lt; 50% for eight AEs, but PPV was &gt; 75% for six AEs. AE reports for viridans group streptococcal bacteremia, a targeted toxicity on AAML0531, had a sensitivity of 78.3% and PPV of 98.1%. PHIS billing data had higher sensitivity (&gt; 50% for nine AEs), but lower PPV (&lt; 75% for 10 AEs). Viridans group streptococcal detection using PHIS microbiology data had high sensitivity (92.3%) and PPV (97.3%).</p>

<p><strong>CONCLUSION: </strong>The current system of AE reporting for cooperative oncology group clinical trials in pediatric acute myeloid leukemia underestimates AE rates. The high sensitivity and PPV of PHIS microbiology data suggest that using external data sources may improve the accuracy of AE reporting.</p>

10.1200/JCO.2015.65.5860

J. Clin. Oncol.

26884558

Lessons learned from the investigational device exemption review of Children's Oncology Group trial AAML1031.

2012

1547-54

2012 Mar 15

1078-0432

<p>The U.S. Food and Drug Administration is now exerting its regulatory authority over the use of molecular diagnostics and related assays for medical decision making in clinical trials, by performing pre-Investigational Device Exemption reviews in all phases of clinical trials. In this review, we assess the analytical performance of the assay for the diagnostic, and consider how that performance affects the diagnostic and the patient and their risks and benefits from treatment. We also discuss the process involved in the first review of a new Children's Oncology Group phase III trial in acute myelogenous leukemia. The lessons learned and recommendations for how to prepare for and incorporate this new level of regulatory review into the protocol development process are presented.</p>

10.1158/1078-0432.CCR-11-2205

Clin. Cancer Res.

22422407

Comparison of in-patient costs for children treated on the AAML0531 clinical trial: A report from the Children's Oncology Group.

2015

1775-81

2015 Oct

1545-5017

<p><strong>BACKGROUND: </strong>A better understanding of drivers of treatment costs may help identify effective cost containment strategies and prioritize resources. We aimed to develop a method for estimating inpatient costs for pediatric patients with acute myeloid leukemia (AML) enrolled on NCI-funded Phase III trials, compare costs between AAML0531 treatment arms (standard chemotherapy ± gemtuzumab ozogamicin (GMTZ)), and evaluate primary drivers of costs for newly diagnosed pediatric AML.</p>

<p><strong>PROCEDURE: </strong>Patients from the AAML0531 trial were matched on hospital, sex, and dates of birth and diagnosis to the Pediatric Health Information Systems (PHIS) database to obtain daily billing data. Inpatient treatment costs were calculated as adjusted charges multiplied by hospital-specific cost-to-charge ratios. Generalized linear models were used to compare costs between treatment arms and courses, and by patient characteristics.</p>

<p><strong>RESULTS: </strong>Inpatient costs did not differ by randomized treatment arm. Costs varied by course with stem cell transplant being most expensive, followed by Intensification II (cytarabine/mitoxantrone) and Induction I (cytarabine/daunorubicin/etoposide). Room/board and pharmacy were the largest contributors to inpatient treatment cost, representing 74% of the total cost. Higher AML risk group (P = 0.0003) and older age (P &lt; 0.0001) were associated with significantly higher daily inpatient cost.</p>

<p><strong>CONCLUSIONS: </strong>Costs from external data sources can be successfully integrated into NCI-funded Phase III clinical trials. Inpatient treatment costs did not differ by GMTZ exposure but varied by chemotherapy course. Variation in cost by course was driven by differences in duration of hospitalization through room/board charges as well as increased clinical and pharmacy charges in specific courses.</p>

10.1002/pbc.25569

Pediatr Blood Cancer

25946708

Merging of the National Cancer Institute-funded cooperative oncology group data with an administrative data source to develop a more effective platform for clinical trial analysis and comparative effectiveness research: a report from the COG.

2012

37-43

2012 May

1099-1557

<p><strong>PURPOSE: </strong>The National Cancer Institute-funded cooperative oncology group trials have improved overall survival for children with cancer from 10% to 85% and have set standards of care for adults with malignancies. Despite these successes, cooperative oncology groups currently face substantial challenges. We are working to develop methods to improve the efficiency and effectiveness of these trials. Specifically, we merged data from the Children's Oncology Group (COG) and the Pediatric Health Information Systems (PHIS) to improve toxicity monitoring, to estimate treatment-associated resource utilization and costs, and to address important clinical epidemiology questions.</p>

<p><strong>METHODS: </strong>COG and PHIS data on patients enrolled on a phase III COG trial for de novo acute myeloid leukemia at 43 PHIS hospitals were merged using a probabilistic algorithm. Resource utilization summary statistics were then tabulated for the first chemotherapy course based on PHIS data.</p>

<p><strong>RESULTS: </strong>Of 416 patients enrolled on the phase III COG trial at PHIS centers, 392 (94%) were successfully matched. Of these, 378 (96%) had inpatient PHIS data available beginning at the date of study enrollment. For these, daily blood product usage and anti-infective exposures were tabulated and standardized costs were described.</p>

<p><strong>CONCLUSIONS: </strong>These data demonstrate that patients enrolled in a cooperative group oncology trial can be successfully identified in an administrative data set and that supportive care resource utilization can be described. Further work is required to optimize the merging algorithm, map resource utilization metrics to the National Cancer Institute Common Toxicity Criteria for monitoring toxicity, to perform comparative effectiveness studies, and to estimate the costs associated with protocol therapy.</p>

10.1002/pds.3241

Pharmacoepidemiol Drug Saf

22552978

The impact of chemotherapy shortages on COG and local clinical trials: a report from the Children's Oncology Group.

2015

940-4

06/2015

1545-5017

<p><strong>BACKGROUND: </strong>Oncology drug shortage is associated with increased patient adverse events and decreased enrollment on clinical trials for adult patients; however, the impact of oncology drug shortages has not been well studied in children with cancer.</p>

<p><strong>PROCEDURE: </strong>The Children's Oncology Group (COG) distributed a 5-item survey to 226 COG site-specific principal investigators (PI's) and 14-item survey to 161 COG pharmacists to gather data the impact of chemotherapeutic shortages on clinical trials and patient care.</p>

<p><strong>RESULTS: </strong>The response rate was 66.4% (150/226) for PI's and 29.8% (48/161) for pharmacists. COG PI's reported daunorubicin (73%), methotrexate (56%), asparaginase/PEG-asparaginase (42%), doxorubicin (26%), thiotepa (21%), and cytarabine (20%) were most commonly in shortage, while COG pharmacists reported daunorubicin (80%), methotrexate (66%), vincristine (21%), thiotepa (41%), asparaginase/PEG-asparaginase (34%), and cytarabine (34%) were most commonly in shortage over the past two years. Pharmacists were twice as likely to report a shortage compared with PI's (OR 2.1, 95% CI: 1.6-2.7, P &lt; 0.0001). Fifty percent (74/147) of COG PI's reported at least one patient enrolled on a clinical trial was impacted by drug shortage, and 66% (98/148) of COG PI's reported at least one patient had clinical care impacted by drug shortage.</p>

<p><strong>CONCLUSIONS: </strong>Chemotherapy shortages remain widespread across institutions, hinder clinical trials, and may contribute to adverse events in children with cancer. The increased frequency of chemotherapy shortages reported by pharmacists suggests that pharmacist efforts may mitigate negative impact chemotherapy shortages. Over half of pediatric institutions are implementing recommendations to address shortages, such as cross-institutional collaboration and center-level guidelines.</p>

10.1002/pbc.25445

Pediatr Blood Cancer

25704486