Interruptive clinical decision support systems, both within and outside of electronic health records, are a resource that should be used sparingly and monitored closely. Excessive use of interruptive alerting can quickly lead to alert fatigue and decreased effectiveness and ignoring of alerts. In this review, we discuss the evidence for effective alert stewardship as well as practices and methods we have found useful to assess interruptive alert burden, reduce excessive firings, optimize alert effectiveness, and establish quality governance at our institutions. We also discuss the importance of a holistic view of the alerting ecosystem beyond the electronic health record.

<p><strong>OBJECTIVES: </strong>To assess the current landscape of clinical decision support (CDS) tools in PICUs in order to identify priority areas of focus in this field.</p>

<p><strong>DESIGN: </strong>International, quantitative, cross-sectional survey.</p>

<p><strong>SETTING: </strong>Role-specific, web-based survey administered in November and December 2020.</p>

<p><strong>SUBJECTS: </strong>Medical directors, bedside nurses, attending physicians, and residents/advanced practice providers at Pediatric Acute Lung Injury and Sepsis Network-affiliated PICUs.</p>

<p><strong>INTERVENTIONS: </strong>None.</p>

<p><strong>MEASUREMENTS AND MAIN RESULTS: </strong>The survey was completed by 109 respondents from 45 institutions, primarily attending physicians from university-affiliated PICUs in the United States. The most commonly used CDS tools were people-based resources (93% used always or most of the time) and laboratory result highlighting (86%), with order sets, order-based alerts, and other electronic CDS tools also used frequently. The most important goal providers endorsed for CDS tools were a proven impact on patient safety and an evidence base for their use. Negative perceptions of CDS included concerns about diminished critical thinking and the burden of intrusive processes on providers. Routine assessment of existing CDS was rare, with infrequent reported use of observation to assess CDS impact on workflows or measures of individual alert burden.</p>

<p><strong>CONCLUSIONS: </strong>Although providers share some consensus over CDS utility, we identified specific priority areas of research focus. Consensus across practitioners exists around the importance of evidence-based CDS tools having a proven impact on patient safety. Despite broad presence of CDS tools in PICUs, practitioners continue to view them as intrusive and with concern for diminished critical thinking. Deimplementing ineffective CDS may mitigate this burden, though postimplementation evaluation of CDS is rare.</p>

<p><strong>OBJECTIVE: </strong>Training disruptions, such as planned curricular adjustments or unplanned global pandemics, impact residency training in ways that are difficult to quantify. Informatics-based medical education tools can help measure these impacts. We tested the ability of a software platform driven by electronic health record data to quantify anticipated changes in trainee clinical experiences during the COVID-19 pandemic.</p>

<p><strong>METHODS: </strong>We previously developed and validated the Trainee Individualized Learning System (TRAILS) to identify pediatric resident clinical experiences (i.e. shifts, resident provider-patient interactions (rPPIs), and diagnoses). We used TRAILS to perform a year-over-year analysis comparing pediatrics residents at a large academic children's hospital during March 15 - June 15 in 2018 (Control #1), 2019 (Control #2) and 2020 (Exposure).</p>

<p><strong>RESULTS: </strong>Residents in the exposure cohort had fewer shifts than those in both control cohorts (p &lt; 0.05). rPPIs decreased an average of 43% across all PGY levels, with interns experiencing a 78% decrease in Continuity Clinic. Patient continuity decreased from 23% to 11%. rPPIs with common clinic and emergency department diagnoses decreased substantially during the exposure period.</p>

<p><strong>CONCLUSIONS: </strong>Informatics tools like TRAILS may help program directors understand the impact of training disruptions on resident clinical experiences and target interventions to learners' needs and development.</p>

<p><strong>BACKGROUND: </strong>Excessive electronic health record (EHR) alerts reduce the salience of actionable alerts. Little is known about the frequency of interruptive alerts across health systems and how the choice of metric affects which users appear to have the highest alert burden.</p>

<p><strong>OBJECTIVE: </strong>(1) Analyze alert burden by alert type, care setting, provider type, and individual provider across 6 pediatric health systems. (2) Compare alert burden using different metrics.</p>

<p><strong>MATERIALS AND METHODS: </strong>We analyzed interruptive alert firings logged in EHR databases at 6 pediatric health systems from 2016-2019 using 4 metrics: (1) alerts per patient encounter, (2) alerts per inpatient-day, (3) alerts per 100 orders, and (4) alerts per unique clinician days (calendar days with at least 1 EHR log in the system). We assessed intra- and interinstitutional variation and how alert burden rankings differed based on the chosen metric.</p>

<p><strong>RESULTS: </strong>Alert burden varied widely across institutions, ranging from 0.06 to 0.76 firings per encounter, 0.22 to 1.06 firings per inpatient-day, 0.98 to 17.42 per 100 orders, and 0.08 to 3.34 firings per clinician day logged in the EHR. Custom alerts accounted for the greatest burden at all 6 sites. The rank order of institutions by alert burden was similar regardless of which alert burden metric was chosen. Within institutions, the alert burden metric choice substantially affected which provider types and care settings appeared to experience the highest alert burden.</p>

<p><strong>CONCLUSION: </strong>Estimates of the clinical areas with highest alert burden varied substantially by institution and based on the metric used.</p>

<p><strong>Importance: </strong>Incoming text messages and calls on nurses' mobile telephones may interrupt medication administration, but whether such interruptions are associated with errors has not been established.</p>

<p><strong>Objective: </strong>To assess whether a temporal association exists between mobile telephone interruptions and subsequent errors by pediatric intensive care unit (PICU) nurses during medication administration.</p>

<p><strong>Design, Setting, and Participants: </strong>A retrospective cohort study was performed using telecommunications and electronic health record data from a PICU in a children's hospital. Data were collected from August 1, 2016, through September 30, 2017. Participants included 257 nurses and the 3308 patients to whom they administered medications.</p>

<p><strong>Exposures: </strong>Primary exposures were incoming telephone calls and text messages received on the institutional mobile telephone assigned to the nurse in the 10 minutes leading up to a medication administration attempt. Secondary exposures were the nurse's PICU experience, work shift (day vs night), nurse to patient ratio, and level of patient care required.</p>

<p><strong>Main Outcomes and Measures: </strong>Primary outcome, errors during medication administration, was a composite of reported medication administration errors and bar code medication administration error alerts generated when nurses attempted to give medications without active orders for the patient whose bar code they scanned.</p>

<p><strong>Results: </strong>Participants included 257 nurses, of whom 168 (65.4%) had 6 months or more of PICU experience; and 3308 patients, of whom 1839 (55.6%) were male, 1539 (46.5%) were white, and 2880 (87.1%) were non-Hispanic. The overall rate of errors during 238 540 medication administration attempts was 3.1% (95% CI, 3.0%-3.3%) when nurses were uninterrupted by incoming telephone calls and 3.7% (95% CI, 3.4%-4.0%) when they were interrupted by such calls. During day shift, the odds ratios (ORs) for error when interrupted by calls (compared with uninterrupted) were 1.02 (95% CI, 0.92-1.13; P = .73) among nurses with 6 months or more of PICU experience and 1.22 (95% CI, 1.00-1.47; P = .046) among nurses with less than 6 months of experience. During night shift, the ORs for error when interrupted by calls were 1.35 (95% CI, 1.16-1.57; P &lt; .001) among nurses with 6 months or more of PICU experience and 1.53 (95% CI, 1.16-2.03; P = .003) among nurses with less than 6 months of experience. Nurses administering medications to 1 or more patients receiving mechanical ventilation and arterial catheterization while caring for at least 1 other patient had an increased risk of error (OR, 1.21; 95% CI, 1.03-1.42; P = .02). Incoming text messages were not associated with error (OR, 0.97; 95% CI, 0.92-1.02; P = .22).</p>

<p><strong>Conclusions and Relevance: </strong>This study's findings suggest that incoming telephone call interruptions may be temporally associated with medication administration errors among PICU nurses. Risk of error varied by shift, experience, nurse to patient ratio, and level of patient care required.</p>

<p>Low-value health care is pervasive in the United States, and clinicians need to be trained to be stewards of health care resources. Despite a mandate by the Accreditation Council for Graduate Medical Education to educate trainee physicians on cost awareness, only 10% of pediatric residency programs have a high-value care (HVC) curriculum. To meet this need, we set out to develop and evaluate the impact of High-Value Pediatrics, an open-access HVC curriculum. High-Value Pediatrics is a 3-part curriculum that includes 4 standardized didactics, monthly interactive morning reports, and an embedded HVC improvement project. Curriculum evaluation through an anonymous, voluntary survey revealed an improvement in the self-reported knowledge of health care costs, charges, reimbursement, and value ( &lt; .05). Qualitative results revealed self-reported behavior changes, and HVC improvement projects resulted in higher-value patient care. The implementation of High-Value Pediatrics is feasible and reveals improved knowledge and attitudes about HVC. HVC improvement projects augmented curricular knowledge gains and revealed behavior changes. It is imperative that formal high-value education be taught to every pediatric trainee to lead the culture change that is necessary to turn the tide against low-value health care. In addition, simultaneous work on faculty education and attention to the hidden curriculum of low-value care is needed for sustained and long-term improvements.</p>

<p><strong>Objective: </strong>Electronic health record (EHR) simulation with realistic test patients has improved recognition of safety concerns in test environments. We assessed if simulation affects EHR use patterns in real clinical settings.</p>

<p><strong>Materials and Methods: </strong>We created a 1-hour educational intervention of a simulated admission for pediatric interns. Data visualization and information retrieval tools were introduced to facilitate recognition of the patient's clinical status. Using EHR audit logs, we assessed the frequency with which these tools were accessed by residents prior to simulation exposure (intervention group, pre-simulation), after simulation exposure (intervention group, post-simulation), and among residents who never participated in simulation (control group).</p>

<p><strong>Results: </strong>From July 2015 to February 2017, 57 pediatric residents participated in a simulation and 82 did not. Residents were more likely to use the data visualization tool after simulation (73% in post-simulation weeks vs 47% of combined pre-simulation and control weeks, P &lt;. 0001) as well as the information retrieval tool (85% vs 36%, P &lt; .0001). After adjusting for residents' experiences measured in previously completed inpatient weeks of service, simulation remained a significant predictor of using the data visualization (OR 2.8, CI: 2.1-3.9) and information retrieval tools (OR 3.0, CI: 2.0-4.5). Tool use did not decrease in interrupted time-series analysis over a median of 19 (IQR: 8-32) weeks of post-simulation follow-up.</p>

<p><strong>Discussion: </strong>Simulation was associated with persistent changes to EHR use patterns among pediatric residents.</p>

<p><strong>Conclusion: </strong>EHR simulation is an effective educational method that can change participants' use patterns in real clinical settings.</p>

<p>We retrospectively studied the effect of introducing procalcitonin into clinical practice on antibiotic use within a large academic pediatric intensive care unit. In the absence of a standardized algorithm, availability of the procalcitonin assay did not reduce the frequency of antibiotic initiations or the continuation of antibiotics for greater than 72 hours.</p>

<p><strong>OBJECTIVES: </strong>To determine whether peak blood procalcitonin (PCT) measured within 48 hours of pediatric intensive care unit (PICU) admission can differentiate severe bacterial infections from sterile inflammation and viral infection and identify potential subgroups of PICU patients for whom PCT may not have clinical utility.</p>

<p><strong>STUDY DESIGN: </strong>This was a retrospective, observational study of 646 critically ill children who had PCT measured within 48 hours of admission to an urban, academic PICU. Patients were stratified into 6 categories by infection status. We compared test characteristics for peak PCT, C-reactive protein (CRP), white blood cell count (WBC), absolute neutrophil count (ANC), and % immature neutrophils. The area under the receiver operating characteristic curve was determined for each biomarker to discriminate bacterial infection.</p>

<p><strong>RESULTS: </strong>The area under the receiver operating characteristic curve was similar for PCT (0.73, 95% CI 0.69, 0.77) and CRP (0.75, 95% CI 0.71, 0.79; P = .36), but both outperformed WBC, ANC, and % immature neutrophils (P &lt; .01 for all pairwise comparisons). The combination of PCT and CRP was no better than either PCT or CRP alone. Diagnostic patterns prone to false-positive and false-negative PCT values were identified.</p>

<p><strong>CONCLUSIONS: </strong>Peak blood PCT measured close to PICU admission was not superior to CRP in differentiating severe bacterial infection from viral illness and sterile inflammation; both PCT and CRP outperformed WBC, ANC, and % immature neutrophils. PCT appeared especially prone to inaccuracies in detecting localized bacterial central nervous system infections or bacterial coinfection in acute viral illness causing respiratory failure.</p>