<p><strong>BACKGROUND AND OBJECTIVE: </strong>Pediatric in-hospital cardiac arrest most commonly occurs in the pediatric intensive care unit (PICU) and is frequently preceded by early warning signs of clinical deterioration. In this study, we describe the implementation and evaluation of criteria to identify high-risk patients from a paper-based checklist into a clinical decision support (CDS) tool in the electronic health record (EHR).</p>

<p><strong>MATERIALS AND METHODS: </strong>The validated paper-based tool was first adapted by PICU clinicians and clinical informaticians and then integrated into clinical workflow following best practices for CDS design. A vendor-based rule engine was utilized. Littenberg's assessment framework helped guide the overall evaluation. Preliminary testing took place in EHR development environments with more rigorous evaluation, testing, and feedback completed in the live production environment. To verify data quality of the CDS rule engine, a retrospective Structured Query Language (SQL) data query was also created. As a process metric, preparedness was measured in pre- and postimplementation surveys.</p>

<p><strong>RESULTS: </strong>The system was deployed, evaluating approximately 340 unique patients monthly across 4 clinical teams. The verification against retrospective SQL of 15-minute intervals over a 30-day period revealed no missing triggered intervals and demonstrated 99.3% concordance of positive triggers. Preparedness showed improvements across multiple domains to our a priori goal of 90%.</p>

<p><strong>CONCLUSION: </strong>We describe the successful adaptation and implementation of a real-time CDS tool to identify PICU patients at risk of deterioration. Prospective multicenter evaluation of the tool's effectiveness on clinical outcomes is necessary before broader implementation can be recommended.</p>

<p><strong>OBJECTIVES: </strong>To evaluate the translation of a paper high-risk checklist for PICU patients at risk of clinical deterioration to an automated clinical decision support tool.</p>

<p><strong>DESIGN: </strong>Retrospective, observational cohort study of an automated clinical decision support tool, the PICU Warning Tool, adapted from a paper checklist to predict clinical deterioration events in PICU patients within 24 hours.</p>

<p><strong>SETTING: </strong>Two quaternary care medical-surgical PICUs-The Children's Hospital of Philadelphia and Cincinnati Children's Hospital Medical Center.</p>

<p><strong>PATIENTS: </strong>The study included all patients admitted from July 1, 2014, to June 30, 2015, the year prior to the initiation of any focused situational awareness work at either institution.</p>

<p><strong>INTERVENTIONS: </strong>We replicated the predictions of the real-time PICU Warning Tool by retrospectively querying the institutional data warehouse to identify all patients that would have flagged as high-risk by the PICU Warning Tool for their index deterioration.</p>

<p><strong>MEASUREMENTS AND MAIN RESULTS: </strong>The primary exposure of interest was determination of high-risk status during PICU admission via the PICU Warning Tool. The primary outcome of interest was clinical deterioration event within 24 hours of a positive screen. The date and time of the deterioration event was used as the index time point. We evaluated the sensitivity, specificity, positive predictive value, and negative predictive value of the performance of the PICU Warning Tool. There were 6,233 patients evaluated with 233 clinical deterioration events experienced by 154 individual patients. The positive predictive value of the PICU Warning Tool was 7.1% with a number needed to screen of 14 patients for each index clinical deterioration event. The most predictive of the individual criteria were elevated lactic acidosis, high mean airway pressure, and profound acidosis.</p>

<p><strong>CONCLUSIONS: </strong>Performance of a clinical decision support translation of a paper-based tool showed inferior test characteristics. Improved feasibility of identification of high-risk patients using automated tools must be balanced with performance.</p>

<p><strong>OBJECTIVES: </strong>To compare the performance of three methods of identifying children with severe sepsis and septic shock from the Virtual Pediatric Systems database to prospective screening using consensus criteria.</p>

<p><strong>DESIGN: </strong>Observational cohort study.</p>

<p><strong>SETTING: </strong>Single-center PICU.</p>

<p><strong>PATIENTS: </strong>Children admitted to the PICU in the period between March 1, 2012, and March 31, 2014.</p>

<p><strong>INTERVENTIONS: </strong>None.</p>

<p><strong>MEASUREMENTS AND MAIN RESULTS: </strong>During the study period, all PICU patients were prospectively screened daily for sepsis, and those meeting consensus criteria for severe sepsis or septic shock on manual chart review were entered into the sepsis registry. Of 7,459 patients admitted to the PICU during the study period, 401 met consensus criteria for severe sepsis or septic shock (reference standard cohort). Within Virtual Pediatric Systems, patients identified using "Martin" (n = 970; κ = 0.43; positive predictive value = 34%; F1 = 0.48) and "Angus" International Classification of Diseases, 9th Edition, Clinical Modification codes (n = 1387; κ = 0.28; positive predictive value = 22%; F1 = 0.34) showed limited agreement with the reference standard cohort. By comparison, explicit International Classification of Diseases, 9th Edition, Clinical Modification codes for severe sepsis (995.92) and septic shock (785.52) identified a smaller, more accurate cohort of children (n = 515; κ = 0.61; positive predictive value = 57%; F1 = 0.64). PICU mortality was 8% in the reference standard cohort and the cohort identified by explicit codes; age, illness severity scores, and resource utilization did not differ between groups. Analysis of discrepancies between the reference standard and Virtual Pediatric Systems explicit codes revealed that prospective screening missed 66 patients with severe sepsis or septic shock. After including these patients in the reference standard cohort as an exploratory analysis, agreement between the cohort of patients identified by Virtual Pediatric Systems explicit codes and the reference standard cohort improved (κ = 0.73; positive predictive value = 70%; F1 = 0.75).</p>

<p><strong>CONCLUSIONS: </strong>Children with severe sepsis and septic shock are best identified in the Virtual Pediatric Systems database using explicit diagnosis codes for severe sepsis and septic shock. The accuracy of these codes and level of clinical detail available in the Virtual Pediatric Systems database allow for sophisticated epidemiologic studies of pediatric severe sepsis and septic shock in this large, multicenter database.</p>