Use of Traumatic Brain Injury Prediction Rules With Clinical Decision Support.

2017

2017 Mar 24

1098-4275

<p><strong>OBJECTIVES: </strong>We determined whether implementing the Pediatric Emergency Care Applied Research Network (PECARN) traumatic brain injury (TBI) prediction rules and providing risks of clinically important TBIs (ciTBIs) with computerized clinical decision support (CDS) reduces computed tomography (CT) use for children with minor head trauma.</p>

<p><strong>METHODS: </strong>Nonrandomized trial with concurrent controls at 5 pediatric emergency departments (PEDs) and 8 general EDs (GEDs) between November 2011 and June 2014. Patients were &lt;18 years old with minor blunt head trauma. Intervention sites received CDS with CT recommendations and risks of ciTBI, both for patients at very low risk of ciTBI (no Pediatric Emergency Care Applied Research Network rule factors) and those not at very low risk. The primary outcome was the rate of CT, analyzed by site, controlling for time trend.</p>

<p><strong>RESULTS: </strong>We analyzed 16 635 intervention and 2394 control patients. Adjusted for time trends, CT rates decreased significantly (P &lt; .05) but modestly (2.3%-3.7%) at 2 of 4 intervention PEDs for children at very low risk. The other 2 PEDs had small (0.8%-1.5%) nonsignificant decreases. CT rates did not decrease consistently at the intervention GEDs, with low baseline CT rates (2.1%-4.0%) in those at very low risk. The control PED had little change in CT use in similar children (from 1.6% to 2.9%); the control GED showed a decrease in the CT rate (from 7.1% to 2.6%). For all children with minor head trauma, intervention sites had small decreases in CT rates (1.7%-6.2%).</p>

<p><strong>CONCLUSIONS: </strong>The implementation of TBI prediction rules and provision of risks of ciTBIs by using CDS was associated with modest, safe, but variable decreases in CT use. However, some secular trends were also noted.</p>

10.1542/peds.2016-2709

Pediatrics

28341799

Identification of Long Bone Fractures in Radiology Reports Using Natural Language Processing to support Healthcare Quality Improvement.

2016

1051-1068

2016 Nov 09

1869-0327

<p><strong>BACKGROUND: </strong>Important information to support healthcare quality improvement is often recorded in free text documents such as radiology reports. Natural language processing (NLP) methods may help extract this information, but these methods have rarely been applied outside the research laboratories where they were developed.</p>

<p><strong>OBJECTIVE: </strong>To implement and validate NLP tools to identify long bone fractures for pediatric emergency medicine quality improvement.</p>

<p><strong>METHODS: </strong>Using freely available statistical software packages, we implemented NLP methods to identify long bone fractures from radiology reports. A sample of 1,000 radiology reports was used to construct three candidate classification models. A test set of 500 reports was used to validate the model performance. Blinded manual review of radiology reports by two independent physicians provided the reference standard. Each radiology report was segmented and word stem and bigram features were constructed. Common English "stop words" and rare features were excluded. We used 10-fold cross-validation to select optimal configuration parameters for each model. Accuracy, recall, precision and the F1 score were calculated. The final model was compared to the use of diagnosis codes for the identification of patients with long bone fractures.</p>

<p><strong>RESULTS: </strong>There were 329 unique word stems and 344 bigrams in the training documents. A support vector machine classifier with Gaussian kernel performed best on the test set with accuracy=0.958, recall=0.969, precision=0.940, and F1 score=0.954. Optimal parameters for this model were cost=4 and gamma=0.005. The three classification models that we tested all performed better than diagnosis codes in terms of accuracy, precision, and F1 score (diagnosis code accuracy=0.932, recall=0.960, precision=0.896, and F1 score=0.927).</p>

<p><strong>CONCLUSIONS: </strong>NLP methods using a corpus of 1,000 training documents accurately identified acute long bone fractures from radiology reports. Strategic use of straightforward NLP methods, implemented with freely available software, offers quality improvement teams new opportunities to extract information from narrative documents.</p>

10.4338/ACI-2016-08-RA-0129

Appl Clin Inform

27826610

Use of a remote clinical decision support service for a multicenter trial to implement prediction rules for children with minor blunt head trauma.

2016

101-10

2016 Mar

1872-8243

<p><strong>OBJECTIVE: </strong>To evaluate the architecture, integration requirements, and execution characteristics of a remote clinical decision support (CDS) service used in a multicenter clinical trial. The trial tested the efficacy of implementing brain injury prediction rules for children with minor blunt head trauma.</p>

<p><strong>MATERIALS AND METHODS: </strong>We integrated the Epic(®) electronic health record (EHR) with the Enterprise Clinical Rules Service (ECRS), a web-based CDS service, at two emergency departments. Patterns of CDS review included either a delayed, near-real-time review, where the physician viewed CDS recommendations generated by the nursing assessment, or a real-time review, where the physician viewed recommendations generated by their own documentation. A backstopping, vendor-based CDS triggered with zero delay when no recommendation was available in the EHR from the web-service. We assessed the execution characteristics of the integrated system and the source of the generated recommendations viewed by physicians.</p>

<p><strong>RESULTS: </strong>The ECRS mean execution time was 0.74 ±0.72 s. Overall execution time was substantially different at the two sites, with mean total transaction times of 19.67 and 3.99 s. Of 1930 analyzed transactions from the two sites, 60% (310/521) of all physician documentation-initiated recommendations and 99% (1390/1409) of all nurse documentation-initiated recommendations originated from the remote web service.</p>

<p><strong>DISCUSSION: </strong>The remote CDS system was the source of recommendations in more than half of the real-time cases and virtually all the near-real-time cases. Comparisons are limited by allowable variation in user workflow and resolution of the EHR clock.</p>

<p><strong>CONCLUSION: </strong>With maturation and adoption of standards for CDS services, remote CDS shows promise to decrease time-to-trial for multicenter evaluations of candidate decision support interventions.</p>

10.1016/j.ijmedinf.2015.12.002

Int J Med Inform

26806717

Development, Evaluation and Implementation of Chief Complaint Groupings to Activate Data Collection: A Multi-Center Study of Clinical Decision Support for Children with Head Trauma.

2015

521-35

2015

1869-0327

<p><strong>BACKGROUND: </strong>Overuse of cranial computed tomography scans in children with blunt head trauma unnecessarily exposes them to radiation. The Pediatric Emergency Care Applied Research Network (PECARN) blunt head trauma prediction rules identify children who do not require a computed tomography scan. Electronic health record (EHR) based clinical decision support (CDS) may effectively implement these rules but must only be provided for appropriate patients in order to minimize excessive alerts.</p>

<p><strong>OBJECTIVES: </strong>To develop, implement and evaluate site-specific groupings of chief complaints (CC) that accurately identify children with head trauma, in order to activate data collection in an EHR.</p>

<p><strong>METHODS: </strong>As part of a 13 site clinical trial comparing cranial computed tomography use before and after implementation of CDS, four PECARN sites centrally developed and locally implemented CC groupings to trigger a clinical trial alert (CTA) to facilitate the completion of an emergency department head trauma data collection template. We tested and chose CC groupings to attain high sensitivity while maintaining at least moderate specificity.</p>

<p><strong>RESULTS: </strong>Due to variability in CCs available, identical groupings across sites were not possible. We noted substantial variability in the sensitivity and specificity of seemingly similar CC groupings between sites. The implemented CC groupings had sensitivities greater than 90% with specificities between 75-89%. During the trial, formal testing and provider feedback led to tailoring of the CC groupings at some sites.</p>

<p><strong>CONCLUSIONS: </strong>CC groupings can be successfully developed and implemented across multiple sites to accurately identify patients who should have a CTA triggered to facilitate EHR data collection. However, CC groupings will necessarily vary in order to attain high sensitivity and moderate-to-high specificity. In future trials, the balance between sensitivity and specificity should be considered based on the nature of the clinical condition, including prevalence and morbidity, in addition to the goals of the intervention being considered.</p>

10.4338/ACI-2015-02-RA-0019

Appl Clin Inform

26448796

Clinical Decision Support for a Multicenter Trial of Pediatric Head Trauma: Development, Implementation, and Lessons Learned.

2016

534-42

2016

1869-0327

<p><strong>INTRODUCTION: </strong>For children who present to emergency departments (EDs) due to blunt head trauma, ED clinicians must decide who requires computed tomography (CT) scanning to evaluate for traumatic brain injury (TBI). The Pediatric Emergency Care Applied Research Network (PECARN) derived and validated two age-based prediction rules to identify children at very low risk of clinically-important traumatic brain injuries (ciTBIs) who do not typically require CT scans. In this case report, we describe the strategy used to implement the PECARN TBI prediction rules via electronic health record (EHR) clinical decision support (CDS) as the intervention in a multicenter clinical trial.</p>

<p><strong>METHODS: </strong>Thirteen EDs participated in this trial. The 10 sites receiving the CDS intervention used the Epic(®) EHR. All sites implementing EHR-based CDS built the rules by using the vendor's CDS engine. Based on a sociotechnical analysis, we designed the CDS so that recommendations could be displayed immediately after any provider entered prediction rule data. One central site developed and tested the intervention package to be exported to other sites. The intervention package included a clinical trial alert, an electronic data collection form, the CDS rules and the format for recommendations.</p>

<p><strong>RESULTS: </strong>The original PECARN head trauma prediction rules were derived from physician documentation while this pragmatic trial led each site to customize their workflows and allow multiple different providers to complete the head trauma assessments. These differences in workflows led to varying completion rates across sites as well as differences in the types of providers completing the electronic data form. Site variation in internal change management processes made it challenging to maintain the same rigor across all sites. This led to downstream effects when data reports were developed.</p>

<p><strong>CONCLUSIONS: </strong>The process of a centralized build and export of a CDS system in one commercial EHR system successfully supported a multicenter clinical trial.</p>

10.4338/ACI-2015-10-CR-0144

Appl Clin Inform

27437059