A design of experiments approach to validation sampling for logistic regression modeling with error-prone medical records.
BACKGROUND AND OBJECTIVE
Electronic medical record (EMR) databases offer significant potential for developing clinical hypotheses and identifying disease risk associations by fitting statistical models that capture the relationship between a binary response variable and a set of predictor variables that represent clinical, phenotypical, and demographic data for the patient. However, EMR response data may be error prone for a variety of reasons. Performing a manual chart review to validate data accuracy is time consuming, which limits the number of chart reviews in a large database. The authors' objective is to develop a new design-of-experiments-based systematic chart validation and review (DSCVR) approach that is more powerful than the random validation sampling used in existing approaches.


METHODS
The DSCVR approach judiciously and efficiently selects the cases to validate (i.e., validate whether the response values are correct for those cases) for maximum information content, based only on their predictor variable values. The final predictive model will be fit using only the validation sample, ignoring the remainder of the unvalidated and unreliable error-prone data. A Fisher information based D-optimality criterion is used, and an algorithm for optimizing it is developed.


RESULTS
The authors' method is tested in a simulation comparison that is based on a sudden cardiac arrest case study with 23 041 patients' records. This DSCVR approach, using the Fisher information based D-optimality criterion, results in a fitted model with much better predictive performance, as measured by the receiver operating characteristic curve and the accuracy in predicting whether a patient will experience the event, than a model fitted using a random validation sample.


CONCLUSIONS
The simulation comparisons demonstrate that this DSCVR approach can produce predictive models that are significantly better than those produced from random validation sampling, especially when the event rate is low.
['Ouyang|Liwen|L|', 'Apley|Daniel W|DW|', 'Mehrotra|Sanjay|S|']
D000465:Algorithms D019540:Area Under Curve D003198:Computer Simulation D016757:Death, Sudden, Cardiac D057286:Electronic Health Records D006801:Humans D016015:Logistic Models D054928:Validation Studies as Topic
design of experiments validation sampling logistic regression electronic medical records sudden cardiac arrest