Information retrieval performance of probabilistically generated, problem-specific computerized provider order entry pick-lists: a pilot study.
OBJECTIVE
The aim of this study was to preliminarily determine the feasibility of probabilistically generating problem-specific computerized provider order entry (CPOE) pick-lists from a database of explicitly linked orders and problems from actual clinical cases.


METHODS
In a pilot retrospective validation, physicians reviewed internal medicine cases consisting of the admission history and physical examination and orders placed using CPOE during the first 24 hours after admission. They created coded problem lists and linked orders from individual cases to the problem for which they were most indicated. Problem-specific order pick-lists were generated by including a given order in a pick-list if the probability of linkage of order and problem (PLOP) equaled or exceeded a specified threshold. PLOP for a given linked order-problem pair was computed as its prevalence among the other cases in the experiment with the given problem. The orders that the reviewer linked to a given problem instance served as the reference standard to evaluate its system-generated pick-list.


METHODS
Recall, precision, and length of the pick-lists.


RESULTS
Average recall reached a maximum of .67 with a precision of .17 and pick-list length of 31.22 at a PLOP threshold of 0. Average precision reached a maximum of .73 with a recall of .09 and pick-list length of .42 at a PLOP threshold of .9. Recall varied inversely with precision in classic information retrieval behavior.


CONCLUSIONS
We preliminarily conclude that it is feasible to generate problem-specific CPOE pick-lists probabilistically from a database of explicitly linked orders and problems. Further research is necessary to determine the usefulness of this approach in real-world settings.
Rothschild
Adam S
AS
Department of Biomedical Informatics, Columbia University, Vanderbilt Clinic 5th Floor, 622 West 168th Street, New York, NY 10032, USA. adam.rothschild@dbmi.columbia.edu
Lehmann
Harold P
HP
D006801:Humans; D016247:Information Storage and Retrieval; D007388:Internal Medicine; D016347:Medical Records Systems, Computerized; D008501:Medical Records, Problem-Oriented; D010865:Pilot Projects; D011336:Probability; D012189:Retrospective Studies; D014584:User-Computer Interface