title: Rates of False-Positive Classification Resulting From the Analysis of Additional Embedded Performance Validity Measures.
abstract: Several studies have documented improvements in the classification accuracy of performance validity tests (PVTs) when they are combined to form aggregated models. Fewer studies have evaluated the impact of aggregating additional PVTs and changing the classification threshold within these models. A recent Monte Carlo simulation demonstrated that to maintain a false-positive rate (FPR) of ≤.10, only 1, 4, 8, 10, and 15 PVTs should be analyzed at classification thresholds of failing at least 1, at least 2, at least 3, at least 4, and at least 5 PVTs, respectively. The current study sought to evaluate these findings with embedded PVTs in a sample of real-life litigants and to highlight a potential danger in analytic flexibility with embedded PVTs. Results demonstrated that to maintain an FPR of ≤.10, only 3, 7, 10, 14, and 15 PVTs should be analyzed at classification thresholds of failing at least 1, at least 2, at least 3, at least 4, and at least 5 PVTs, respectively. Analyzing more than these numbers of PVTs resulted in a dramatic increase in the FPR. In addition, in the most extreme case, flexibility in analyzing and reporting embedded PVTs increased the FPR by 67%. Given these findings, a more objective approach to analyzing and reporting embedded PVTs should be introduced.
authors.affiliation: a Psychology , University at Albany , Albany , New York.
authors.forename: Graham M
authors.initials: GM
authors.lastname: Silk-Eglit
authors.forename: Jessica H
authors.initials: JH
authors.lastname: Stenclik
authors.forename: Andrea S
authors.initials: AS
authors.lastname: Miele
authors.forename: Julie K
authors.initials: JK
authors.lastname: Lynch
authors.forename: Robert J
authors.initials: RJ
authors.lastname: McCaffrey
mesh_terms: D000328:Adult; D003627:Data Interpretation, Statistical; D005260:Female; D006801:Humans; D008297:Male; D008306:Malingering / Q000175:diagnosis*; D008875:Middle Aged; D009010:Monte Carlo Method; D009483:Neuropsychological Tests / Q000706:statistics & numerical data*; D013647:Task Performance and Analysis
keywords: embedded measures; neuropsychology; symptom validity testing