title: Determining thresholds using adaptive procedures and psychometric fits: evaluating efficiency using theory, simulations, and human experiments.
abstract: When measuring thresholds, careful selection of stimulus amplitude can increase efficiency by increasing the precision of psychometric fit parameters (e.g., decreasing the fit parameter error bars). To find efficient adaptive algorithms for psychometric threshold ("sigma") estimation, we combined analytic approaches, Monte Carlo simulations, and human experiments for a one-interval, binary forced-choice, direction-recognition task. To our knowledge, this is the first time analytic results have been combined and compared with either simulation or human results. Human performance was consistent with theory and not significantly different from simulation predictions. Our analytic approach provides a bound on efficiency, which we compared against the efficiency of standard staircase algorithms, a modified staircase algorithm with asymmetric step sizes, and a maximum likelihood estimation (MLE) procedure. Simulation results suggest that optimal efficiency at determining threshold is provided by the MLE procedure targeting a fraction correct level of 0.92, an asymmetric 4-down, 1-up staircase targeting between 0.86 and 0.92 or a standard 6-down, 1-up staircase. Psychometric test efficiency, computed by comparing simulation and analytic results, was between 41 and 58% for 50 trials for these three algorithms, reaching up to 84% for 200 trials. These approaches were 13-21% more efficient than the commonly used 3-down, 1-up symmetric staircase. We also applied recent advances to reduce accuracy errors using a bias-reduced fitting approach. Taken together, the results lend confidence that the assumptions underlying each approach are reasonable and that human threshold forced-choice decision making is modeled well by detection theory models and mimics simulations based on detection theory models.
authors.affiliation: Jenks Vestibular Physiology Lab, Massachusetts Eye and Ear Infirmary, 243 Charles St., Boston, MA, 02114, USA. faisal_karmali@meei.harvard.edu.|Department of Otology and Laryngology, Harvard Medical School, Boston, MA, USA. faisal_karmali@meei.harvard.edu.
authors.forename: Faisal
authors.initials: F
authors.lastname: Karmali
authors.affiliation: Jenks Vestibular Physiology Lab, Massachusetts Eye and Ear Infirmary, 243 Charles St., Boston, MA, 02114, USA.|Department of Electrical Engineering and Computer Science, MIT, Cambridge, MA, USA.
authors.forename: Shomesh E
authors.initials: SE
authors.lastname: Chaudhuri
authors.affiliation: Jenks Vestibular Physiology Lab, Massachusetts Eye and Ear Infirmary, 243 Charles St., Boston, MA, 02114, USA.|Department of Otology and Laryngology, Harvard Medical School, Boston, MA, USA.
authors.forename: Yongwoo
authors.initials: Y
authors.lastname: Yi
authors.affiliation: Jenks Vestibular Physiology Lab, Massachusetts Eye and Ear Infirmary, 243 Charles St., Boston, MA, 02114, USA.|Department of Otology and Laryngology, Harvard Medical School, Boston, MA, USA.
authors.forename: Daniel M
authors.initials: DM
authors.lastname: Merfeld
mesh_terms: D000328:Adult; D003198:Computer Simulation; D005260:Female; D006801:Humans; D016013:Likelihood Functions; D008297:Male; D008875:Middle Aged; D011594:Psychometrics; D011597:Psychomotor Performance / Q000502:physiology*; D012684:Sensory Thresholds / Q000502:physiology*
keywords: Efficiency; Precision; Psychometric curve; Psychophysics