title: Using ROC curves to choose minimally important change thresholds when sensitivity and specificity are valued equally: the forgotten lesson of pythagoras. theoretical considerations and an example application of change in health status.
abstract: BACKGROUND
  Receiver Operator Characteristic (ROC) curves are being used to identify Minimally Important Change (MIC) thresholds on scales that measure a change in health status. In quasi-continuous patient reported outcome measures, such as those that measure changes in chronic diseases with variable clinical trajectories, sensitivity and specificity are often valued equally. Notwithstanding methodologists agreeing that these should be valued equally, different approaches have been taken to estimating MIC thresholds using ROC curves.
  
  
  OBJECTIVE
  We aimed to compare the different approaches used with a new approach, exploring the extent to which the methods choose different thresholds, and considering the effect of differences on conclusions in responder analyses.
  
  
  METHODS
  Using graphical methods, hypothetical data, and data from a large randomised controlled trial of manual therapy for low back pain, we compared two existing approaches with a new approach that is based on the addition of the sums of squares of 1-sensitivity and 1-specificity.
  
  
  RESULTS
  There can be divergence in the thresholds chosen by different estimators. The cut-point selected by different estimators is dependent on the relationship between the cut-points in ROC space and the different contours described by the estimators. In particular, asymmetry and the number of possible cut-points affects threshold selection.
  
  
  CONCLUSIONS
  Choice of MIC estimator is important. Different methods for choosing cut-points can lead to materially different MIC thresholds and thus affect results of responder analyses and trial conclusions. An estimator based on the smallest sum of squares of 1-sensitivity and 1-specificity is preferable when sensitivity and specificity are valued equally. Unlike other methods currently in use, the cut-point chosen by the sum of squares method always and efficiently chooses the cut-point closest to the top-left corner of ROC space, regardless of the shape of the ROC curve.
authors.affiliation: Clinical Trials Unit, Warwick Medical School, University of Warwick, Gibbet Hill Road, Coventry, United Kingdom; Norge Helsehøyskole, Campus Kristiania, Prinsens Gate 7-9, Oslo, Norway.
authors.forename: Robert
authors.initials: R
authors.lastname: Froud
authors.affiliation: Cambridge Centre for Health Services Research, University of Cambridge, Robinson Way Cambridge, Cambridgeshire, United Kingdom.
authors.forename: Gary
authors.initials: G
authors.lastname: Abel
mesh_terms: D003627:Data Interpretation, Statistical; D006304:Health Status; D006801:Humans; D017116:Low Back Pain / Q000175:diagnosis* / Q000628:therapy; D015337:Multicenter Studies as Topic; D012372:ROC Curve; D016032:Randomized Controlled Trials as Topic; D016896:Treatment Outcome