title: Assessing the likelihood of an important clinical outcome: new insights from a comparison of clinical and actuarial judgment.
abstract: OBJECTIVE
  To assess and rank the performance of different methods of predicting the probability of death following a specified surgical procedure.
  
  
  METHODS
  Actuarial estimates of the probability of early mortality for 40 patients were derived from 2 sources: a large published surgical series and a smaller series from the center where surgery was performed. Surgeons and cardiologists also provided probability estimates for these patients.
  
  
  RESULTS
  Estimates derived from the published literature were too optimistic and did not differentiate between patients more, or less, likely to die (i.e., failed to discriminate). Doctors' judgments were unbiased but failed to discriminate. Local actuarial estimates (influenced by only 1 or 2 variables) were unbiased, did discriminate, but exhibited more random variation.
  
  
  CONCLUSIONS
  The preferred source of estimates depends upon which aspect of accuracy is of greatest importance. Differences in patient selection and error in the identification of risk factors mean that published results will not always appropriately predict surgical risk at other institutions. Risk stratification may be more robust when based on a small set of cross-validated predictors rather than a larger set of predictors that includes some whose reliability has not been confirmed.
authors.affiliation: Department of Psychology, University of Essex, Colchester, UK. timrakow@essex.ac.uk
authors.forename: Tim
authors.initials: T
authors.lastname: Rakow
authors.forename: Charles
authors.initials: C
authors.lastname: Vincent
authors.forename: Kate
authors.initials: K
authors.lastname: Bull
authors.forename: Nigel
authors.initials: N
authors.lastname: Harvey
mesh_terms: D000206:Actuarial Analysis; D000367:Age Factors; D002675:Child, Preschool; D003661:Decision Support Techniques; D018729:Fontan Procedure / Q000401:mortality*; D006330:Heart Defects, Congenital / Q000601:surgery*; D006352:Heart Ventricles / Q000002:abnormalities / Q000601:surgery*; D017052:Hospital Mortality; D006801:Humans; D007223:Infant; D007231:Infant, Newborn; D007600:Judgment; D016015:Logistic Models; D015999:Multivariate Analysis; D018579:Patient Selection; D011379:Prognosis; D016016:Proportional Hazards Models; D015203:Reproducibility of Results; D018570:Risk Assessment / Q000379:methods / Q000706:statistics & numerical data*; D012307:Risk Factors; D016019:Survival Analysis; D016896:Treatment Outcome