title: Intelligent Monitoring? Assessing the ability of the Care Quality Commission's statistical surveillance tool to predict quality and prioritise NHS hospital inspections.
abstract: The Care Quality Commission (CQC) is responsible for ensuring the quality of the health and social care delivered by more than 30 000 registered providers in England. With only limited resources for conducting on-site inspections, the CQC has used statistical surveillance tools to help it identify which providers it should prioritise for inspection. In the face of planned funding cuts, the CQC plans to put more reliance on statistical surveillance tools to assess risks to quality and prioritise inspections accordingly.
  
  
  
  To evaluate the ability of the CQC's latest surveillance tool, Intelligent Monitoring (IM), to predict the quality of care provided by National Health Service (NHS) hospital trusts so that those at greatest risk of providing poor-quality care can be identified and targeted for inspection.
  
  
  
  The predictive ability of the IM tool is evaluated through regression analyses and χ2testing of the relationship between the quantitative risk score generated by the IM tool and the subsequent quality rating awarded following detailed on-site inspection by large expert teams of inspectors.
  
  
  
  First, the continuous risk scores generated by the CQC's IM statistical surveillance tool cannot predict inspection-based quality ratings of NHS hospital trusts (OR 0.38 (0.14 to 1.05) for Outstanding/Good, OR 0.94 (0.80 to -1.10) for Good/Requires improvement, and OR 0.90 (0.76 to 1.07) for Requires improvement/Inadequate). Second, the risk scores cannot be used more simply to distinguish the trusts performing poorly-those subsequently rated either 'Requires improvement' or 'Inadequate'-from the trusts performing well-those subsequently rated either 'Good' or 'Outstanding' (OR 1.07 (0.91 to 1.26)). Classifying CQC's risk bandings 1-3 as high risk and 4-6 as low risk, 11 of the high risk trusts were performing well and 43 of the low risk trusts were performing poorly, resulting in an overall accuracy rate of 47.6%. Third, the risk scores cannot be used even more simply to distinguish the worst performing trusts-those subsequently rated 'Inadequate'-from the remaining, better performing trusts (OR 1.11 (0.94 to 1.32)). Classifying CQC's risk banding 1 as high risk and 2-6 as low risk, the highest overall accuracy rate of 72.8% was achieved, but still only 6 of the 13 Inadequate trusts were correctly classified as being high risk.
  
  
  
  Since the IM statistical surveillance tool cannot predict the outcome of NHS hospital trust inspections, it cannot be used for prioritisation. A new approach to inspection planning is therefore required.
authors.affiliation: School of Management & Business, King's College London, London, UK.
authors.forename: Alex
authors.initials: A
authors.lastname: Griffiths
authors.affiliation: Department of Geography, King's College London, London, UK.
authors.forename: Anne-Laure
authors.initials: AL
authors.lastname: Beaussier
authors.affiliation: Department of Geography, King's College London, London, UK.
authors.forename: David
authors.initials: D
authors.lastname: Demeritt
authors.affiliation: Department of Geography, King's College London, London, UK.
authors.forename: Henry
authors.initials: H
authors.lastname: Rothstein
mesh_terms: D026683:Advisory Committees; D004739:England; D006783:Hospitals, State; D006801:Humans; D019984:Quality Indicators, Health Care / Q000706:statistics & numerical data; D011787:Quality of Health Care / Q000592:standards*; D013222:State Medicine
keywords: Health policy; Performance measures; Quality improvement methodologies; Quality measurement; Risk management