title: Interformat reliability of digital psychiatric self-report questionnaires: a systematic review.
abstract: BACKGROUND
  Research on Internet-based interventions typically use digital versions of pen and paper self-report symptom scales. However, adaptation into the digital format could affect the psychometric properties of established self-report scales. Several studies have investigated differences between digital and pen and paper versions of instruments, but no systematic review of the results has yet been done.
  
  
  OBJECTIVE
  This review aims to assess the interformat reliability of self-report symptom scales used in digital or online psychotherapy research.
  
  
  METHODS
  Three databases (MEDLINE, Embase, and PsycINFO) were systematically reviewed for studies investigating the reliability between digital and pen and paper versions of psychiatric symptom scales.
  
  
  RESULTS
  From a total of 1504 publications, 33 were included in the review, and interformat reliability of 40 different symptom scales was assessed. Significant differences in mean total scores between formats were found in 10 of 62 analyses. These differences were found in just a few studies, which indicates that the results were due to study effects and sample effects rather than unreliable instruments. The interformat reliability ranged from r=.35 to r=.99; however, the majority of instruments showed a strong correlation between format scores. The quality of the included studies varied, and several studies had insufficient power to detect small differences between formats.
  
  
  CONCLUSIONS
  When digital versions of self-report symptom scales are compared to pen and paper versions, most scales show high interformat reliability. This supports the reliability of results obtained in psychotherapy research on the Internet and the comparability of the results to traditional psychotherapy research. There are, however, some instruments that consistently show low interformat reliability, suggesting that these conclusions cannot be generalized to all questionnaires. Most studies had at least some methodological issues with insufficient statistical power being the most common issue. Future studies should preferably provide information about the transformation of the instrument into digital format and the procedure for data collection in more detail.
authors.affiliation: U-CARE, Department of Public Health and Caring Sciences, Uppsala University, Uppsala, Sweden. sven.alfonsson@pubcare.uu.se.
authors.forename: Sven
authors.identifier: http://orcid.org/0000-0003-4570-5891
authors.initials: S
authors.lastname: Alfonsson
authors.forename: Pernilla
authors.identifier: http://orcid.org/0000-0003-4118-1613
authors.initials: P
authors.lastname: Maathz
authors.forename: Timo
authors.identifier: http://orcid.org/0000-0003-1990-5785
authors.initials: T
authors.lastname: Hursti
mesh_terms: D006801:Humans; D020407:Internet; D001523:Mental Disorders / Q000628:therapy*; D010209:Paper; D011613:Psychotherapy / Q000379:methods*; D015203:Reproducibility of Results; D057566:Self Report; D011795:Surveys and Questionnaires
keywords: Internet; computer; psychometric; psychotherapy; questionnaire; reliability