title: Development of the Patient Education Materials Assessment Tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information.
abstract: OBJECTIVE
  To develop a reliable and valid instrument to assess the understandability and actionability of print and audiovisual materials.
  
  
  METHODS
  We compiled items from existing instruments/guides that the expert panel assessed for face/content validity. We completed four rounds of reliability testing, and produced evidence of construct validity with consumers and readability assessments.
  
  
  RESULTS
  The experts deemed the PEMAT items face/content valid. Four rounds of reliability testing and refinement were conducted using raters untrained on the PEMAT. Agreement improved across rounds. The final PEMAT showed moderate agreement per Kappa (Average K=0.57) and strong agreement per Gwet's AC1 (Average=0.74). Internal consistency was strong (α=0.71; Average Item-Total Correlation=0.62). For construct validation with consumers (n=47), we found significant differences between actionable and poorly-actionable materials in comprehension scores (76% vs. 63%, p<0.05) and ratings (8.9 vs. 7.7, p<0.05). For understandability, there was a significant difference for only one of two topics on consumer numeric scores. For actionability, there were significant positive correlations between PEMAT scores and consumer-testing results, but no relationship for understandability. There were, however, strong, negative correlations between grade-level and both consumer-testing results and PEMAT scores.
  
  
  CONCLUSIONS
  The PEMAT demonstrated strong internal consistency, reliability, and evidence of construct validity.
  
  
  CONCLUSIONS
  The PEMAT can help professionals judge the quality of materials (available at: http://www.ahrq.gov/pemat).
authors: Health Policy, Abt Associates, Inc., Cambridge, USA. Electronic address: sarah_shoemaker@abtassoc.com.
authors: Sarah J
authors: SJ
authors: Shoemaker
authors: Feinberg School of Medicine, Northwestern University, Chicago, USA.
authors: Michael S
authors: MS
authors: Wolf
authors: Agency for Healthcare Research and Quality (AHRQ), Rockville, USA.
authors: Cindy
authors: C
authors: Brach
mesh_terms: D001296:Audiovisual Aids; D003142:Communication; D003256:Community Participation; D032882:Comprehension; D057220:Health Literacy / Q000592:standards*; D006801:Humans; D008402:Mass Media; D010353:Patient Education as Topic / Q000592:standards*; D015203:Reproducibility of Results; D013664:Teaching Materials / Q000592:standards*
keywords: Assessment; Audiovisual materials; Clear communication; Educational materials; Health literacy; Instrument development; Measurement; Patient education; Plain language; Readability