title: Establishing meaningful cut points for online user ratings.
abstract: Subjective perceptions of websites can be reliably measured with questionnaires. But it is unclear how such scores should be interpreted in practice, e.g. is an aesthetics score of 4 points on a seven-point-scale satisfactory? The current paper introduces a receiver-operating characteristic (ROC)-based methodology to establish meaningful cut points for the VisAWI (visual aesthetics of websites inventory) and its short form the VisAWI-S. In two studies we use users' global ratings (UGRs) and website rankings as anchors. A total of 972 participants took part in the studies which yielded similar results. First, one-item UGRs correlate highly with the VisAWI. Second, cut points on the VisAWI reliably differentiate between sites that are perceived as attractive versus unattractive. Third, these cut points are variable, but only within a certain range. Together the research presented here establishes a score of 4.5 on the VisAWI which is a reasonable goal for website designers and highlights the utility of the ROC methodology to derive relevant scores for rating scales.
authors.affiliation: a German Paediatric Pain Centre , Children's Hospital Datteln , Dr.-Friedrich-Steiner Str. 5, 45711 Datteln , Germany.
authors.forename: Gerrit
authors.initials: G
authors.lastname: Hirschfeld
authors.forename: Meinald T
authors.initials: MT
authors.lastname: Thielsch
mesh_terms: D000293:Adolescent; D000328:Adult; D000368:Aged; D000369:Aged, 80 and over; D003258:Consumer Behavior; D004954:Esthetics; D005260:Female; D006801:Humans; D020407:Internet / Q000592:standards*; D008297:Male; D008875:Middle Aged; D012372:ROC Curve; D012015:Reference Standards; D015203:Reproducibility of Results; D011795:Surveys and Questionnaires; D014584:User-Computer Interface; D055815:Young Adult
keywords: VisAWI; aesthetics; global ratings; website evaluation; first impression