title: How Many Alternatives Can Be Ranked? A Comparison of the Paired Comparison and Ranking Methods.
abstract: To determine the feasibility of converting ranking data into paired comparison (PC) data and suggest the number of alternatives that can be ranked by comparing a PC and a ranking method.
  
  
  
  Using a total of 222 health states, a household survey was conducted in a sample of 300 individuals from the general population. Each respondent performed a PC 15 times and a ranking method 6 times (two attempts of ranking three, four, and five health states, respectively). The health states of the PC and the ranking method were constructed to overlap each other. We converted the ranked data into PC data and examined the consistency of the response rate. Applying probit regression, we obtained the predicted probability of each method. Pearson correlation coefficients were determined between the predicted probabilities of those methods. The mean absolute error was also assessed between the observed and the predicted values.
  
  
  
  The overall consistency of the response rate was 82.8%. The Pearson correlation coefficients were 0.789, 0.852, and 0.893 for ranking three, four, and five health states, respectively. The lowest mean absolute error was 0.082 (95% confidence interval [CI] 0.074-0.090) in ranking five health states, followed by 0.123 (95% CI 0.111-0.135) in ranking four health states and 0.126 (95% CI 0.113-0.138) in ranking three health states.
  
  
  
  After empirically examining the consistency of the response rate between a PC and a ranking method, we suggest that using five alternatives in the ranking method may be superior to using three or four alternatives.
authors: Department of Preventive Medicine, University of Ulsan College of Medicine, Seoul, South Korea.
authors: Minsu
authors: M
authors: Ock
authors: Vital Statistics Division, Statistics Korea, Daejeon, South Korea.
authors: Nari
authors: N
authors: Yi
authors: Department of Health Convergence, Ewha Womans University, Seoul, South Korea.
authors: Jeonghoon
authors: J
authors: Ahn
authors: Department of Preventive Medicine, University of Ulsan College of Medicine, Seoul, South Korea. Electronic address: mdjominwoo@gmail.com.
authors: Min-Woo
authors: MW
authors: Jo
mesh_terms: D000328:Adult; D002755:Choice Behavior; D005240:Feasibility Studies; D005260:Female; D006304:Health Status; D006801:Humans; D008297:Male; D016555:Matched-Pair Analysis; D008875:Middle Aged; D015233:Models, Statistical; D057240:Patient Preference; D011795:Surveys and Questionnaires; D055815:Young Adult
keywords: consistency; discrete choice experiments; paired comparison; ranking