title: How many clusters to report: a recursive heuristic.
abstract: Clustering can be a valuable tool for analyzing large amounts of data, but anyone who clusters must choose how many item clusters, K, to report. Unfortunately, one must guess at K or some related parameter when working within each of the three available frameworks where one thinks of clustering: as a Euclidean distance problem; as a statistical model problem; or as a complexity theory problem. We report here a novel recursive square root heuristic, RSQRT, which accurately predicts K(reported) as a function of the attribute or item count, depending on attribute scales. We tested the heuristic on 226 widely-varying, but mostly scientific, studies, and found that the heuristic's K(best-predicted) rounded to exactly K(reported) in over half of the studies and was close in almost all of them. We claim that this strongly-supported heuristic makes sense and that, although it is not prescriptive, using it prospectively is much better than guessing.
authors.affiliation: Computer Science and Engineering Department, University of Minnesota, Minneapolis, MN 55455, USA. carlis@cs.umn.edu
authors.forename: John
authors.initials: J
authors.lastname: Carlis
authors.forename: Kelsey
authors.initials: K
authors.lastname: Bruso
mesh_terms: D000465:Algorithms; D016000:Cluster Analysis; D003627:Data Interpretation, Statistical; D006801:Humans; D015994:Incidence; D016016:Proportional Hazards Models; D018570:Risk Assessment / Q000379:methods; D012307:Risk Factors; D012559:Schizophrenia / Q000453:epidemiology*