title: Experiments in computing: a survey.
abstract: Experiments play a central role in science. The role of experiments in computing is, however, unclear. Questions about the relevance of experiments in computing attracted little attention until the 1980s. As the discipline then saw a push towards experimental computer science, a variety of technically, theoretically, and empirically oriented views on experiments emerged. As a consequence of those debates, today's computing fields use experiments and experiment terminology in a variety of ways. This paper analyzes experimentation debates in computing. It presents five ways in which debaters have conceptualized experiments in computing: feasibility experiment, trial experiment, field experiment, comparison experiment, and controlled experiment. This paper has three aims: to clarify experiment terminology in computing; to contribute to disciplinary self-understanding of computing; and, due to computing's centrality in other fields, to promote understanding of experiments in modern science in general.
authors.affiliation: Department of Computer and Systems Sciences, Stockholm University, 16440 Kista, Sweden.
authors.forename: Matti
authors.identifier: 0000-0003-1037-3313
authors.initials: M
authors.lastname: Tedre
authors.affiliation: Faculty of Behavioural Sciences, University of Helsinki, 00014 Helsinki, Finland.
authors.forename: Nella
authors.initials: N
authors.lastname: Moisseinen
mesh_terms: D003205:Computing Methodologies; D003625:Data Collection; D012106:Research; D012107:Research Design