title: Point of data saturation was assessed using resampling methods in a survey with open-ended questions.
abstract: To describe methods to determine sample sizes in surveys using open-ended questions and to assess how resampling methods can be used to determine data saturation in these surveys.
  
  
  
  We searched the literature for surveys with open-ended questions and assessed the methods used to determine sample size in 100 studies selected at random. Then, we used Monte Carlo simulations on data from a previous study on the burden of treatment to assess the probability of identifying new themes as a function of the number of patients recruited.
  
  
  
  In the literature, 85% of researchers used a convenience sample, with a median size of 167 participants (interquartile range [IQR] = 69-406). In our simulation study, the probability of identifying at least one new theme for the next included subject was 32%, 24%, and 12% after the inclusion of 30, 50, and 100 subjects, respectively. The inclusion of 150 participants at random resulted in the identification of 92% themes (IQR = 91-93%) identified in the original study.
  
  
  
  In our study, data saturation was most certainly reached for samples >150 participants. Our method may be used to determine when to continue the study to find new themes or stop because of futility.
authors: Department of General Medicine, Paris Diderot University, 16 rue Henri Huchard, Paris 75018, France; METHODS Team, Centre de Recherche en Epidémiologie et Statistiques (CRESS), INSERM U1153, 1 Place du Parvis Notre Dame, Paris 75004, France; Centre d'Épidémiologie Clinique, Hôpital Hôtel-Dieu, Assistance Publique-Hôpitaux de Paris, 1 Place du Parvis Notre Dame, Paris 75004, France. Electronic address: thi.tran-viet@htd.aphp.fr.
authors: Viet-Thi
authors: VT
authors: Tran
authors: METHODS Team, Centre de Recherche en Epidémiologie et Statistiques (CRESS), INSERM U1153, 1 Place du Parvis Notre Dame, Paris 75004, France; Centre d'Épidémiologie Clinique, Hôpital Hôtel-Dieu, Assistance Publique-Hôpitaux de Paris, 1 Place du Parvis Notre Dame, Paris 75004, France; Paris Descartes University, 12 Rue de l'École de Médecine, Paris 75006, France.
authors: Raphael
authors: R
authors: Porcher
authors: Paris Sud University, 15 Rue Georges Clemenceau, Orsay 91400, France; Centre de Recherche en Epidémiologie et Santé des populations (CESP), INSERM U1018, 16, avenue Paul Vaillant-Couturier, Villejuif 94807, France.
authors: Bruno
authors: B
authors: Falissard
authors: METHODS Team, Centre de Recherche en Epidémiologie et Statistiques (CRESS), INSERM U1153, 1 Place du Parvis Notre Dame, Paris 75004, France; Centre d'Épidémiologie Clinique, Hôpital Hôtel-Dieu, Assistance Publique-Hôpitaux de Paris, 1 Place du Parvis Notre Dame, Paris 75004, France; Paris Descartes University, 12 Rue de l'École de Médecine, Paris 75006, France; Department of Epidemiology, Columbia University Mailman School of Public Health, 116th St & Broadway, New York, NY, USA.
authors: Philippe
authors: P
authors: Ravaud
mesh_terms: D015340:Epidemiologic Research Design; D006801:Humans; D009010:Monte Carlo Method; D012494:Sampling Studies; D011795:Surveys and Questionnaires
keywords: Computer simulation; Data saturation; Qualitative research; Simulation methods; Surveys and questionnaires; Surveys with open-ended questions