title: Multilevel analysis quantifies variation in the experimental effect while optimizing power and preventing false positives.
abstract: BACKGROUND
  In neuroscience, experimental designs in which multiple measurements are collected in the same research object or treatment facility are common. Such designs result in clustered or nested data. When clusters include measurements from different experimental conditions, both the mean of the dependent variable and the effect of the experimental manipulation may vary over clusters. In practice, this type of cluster-related variation is often overlooked. Not accommodating cluster-related variation can result in inferential errors concerning the overall experimental effect.
  
  
  RESULTS
  The exact effect of ignoring the clustered nature of the data depends on the effect of clustering. Using simulation studies we show that cluster-related variation in the experimental effect, if ignored, results in a false positive rate (i.e., Type I error rate) that is appreciably higher (up to ~20-~50 %) than the chosen [Formula: see text]-level (e.g., [Formula: see text] = 0.05). If the effect of clustering is limited to the intercept, the failure to accommodate clustering can result in a loss of statistical power to detect the overall experimental effect. This effect is most pronounced when both the magnitude of the experimental effect and the sample size are small (e.g., ~25 % less power given an experimental effect with effect size d of 0.20, and a sample size of 10 clusters and 5 observations per experimental condition per cluster).
  
  
  CONCLUSIONS
  When data is collected from a research design in which observations from the same cluster are obtained in different experimental conditions, multilevel analysis should be used to analyze the data. The use of multilevel analysis not only ensures correct statistical interpretation of the overall experimental effect, but also provides a valuable test of the generalizability of the experimental effect over (intrinsically) varying settings, and a means to reveal the cause of cluster-related variation in experimental effect.
authors.affiliation: Department of Functional Genomics, Center for Neurogenomics and Cognitive Research, VU University Amsterdam, De Boelelaan 1085, 1081 HV, Amsterdam, The Netherlands. E.Aarts@vu.nl.|Department of Molecular Computational Biology, Max Planck Institute of Molecular Genetics, Ihnestraße 63-73, 14195, Berlin, Germany. E.Aarts@vu.nl.
authors.forename: Emmeke
authors.initials: E
authors.lastname: Aarts
authors.affiliation: Department of Biological Psychology, VU University Amsterdam, Van der Boechorststraat 1, 1081 BT, Amsterdam, The Netherlands. C.V.Dolan@vu.nl.
authors.forename: Conor V
authors.initials: CV
authors.lastname: Dolan
authors.affiliation: Department of Functional Genomics, Center for Neurogenomics and Cognitive Research, VU University Amsterdam, De Boelelaan 1085, 1081 HV, Amsterdam, The Netherlands. M.Verhage@vu.nl.|Department of Clinical Genetics, Section Functional Genomics, VU Medical Center Amsterdam, De Boelelaan 1085, 1081 HV, Amsterdam, The Netherlands. M.Verhage@vu.nl.
authors.forename: Matthijs
authors.initials: M
authors.lastname: Verhage
authors.affiliation: Department of Clinical Genetics, Section Complex Trait Genetics, VU Medical Center, De Boelelaan 1085, 1081 HV, Amsterdam, The Netherlands. S.vander.Sluis@vu.nl.
authors.forename: Sophie
authors.initials: S
authors.lastname: van der Sluis
mesh_terms: D000818:Animals; D003198:Computer Simulation; D003627:Data Interpretation, Statistical; D005189:False Positive Reactions; D055785:Gene Knockdown Techniques / Q000379:methods; D051379:Mice; D055361:Multilevel Analysis / Q000379:methods*; D009474:Neurons / Q000502:physiology; D009488:Neurosciences / Q000379:methods; D034741:RNA, Small Interfering; D012107:Research Design
chemical_list: D034741:RNA, Small Interfering