title: Confidence intervals for single-case effect size measures based on randomization test inversion.
abstract: In the current paper, we present a method to construct nonparametric confidence intervals (CIs) for single-case effect size measures in the context of various single-case designs. We use the relationship between a two-sided statistical hypothesis test at significance level α and a 100 (1 - α) % two-sided CI to construct CIs for any effect size measure θ that contain all point null hypothesis θ values that cannot be rejected by the hypothesis test at significance level α. This method of hypothesis test inversion (HTI) can be employed using a randomization test as the statistical hypothesis test in order to construct a nonparametric CI for θ. We will refer to this procedure as randomization test inversion (RTI). We illustrate RTI in a situation in which θ is the unstandardized and the standardized difference in means between two treatments in a completely randomized single-case design. Additionally, we demonstrate how RTI can be extended to other types of single-case designs. Finally, we discuss a few challenges for RTI as well as possibilities when using the method with other effect size measures, such as rank-based nonoverlap indices. Supplementary to this paper, we provide easy-to-use R code, which allows the user to construct nonparametric CIs according to the proposed method.
authors.affiliation: Faculty of Psychology and Educational Sciences, KU Leuven - University of Leuven, Leuven, Belgium. Bart.Michiels@ppw.kuleuven.be.
authors.forename: Bart
authors.initials: B
authors.lastname: Michiels
authors.affiliation: Faculty of Psychology and Educational Sciences, KU Leuven - University of Leuven, Leuven, Belgium.
authors.forename: Mieke
authors.initials: M
authors.lastname: Heyvaert
authors.affiliation: Faculty of Psychology and Educational Sciences, KU Leuven - University of Leuven, Leuven, Belgium.
authors.forename: Ann
authors.initials: A
authors.lastname: Meulders
authors.affiliation: Faculty of Psychology and Educational Sciences, KU Leuven - University of Leuven, Leuven, Belgium.
authors.forename: Patrick
authors.initials: P
authors.lastname: Onghena
mesh_terms: D016001:Confidence Intervals; D003627:Data Interpretation, Statistical; D006801:Humans; D011897:Random Allocation; D018401:Sample Size; D018709:Statistics, Nonparametric
keywords: Confidence intervals; Effect size; Hypothesis testing; Nonparametric statistics; Randomization tests; Single-case experiments