title: Measuring and Specifying Combinatorial Coverage of Test Input Configurations.
abstract: A key issue in testing is how many tests are needed for a required level of coverage or fault detection. Estimates are often based on error rates in initial testing, or on code coverage. For example, tests may be run until a desired level of statement or branch coverage is achieved. Combinatorial methods present an opportunity for a different approach to estimating required test set size, using characteristics of the test set. This paper describes methods for estimating the coverage of, and ability to detect,t-way interaction faults of a test set based on a covering array. We also develop a connection between (static) combinatorial coverage and (dynamic) code coverage, such that if a specific condition is satisfied, 100% branch coverage is assured. Using these results, we propose practical recommendations for using combinatorial coverage in specifying test requirements.
authors.affiliation: National Institute of Standards and Technology, Gaithersburg, MD 20899, USA.
authors.forename: D Richard
authors.initials: DR
authors.lastname: Kuhn
authors.affiliation: National Institute of Standards and Technology, Gaithersburg, MD 20899, USA.
authors.forename: Raghu N
authors.initials: RN
authors.lastname: Kacker
authors.affiliation: Computer Science and Engineering, Univ. of Texas Arlington, Arlington, TX 76019.
authors.forename: Yu
authors.initials: Y
authors.lastname: Lei
keywords: combinatorial testing; configuration model; factor covering array; state-space coverage; t-way testing; verification and validation (V&V)