title: Transformation Model Choice in Nonlinear Regression Analysis of Fluorescence-based Serial Dilution Assays.
abstract: Many modern serial dilution assays are based on fluorescence intensity (FI) readouts. We study optimal transformation model choice for fitting five parameter logistic curves (5PL) to FI-based serial dilution assay data. We first develop a generalized least squares-pseudolikelihood type algorithm for fitting heteroscedastic logistic models. Next we show that the 5PL and log 5PL functions can approximate each other well. We then compare four 5PL models with different choices of log transformation and variance modeling through a Monte Carlo study and real data. Our findings are that the optimal choice depends on the intended use of the fitted curves.
authors.affiliation: Vaccine and Infectious Diseases Division, Public Health Sciences Division, Fred Hutchinson Cancer Research Center, yfong@fhcrc.org , xyu@fhcrc.org.
authors.forename: Youyi
authors.initials: Y
authors.lastname: Fong
authors.affiliation: Vaccine and Infectious Diseases Division, Public Health Sciences Division, Fred Hutchinson Cancer Research Center, yfong@fhcrc.org , xyu@fhcrc.org.
authors.forename: Xuesong
authors.initials: X
authors.lastname: Yu
keywords: curve fitting; heteroscedasticity; nonlinear calibration; weighted least squares