title: A measure of model reliability.
abstract: A method for evaluating predictive models is developed by giving a precise and statistically meaningful interpretation to the statement that a model is accurate within a factor of k. This method is applicable to any model for which there is a set [y1,..., yn] of observations corresponding to a set [x1,..., xn] of model predictions. We define a geometrically intuitive measure of model reliability kg in terms of the ratios yi/xi and a statistically rigorous measure ks in terms of 1n (yi/xi). For reasonably accurate models, kg and ks are in virtual agreement and thus can be used interchangeably as a reliability index. The index ks estimates a model parameter of the form exp[(V1 + V2)1/2], where V1 describes an observational variance and V2 is related to an uncertainty associated with the model itself. The computed value for ks is not unique but depends on the sample of observations. The probability distribution of ks can be characterized provided the observational distributions are lognormal, independent, and satisfy a homoscedasticity condition. These requirements are often satisfied by quantities of interest in radiation risk analyses. The reliability indices kg and ks may be applied even if the underlying observational distributions are not lognormal, although the probability distribution of ks cannot be characterized in this case.
mesh_terms: D000818:Animals; D001822:Body Burden; D001842:Bone and Bones / Q000528:radiation effects; D002588:Cesium Radioisotopes / Q000032:analysis; D004285:Dogs; D006801:Humans; D008954:Models, Biological; D011336:Probability; D011830:Radiation Effects; D011868:Radioisotopes / Q000032:analysis; D051381:Rats; D012306:Risk; D012413:Rubidium / Q000032:analysis; D013223:Statistics as Topic; D013326:Strontium Radioisotopes / Q000032:analysis; D013997:Time Factors
chemical_list: D002588:Cesium Radioisotopes; D011868:Radioisotopes; D013326:Strontium Radioisotopes; D012413:Rubidium