title: A permutation method to assess heterogeneity in external validation for risk prediction models.
abstract: The value of a developed prediction model depends on its performance outside the development sample. The key is therefore to externally validate the model on a different but related independent data. In this study, we propose a permutation method to assess heterogeneity in external validation for risk prediction models. The permutation p value measures the extent of homology between development and validation datasets. If p < 0.05, the model may not be directly transported to the external validation population without further revision or updating. Monte-Carlo simulations are conducted to evaluate the statistical properties of the proposed method, and two microarray breast cancer datasets are analyzed for demonstration. The permutation method is easy to implement and is recommended for routine use in external validation for risk prediction models.
authors.affiliation: Department of Medical Research, Tzu Chi General Hospital, Hualien, Taiwan.
authors.forename: Ling-Yi
authors.initials: LY
authors.lastname: Wang
authors.affiliation: Research Center for Genes, Environment and Human Health and Institute of Epidemiology and Preventive Medicine, College of Public Health, National Taiwan University, Taipei, Taiwan.
authors.forename: Wen-Chung
authors.initials: WC
authors.lastname: Lee
mesh_terms: D001943:Breast Neoplasms / Q000175:diagnosis* / Q000235:genetics; D005260:Female; D020869:Gene Expression Profiling; D006801:Humans; D015233:Models, Statistical; D009010:Monte Carlo Method; D020411:Oligonucleotide Array Sequence Analysis; D015203:Reproducibility of Results; D012306:Risk