title: Assessing agreement of repeated binary measurements with an application to the CDC's anthrax vaccine clinical trial.
abstract: Cohen's kappa coefficient, which was introduced in 1960, serves as the most widely employed coefficient to assess inter-observer agreement for categorical outcomes. However, the original kappa can only be applied to cross-sectional binary measurements and, therefore, cannot be applied in the practical situation when the observers evaluate the same subjects at repeated time intervals. This study summarizes six methods of assessing agreement of repeated binary outcomes under different assumptions and discusses under which condition we should use the most appropriate method in practice. These approaches are illustrated using data from the CDC anthrax vaccine adsorbed (AVA) human clinical trial comparing the agreement for two solicited adverse events after AVA between the 1-3 day in-clinic medical record and the patient's diary on the same day. We hope this article can inspire researchers to choose the most appropriate method to assess agreement for their own study with longitudinal binary data.
authors.affiliation: Immunization Safety Office, Division of Healthcare Quality Promotion, National Center for Emerging and Zoonotic Infectious Diseases, Centers for Disease Control and Prevention, Atlanta, GA, USA. jun5@cdc.gov
authors.forename: Yi
authors.initials: Y
authors.lastname: Pan
authors.forename: Charles E
authors.initials: CE
authors.lastname: Rose
authors.forename: Michael
authors.initials: M
authors.lastname: Haber
authors.forename: Yan
authors.initials: Y
authors.lastname: Ma
authors.forename: Josep L
authors.initials: JL
authors.lastname: Carrasco
authors.forename: Brock
authors.initials: B
authors.lastname: Stewart
authors.forename: Wendy A
authors.initials: WA
authors.lastname: Keitel
authors.forename: Harry
authors.initials: H
authors.lastname: Keyserling
authors.forename: Robert M
authors.initials: RM
authors.lastname: Jacobson
authors.forename: Gregory
authors.initials: G
authors.lastname: Poland
authors.forename: Michael M
authors.initials: MM
authors.lastname: McNeil
mesh_terms: D000328:Adult; D000881:Anthrax / Q000517:prevention & control*; D022122:Anthrax Vaccines / Q000008:administration & dosage* / Q000009:adverse effects; D002487:Centers for Disease Control and Prevention, U.S.; D002986:Clinical Trials as Topic / Q000379:methods*; D003627:Data Interpretation, Statistical; D005260:Female; D006801:Humans; D008137:Longitudinal Studies / Q000379:methods*; D008297:Male; D008875:Middle Aged; D015233:Models, Statistical; D014481:United States
chemical_list: D022122:Anthrax Vaccines
references.citation: JAMA. 2008 Oct 1;300(13):1532-43
references.pmid: 18827210
references.citation: J Clin Epidemiol. 1992 Nov;45(11):1237-41
references.pmid: 1432004
references.citation: Psychol Bull. 1979 Mar;86(2):420-8
references.pmid: 18839484
references.citation: Biometrics. 2004 Sep;60(3):807-11
references.pmid: 15339305
references.citation: Stat Med. 1994 Dec 15-30;13(23-24):2465-76
references.pmid: 7701147
references.citation: Biometrics. 1999 Mar;55(1):137-48
references.pmid: 11318148
references.citation: Stat Med. 2001 Jul 30;20(14):2131-47
references.pmid: 11439426
references.citation: J Clin Oncol. 2005 Jul 20;23(21):4679-86
references.pmid: 15851764
references.citation: Phys Ther. 1994 Aug;74(8):777-88
references.pmid: 8047565
references.citation: Psychol Bull. 1968 Oct;70(4):213-20
references.pmid: 19673146
references.citation: Biometrics. 1977 Mar;33(1):159-74
references.pmid: 843571
references.citation: Stat Med. 2007 Jul 20;26(16):3095-113
references.pmid: 17216594
references.citation: Biometrics. 1989 Mar;45(1):255-68
references.pmid: 2720055
references.citation: Age Ageing. 2008 Jul;37(4):379-83
references.pmid: 18586834
references.citation: Lancet. 1986 Feb 8;1(8476):307-10
references.pmid: 2868172
references.citation: Biometrics. 2008 Sep;64(3):781-789
references.pmid: 18047535
references.citation: Psychol Rep. 1966 Aug;19(1):3-11
references.pmid: 5942109
references.citation: Cephalalgia. 2007 Jan;27(1):1-8
references.pmid: 17212676
references.citation: J Biopharm Stat. 2007;17(4):673-84
references.pmid: 17613647
references.citation: J Biopharm Stat. 2007;17(4):653-72
references.pmid: 17613646
references.citation: J Biopharm Stat. 2009;19(1):90-105
references.pmid: 19127469