Inter-rater reliability of the PIPES tool: validation of a surgical capacity index for use in resource-limited settings.
BACKGROUND
In response to the need for simple, rapid means of quantifying surgical capacity in low resource settings, Surgeons OverSeas (SOS) developed the personnel, infrastructure, procedures, equipment and supplies (PIPES) tool. The present investigation assessed the inter-rater reliability of the PIPES tool.


METHODS
As part of a government assessment of surgical services in Santa Cruz, Bolivia, the PIPES tool was translated into Spanish and applied in interviews with physicians at 31 public hospitals. An additional interview was conducted with nurses at a convenience sample of 25 of these hospitals. Physician and nurse responses were then compared to generate an estimate of reliability. For dichotomous survey items, inter-rater reliability between physicians and nurses was assessed using the Cohen's kappa statistic and percent agreement. The Pearson correlation coefficient was used to assess agreement for continuous items.


RESULTS
Cohen's kappa was 0.46 for infrastructure, 0.43 for procedures, 0.26 for equipment, and 0 for supplies sections. The median correlation coefficient was 0.91 for continuous items. Correlation was 0.79 for the PIPES index, and ranged from 0.32 to 0.98 for continuous response items.


CONCLUSIONS
Reliability of the PIPES tool was moderate for the infrastructure and procedures sections, fair for the equipment section, and poor for supplies section when comparing surgeons' responses to nurses' responses-an extremely rigorous test of reliability. These results indicate that the PIPES tool is an effective measure of surgical capacity but that the equipment and supplies sections may need to be revised.
Surgeons OverSeas (SOS), New York, NY, USA, mark0327@umn.edu.
Abraham
A
Markin
Roxana
R
Barbero
Jeffrey J
JJ
Leow
Reinou S
RS
Groen
Greg
G
Perlman
Elizabeth B
EB
Habermann
Keith N
KN
Apelgren
Adam L
AL
Kushner
Benedict C
BC
Nwomeh
D001838:Bolivia; D003906:Developing Countries; D013502:General Surgery; D006301:Health Services Needs and Demand; D006740:Hospital Administrators; D006779:Hospitals, Public; D006801:Humans; D007407:Interviews as Topic; D008505:Medical Staff, Hospital; D020380:Needs Assessment; D009741:Nursing Staff, Hospital; D015588:Observer Variation; D015203:Reproducibility of Results; D013523:Surgical Equipment / Q000600:supply & distribution*; D013514:Surgical Procedures, Operative; D000078329:Workforce