Exploring inter-rater reliability and measurement properties of environmental ratings using kappa and colocation quotientsReport as inadecuate

Exploring inter-rater reliability and measurement properties of environmental ratings using kappa and colocation quotients - Download this document for free, or read online. Document in PDF available to download.

Environmental Health

, 13:86

First Online: 23 October 2014Received: 26 June 2014Accepted: 14 October 2014DOI: 10.1186-1476-069X-13-86

Cite this article as: Björk, J., Rittner, R. & Cromley, E. Environ Health 2014 13: 86. doi:10.1186-1476-069X-13-86


BackgroundAvailable evidence suggest that perceptions or ratings of the neighborhood, e.g. as being green, walkable or noisy, are important for effects on health and wellbeing, also after controlling for objective measures of identical or similar features. When evaluating effects of the perceived environment, it is important that measurement properties and the reliability of the environmental ratings are evaluated before decisions about how these ratings should be handled in the statistical analyses are made. In this paper we broaden the usage of two association measures, the well-known kappa statistic and the novel colocation quotient CLQ, to studies of inter-rater reliability and of associations between different categorical ratings in spatial contexts.

MethodsWe conducted reliability analysis of a survey instrument for assessing perceived greenness at geographical point locations, here the close outdoor environment within 5–10 minutes walking distance from home. Data were obtained from a public health survey conducted in 2008 in Scania, southern Sweden n =27 967 participants.

ResultsThe results demonstrate the usefulness of kappa and CLQ as tools for assessing reliability and measurement properties of environmental rating scales when used at geographical point locations. We further show that the two measures are interchangeable, i.e. kappa can be accurately approximated from CLQ and vice versa, but can be used for somewhat different purposes in reliability analyses. Inter-rater reliability between the nearest neighbors was demonstrated for all five items of the evaluated instrument for assessing perceived greenness, albeit with clear differences across the items.

ConclusionReliability analysis employing kappa and CLQ can be used as a basis for informed decisions about, for instance, how dichotomizations of the ratings should be defined and how missing or indefinite ratings should be handled. Such reliability analyses can thus serve as guidance for subsequent epidemiological studies of associations between environmental ratings, health and wellbeing.

KeywordsEpidemiologic methods Spatial analysis Reproducibility of results Statistics as topic Perceived greenness AbbreviationCLQColocation quotient.

Electronic supplementary materialThe online version of this article doi:10.1186-1476-069X-13-86 contains supplementary material, which is available to authorized users.

Download fulltext PDF

Author: Jonas Björk - Ralf Rittner - Ellen Cromley

Source: https://link.springer.com/


Related documents