Original ResearchAverage vs item response theory scores: an illustration using neighbourhood measures in relation to physical activity in adults with arthritis
Introduction
Health outcome models often use average or sum scores from classical test theory. We modelled both classical test theory (or average) scores and modern measurement theory (or item response theory [IRT]) scores. These two measurement frameworks are supposedly comparable yet their comparisons are novel.1
IRT models describe the relationship between item responses and latent constructs in terms of item and person parameters.2 The item parameters, which are very often the primary focus of an IRT analysis, tell us things like 1) what range of the construct do the items measure?; 2) do the items measure the construct well or poorly?; and 3) do any of the items behave in unexpected ways? These are very useful questions when one is developing or evaluating a scale. The other part of the model is the person parameter. In IRT, there is an estimate for each individual (which is oven called an ‘IRT scale score’ or ‘IRT score’ for short), which takes this rich item-level information and applies it to the individual pattern of responses to produce a score that takes into account everything learned during the IRT analysis. As such, an IRT score considers not only how many items were endorsed, but which items were endorsed. Unless the items are very similar in their parameters, this has the effect of creating different scores for each pattern of responses to a set of items. In classical test theory (CTT), a measurement paradigm most often associated with coefficient alpha, scores are created by adding (or averaging) item responses. This is equivalent to unit-weighting each response. It also imposes equal steps between Likert-type responses (i.e. there is one unit between a 2 and 1 and one unit between a 3 and 2) and further constrains all items to have the exact same steps. To the extent that an IRT model with varying parameters fits a data set well (i.e. better than a model that constrains everything to be equal), one could anticipate that the resulting IRT scores should better reflect the construct of interest.
As an illustration, we used average scores and IRT scores to examine the relation of the perceived neighbourhood physical features with individual physical activity levels in a cohort of adults with arthritis. Studies have consistently identified possible perceived neighbourhood physical environmental barriers as risk factors for decreased physical activity levels among older adults.3, 4 A clearer understanding of the neighbourhood-level risk factors correlated with physical activity among individuals with arthritis will help to develop and implement interventions targeting the perceived neighbourhood physical environment.
In summary, the purposes of this study were to 1) determine whether perceived neighbourhood physical features are associated with physical activity levels in adults with arthritis; and 2) determine whether the conclusions are more precise when item response theory (IRT) scores are used or when average scores are used for the perceived neighbourhood physical features scales. We focus on two perceived neighbourhood physical features, a built factor (walkability) and a social capital factor (aesthetics).4 We hypothesize that participants with a poorer perceived neighbourhood environment will have a decreased physical activity level compared to participants with enriched perceived physical neighbourhood features. Additionally, we hypothesize that the use of the IRT scores will impact the conclusions reached by being more precise.
Section snippets
Sample
This is a cross-sectional survey of adults (n = 2420) from the Individual and Community Social Determinants of Arthritis Outcomes Study (SODE) which was initiated in 2006 as a telephone follow-up survey to the North Carolina Family Medicine Research Network, a practice-based network cohort devoted to research on chronic diseases in primary care in North Carolina.5 This network cohort was purposefully sampled to represent both the urban and rural communities and the ethnic diversity of North
Results
Out of the 2420 individuals from the North Carolina Family Medicine Research Network who agreed to be contacted, 1541 completed the survey with a response rate of 65.2%. Of this sample, 937 adults self-identified as having doctor diagnosed arthritis (including osteoarthritis, rheumatoid arthritis, or fibromyalgia) and 910 provided complete data for our models.
For the AE analyses, the one-dimensional CFA model revealed some local dependence. To avoid any potential local dependence in the IRT
Discussion
In the present large cross-sectional study, we evaluated the relation of perceived neighbourhood physical features, specifically walkability and aesthetics, with individual physical activity levels in adults with self-reported doctor diagnosis of arthritis. Results of this study indicate that individuals with poorer perceived neighbourhood physical features have decreased individual physical activity levels compared to participants with better-perceived neighbourhood physical features. These
Acknowledgements
The authors want to acknowledge the participating family practices in the NC-FM-RN, including Black River Health Services, Burgaw; Bladen Medical Associates, Elizabethtown; Blair Family Medicine, Wallace; Cabarrus Family Medicine, Concord; Cabarrus Family Medicine, Harrisburg; Cabarrus Family Medicine, Kannapolis; Cabarrus Family Medicine, Mt Pleasant; Chatham Primary Care, Siler City; Carolinas Medical Center Biddle Point, Charlotte; Carolinas Medical Center North Park, Charlotte; Community
References (24)
- et al.
Associations of neighborhood problems and neighborhood social cohesion with mental health and health behaviors: the Multi-Ethnic Study of Atherosclerosis
Health Place
(2008) - et al.
Perceived and objective environmental measures and physical activity among urban adults
Am J Prev Med
(2005) - et al.
Establishing a common metric for self-reported anxiety: linking the MASQ, PANAS, and GAD-7 to PROMIS anxiety
J Anxiety Disord
(2014) Item response theory and classical test theory: an empirical comparison of their item/person statistics
Educ Psychol Meas
(1998)An introduction to item response theory using the Need for Cognition Scale
Soc Personal Psychol Compass
(2009)- et al.
The urban built environment and mobility in older adults: a comprehensive review
J Aging Res
(2011) Neighborhood and individual factors in activity in older adults: results from the neighborhood and senior health study
J Aging Phys Act
(2008)- et al.
Development of a practice-based patient cohort for primary care research
Fam Med
(2006) - et al.
Associations of perceived neighborhood environment on health status outcomes in persons with arthritis
Arthritis Care Res
(2010) - et al.
Reliability and validity of the instrument used in BRFSS to assess physical activity
Med Sci Sports Exerc
(2007)