Funded Research
Aging and the Reliability of Survey Data
Source: NIH/NIA
Active: 02/15/03 - 01/31/07
Investigator(s):
Duane Alwin
Willard Rodgers
This research investigates the relationship of age to measurement errors in survey-administered self-report questionnaires. The overall aims of this project are to better understand the nature of survey measurement errors and the processes by which they are generated, and to make practical recommendations about the characteristics of survey questions that will improve the quality of data in surveys of the aging population.
To accomplish these goals the research conducts a systematic analysis of the reliability of responses to survey questions using several nationally-representative panel data sets. First, we build upon our prior NSF-supported research using three surveys from the National Election Study (NES) series -- the 1956-58-60, 1972-74-76, and 1992-94-96 panel studies of the American electorate, and a fourth panel survey, the 1986-89-94 Americans' Changing Lives (ACL) study of health and well-being, to examine reliability by age. Second, we investigate these issues using two innovative panel surveys of middle-aged and older adults: the original Health & Retirement Study (HRS) panel study of pre-retirement men and women aged 51-61 assessed in 1992 (n=9,824), and re-interviewed in 1994, 1996, 1998, 2000, and 2002 and the parallel study of Asset and Health Dynamics (AHEAD) which interviewed adults aged 70 and above in 1993, and re-interviewed them in 1995, 1998, 2000 and 2002 (n= 7,447). The younger HRS (birth cohorts of 1931-41) and older AHEAD (birth cohorts of 1923 and before) respondents were asked many of the same questions, permitting the comparison of measurement errors across groups.
Age-specific levels of reliability will be estimated for approximately 1,000 survey questions using a variety of state-of-the-art estimation strategies. Two basic approaches will be employed in the estimation of reliability first the SEM-based maximum-likelihood approach for situations where it makes sense to assume continuous unobserved latent variables, and second the latent transition models that are appropriate where the unobserved latent variables are latent classes. Within the SEM approach the research will employ several different estimation strategies depending on the scale assumptions appropriate to the observed data, including standard Pearson-based covariance approaches for continuous variables, tetrachoric correlations for dichotomous variables and the polychoric-based asymptotic covariance approach with weighted least squares estimation for ordinal variables. A major focus of the analysis is on age-related differences in the impact of the formal properties of survey questions (e.g., question content, length of question text, number and complexity of response options, and the provision of explicit Don't Know response options) on reliability.
Assembling information on reliability from these data sources can help improve knowledge about the strengths and weaknesses of survey data. It is expected that the results of this research will be relevant to the general task of uncovering the sources of age-related measurement errors in surveys and the improvement of methods of survey data collection across the life span through the application of this knowledge.
