Digital Module 02: Scale Reliability in Structural Equation Modeling

4 (7 votes)

Recorded On: 06/14/2019

In this digital ITEMS module, we frame the topic of scale reliability within a confirmatory factor analysis and structural equation modeling (SEM) context and address some of the limitations of Cronbach’s α. This modeling approach has two major advantages: (1) it allows researchers to make explicit the relation between their items and the latent variables representing the constructs those items intend to measure, and (2) it facilitates a more principled and formal practice of scale reliability evaluation. Specifically, we begin the module by discussing key conceptual and statistical foundations of the classical test theory (CTT) model and then framing it within an SEM context; we do so first with a single item and then expand this approach to a multi-item scale. This allows us to set the stage for presenting different measurement structures that might underlie a scale and, more importantly, for assessing and comparing those structures formally within the SEM context. We then make explicit the connection between measurement model parameters and different measures of reliability, emphasizing the challenges and benefits of key measures while ultimately endorsing the flexible McDonald’s ω over Cronbach’s α. We then demonstrate how to estimate key measures in both a commercial software program (Mplus) and three packages within an open-source environment (R). In closing, we make recommendations for practitioners about best practices in reliability estimation based on the ideas presented in the module.

Keywords:  congeneric, Cronbach’s alpha, reliability, scale reliability, SEM, structural equation modeling, McDonald’s omega, model fit, parallel, tau-equivalent

Key:

Complete
Failed
Available
Locked
Digital Module
Recorded 06/13/2019
Recorded 06/13/2019 Full digital module with all resources and activities.
VIDEO DM02 (Sections 1-3)
Open to view video.
Open to view video. Video of the first three content sections.
VIDEO Shiny R App
Open to view video.
Open to view video. An introduction for how to install the Shiny R app with an overview of its core functionalities.
VIDEO: Data Activity
Open to view video.
Open to view video. A video with an illustrative data activity within the Shiny R app.
VIDEO Statistics Refresher
Open to view video.
Open to view video. Video of one of the refresher sections included in this module covering basic concepts in statistics relevant for this module, specifically (1) means, (2) variances / standard deviations, (3) correlations / covariances, and (4) confirmatory factor analysis.
Data Files
Open to download resource.
Open to download resource. All data files for the worked examples and data activities.

Gregory R. Hancock

Professor

Gregory R. Hancock is Professor, Distinguished Scholar-Teacher, and Director of the Measurement, Statistics and Evaluation program in the Department of Human Development and Quantitative Methodology at the University of Maryland, College Park, and Director of the Center for Integrated Latent Variable Research (CILVR). His research interests include structural equation modeling and latent growth models, and the use of latent variables in (quasi)experimental design. His research has appeared in a wide variety of prestigious peer-reviewed journals. He also co-edited the volumes Structural Equation Modeling: A Second Course (2006; 2013), The Reviewer's Guide to Quantitative Methods in the Social Sciences (2010), and Advances in Longitudinal Methods in the Social and Behavioral Sciences (2012). He has taught dozens of methodological workshops in the United States, Canada, and outside North America and has received the 2011 Jacob Cohen Award for Distinguished Contributions to Teaching and Mentoring by the APA.

Contact Greg via ghancock@umd.edu

Ji An

Ph.D. Candidate

Ji An is a Ph.D. candidate in the Measurement, Statistics and Evaluation program in the Department of Human Development and Quantitative Methodology at the University of Maryland, College Park. Her research interests include propensity score methods, analysis of survey data with complex sampling designs, structural equation modeling, and multilevel modeling. Before her time at College Park she had earned an M.A. in Teaching and Curriculum from Michigan State University. She has subsequently worked as an instructor for introductory courses in educational statistics and as a graduate assistant for various courses at the undergraduate and graduate level.  

Contact Ji via anji.nihao@gmail.com