In this ITEMS module the authors frame the topic of scale reliability within a confirmatory factor analysis and structural equation modeling (SEM) context and address some of the limitations of Cronbach’s α. This modeling approach has two major advantages: (1) it allows researchers to make explicit the relation between their items and the latent variables representing the constructs those items intend to measure, and (2) it facilitates a more principled and formal practice of scale reliability evaluation. Specifically, the authors begin the module by discussing key conceptual and statistical foundations of the classical test theory (CTT) model and then framing it within an SEM context; they do so first with a single item and then expand this approach to a multi-item scale. This allows them to set the stage for presenting different measurement structures that might underlie a scale and, more importantly, for assessing and comparing those structures formally within the SEM context. They then make explicit the connection between measurement model parameters and different measures of reliability, emphasizing the challenges and benefits of key measures while ultimately endorsing the flexible McDonald’s ω over Cronbach’s α. The authors then demonstrate how to estimate key measures in both a commercial software program (Mplus) and three packages within an open-source environment (R). In closing, they make recommendations for practitioners about best practices in reliability estimation based on the ideas presented in the module.

Keywords:  scale reliability; structural equation modeling; Cronbach’s α; McDonald’s ω