Digital Module 06: Posterior Predictive Model Checking

3.83 (6 votes)

Recorded On: 04/24/2019

In this digital ITEMS module, Dr. Allison Ames and Aaron Myers discuss the most common Bayesian approach to model-data fit evaluation called Posterior Predictive Model Checking (PPMC). Specifically, drawing valid inferences from modern measurement models is contingent upon a good fit of the data to the model and violations of model-data fit have numerous adverse consequences, limiting the usefulness and applicability of the model. As Bayesian estimation is becoming more common, understanding the Bayesian approaches for evaluating model-data fit models is critical. The instructors review the conceptual foundation of Bayesian inference as well as PPMC and walk through the computational steps of PPMC using real-life data examples from simple linear regression and item response theory (IRT) analysis. They provide guidance for how to interpret PPMC results and discuss how to implement PPMC for other model(s) and data. The digital module contains sample data, SAS code, diagnostic quiz questions, data-based activities, curated resources, and a glossary.

Keywords: Bayesian inference, simple linear regression, item response theory, IRT, model fit, posterior predictive model checking, PPMC, Bayes theorem, Yen’s Q3, item fit

Key:

Complete
Failed
Available
Locked
Digital Module (COMPLETE INTERACTIVE VERSION)
Recorded 04/24/2019
Recorded 04/24/2019 Complete digital module with all content slides, data activities, quiz questions, glossary, and resources.
DM06 VIDEO (Introduction)
Open to view video.
Open to view video. Video version of introduction without author intro videos [4:39 minutes]
DM06 VIDEO (Section 1 - Conceptual Foundations)
Open to view video.
Open to view video. Video version of Section 1 (Conceptual Foundations) without interactive components [29:25 minutes]
DM06 VIDEO (Section 2 - Linear Regression)
Open to view video.
Open to view video. Video version of Section 2 (Linear Regression) without interactive components [47:04 minutes]
DM06 VIDEO (Section 3 - Item Response Theory)
Open to view video.
Open to view video. Video version of Section 3 (Item Response Theory) without interactive components [XX:XX minutes]
VIDEO Regression Example 1 (Linearity Assumption)
Open to view video.
Open to view video. Video of all steps for PPMC of the linearity assumption for a simple linear regression model [11:13 minutes]
VIDEO Regression Example 2 (Equal Variance Assumption)
Open to view video.
Open to view video. Video of all steps for PPMC of the equal variance / homoscedasticity assumption for a simple linear regression model [10:28 minutes]
VIDEO IRT Example 1 (Model Estimation)
Open to view video.
Open to view video. Video of all steps for specifying and estimating an IRT model in SAS [6:28 minutes]
VIDEO IRT Example 2 (Frequency Distribution, Part 1)
Open to view video.
Open to view video. Video of all steps for using summary statistics from the total score distribution to evaluate model-data fit [5:09 minutes]
VIDEO IRT Example 3 (Frequency Distribution, Part 2)
Open to view video.
Open to view video. Video of all steps for using the total score distribution globally to evaluate model-data fit [7:12 minutes]
VIDEO IRT Example 4 (Item Difficulty)
Open to view video.
Open to view video. Video of using item difficulty statistics to evaluate model-data fit [4:31 minutes]
VIDEO IRT Example 5 (Item Discrimination)
Open to view video.
Open to view video. Video of using item discrimination statistics to evaluate model-data fit [5:36 minutes]
VIDEO IRT Example 6 (Yen's Q3)
Open to view video.
Open to view video. Video of using Yen's Q3 statistic to evaluate dependency assumptions for model-data fit [6:39 minutes]
Data Files
Open to download resource.
Open to download resource. All data files for all examples and activities.

Allison J. Ames

Assistant Professor

Allison is an assistant professor in the Educational Statistics and Research Methods program in the Department of Rehabilitation, Human Resources and Communication Disorders, Research Methodology, and Counseling at the University of Arkansas. There, she teaches courses in educational statistics, including a course on Bayesian inference. Allison received her Ph.D. from the University of North Carolina at Greensboro. Her research interests include Bayesian item response theory, with an emphasis on prior specification; model-data fit; and models for response processes. Her research has been published in prominent peer-reviewed journals. She enjoyed collaborating on this project with a graduate student, senior faculty member, and the Instructional Design Team.
Contact Allison via boykin@uark.edu

Aaron Myers

Graduate Assistant / Doctoral Student

Aaron is a doctoral student in the Educational Statistics and Research Methods program at the University of Arkansas. His research interests include Bayesian inference, data mining, multidimensional item response theory, and multilevel modeling. Aaron previously received his M.A. in Quantitative Psychology from James Madison University. He currently serves as a graduate assistant where he teaches introductory statistics and works in a statistical consulting lab. 

Contact Aaron via ajm045@uark.edu