Digital Module 07: Subscore Evaluation & Reporting

5 (3 votes)

Recorded On: 09/09/2019

In this digital ITEMS module, Dr. Sandip Sinharay reviews the status quo on the reporting of subscores. Specifically, he first provides examples of operationally-reported subscores, discusses why subscores are in high demand, and discusses professional quality standards that subscores have to satisfy. He then describes various statistical methods that can be used to evaluate whether subscores satisfy professional standards, which include descriptive statistics, DIMTEST / DETECT, factor analysis, multidimensional item response theory, and the Haberman method. He provides guidance for how to implement these methods on real data using the R package ‘subscores’.

Keywords: Diagnostic scores, disattenuation, DETECT, DIMTEST, factor analysis, multidimensional item response theory (MIRT), proportional reduction in mean squared error (PRMSE), reliability, subscores  


Sandip Sinharay

Principal Research Scientist

Dr. Sandip Sinharay is a Principal Research Scientist at ETS. He received his Ph.D. and M.S. degrees from the Department of Statistics at Iowa State University. He has received five awards from the National Council on Measurement in Education including the Bradley Hanson Award (2018), the Technical or Scientific Contributions to the Field of Educational Measurement (2009 and 2015), the Jason Millman Promising Measurement Scholar Award (2006), and the Alicia Cascallar Award for an Outstanding Paper by an Early Career Scholar (2005). Sandip has coedited two published volumes and authored or coauthored more than 100 articles in peer-reviewed statistics and psychometrics journals and edited books. His research interests include statistical methods for detecting test fraud, reporting of subscores, Bayesian statistical methods, as well as model-data fit and model selection methods. The collaboration with the instructional design team on this project was a unique learning experience for Sandip. 

Contact Sandip via ssinharay@ets.org 

Key:

Complete
Failed
Available
Locked
Digital Module (Complete Interactive Version)
Recorded 09/09/2019
Recorded 09/09/2019 The full interactive digital module with all videos and resources.
VIDEO DM07 (Sections 1-3)
Open to view video.
Open to view video. Video of the first three content sections without author introductions, interactivity, and quiz questions.
Data Files
Open to download resource.
Open to download resource. All files for the data activity and worked example (R code files, TIMSS data file, R package manual)