Digital Module 07: Subscore Evaluation & Reporting

5 (4 votes)

Recorded On: 09/09/2019

In this digital ITEMS module, Dr. Sandip Sinharay reviews the status quo on the reporting of subscores. Specifically, he first provides examples of operationally-reported subscores, discusses why subscores are in high demand, and discusses professional quality standards that subscores have to satisfy. He then describes various statistical methods that can be used to evaluate whether subscores satisfy professional standards, which include descriptive statistics, DIMTEST / DETECT, factor analysis, multidimensional item response theory, and the Haberman method. He provides guidance for how to implement these methods on real data using the R package ‘subscores’.

Keywords: Diagnostic scores, disattenuation, DETECT, DIMTEST, factor analysis, multidimensional item response theory (MIRT), proportional reduction in mean squared error (PRMSE), reliability, subscores

Module Overview

Core Slides

Sandip Sinharay

Principal Research Scientist

Dr. Sandip Sinharay is a Principal Research Scientist at ETS. He received his Ph.D. and M.S. degrees from the Department of Statistics at Iowa State University. He has received five awards from the National Council on Measurement in Education including the Bradley Hanson Award (2018), the Technical or Scientific Contributions to the Field of Educational Measurement (2009 and 2015), the Jason Millman Promising Measurement Scholar Award (2006), and the Alicia Cascallar Award for an Outstanding Paper by an Early Career Scholar (2005). Sandip has coedited two published volumes and authored or coauthored more than 100 articles in peer-reviewed statistics and psychometrics journals and edited books. His research interests include statistical methods for detecting test fraud, reporting of subscores, Bayesian statistical methods, as well as model-data fit and model selection methods. The collaboration with the instructional design team on this project was a unique learning experience for Sandip.

Contact Sandip via ssinharay@ets.org