Module 32: Subscores

Product not yet rated

The purpose of this ITEMS module is to provide an introduction to subscores. First, examples of subscores from an operational test are provided. Then, a review of methods that can be used to examine if subscores have adequate psychometric quality is provided. It is demonstrated, using results from operational and simulated data, that subscores have to be based on a sufficient number of items and have to be sufficiently distinct from each other to have adequate psychometric quality. It is also demonstrated that several operationally reported subscores do not have adequate psychometric quality. Recommendations are made for those interested in reporting subscores for educational tests.

Keywords: added value, augmented subscore, classical test theory, CTT, diagnostic score, item response theory, IRT, mean squared error, proportional reduction in mean squared error, PRMSE, reliability

Sandip Sinharay

Principal Research Scientist, Educational Testing Service

Sandip Sinharay is a principal research scientist in the Research and Development division at ETS. He received his Ph.D degree in statistics from Iowa State University in 2001. He was editor of the Journal of Educational and Behavioral Statistics between 2011 and 2014. Sandip Sinharay has received four awards from the National Council on Measurement in Education: the award for Technical or Scientific Contributions to the Field of Educational Measurement (in 2009 and 2015), the Jason Millman Promising Measurement Scholar Award (2006), and the Alicia Cascallar Award for an Outstanding Paper by an Early Career Scholar (2005). He received the ETS Scientist award in 2008 and the ETS Presidential award twice. He has coedited two published volumes and authored or coauthored more than 75 articles in peer-reviewed statistics and psychometrics journals and edited books.

Gautam Puhan

Principal Psychometrician, Educational Testing Service

Gautam Puhan is a Principal Psychometrician in the SAT team at Educational Testing Service. Gautam received a Ph.D. in educational psychology from the University of Alberta, Canada, and an M.A. in psychology from the University of Delhi, India. His research interests include test score equating, diagnostic subscores, test reliability, differential item functioning (DIF) and test fairness.