Module 37: Improving Subscore Value through Item Removal

Product not yet rated

Subscores can be of diagnostic value for tests that cover multiple underlying traits. Some items require knowledge or ability that spans more than a single trait. It is thus natural for such items to be included on more than a single subscore. Subscores only have value if they are reliable enough to justify conclusions drawn from them and if they contain information about the examinee that is distinct from what is in the total test score. In this study we show, for a broad range of conditions of item overlap on subscores, that the value of the subscore is always improved through the removal of items with little diagnostic value.

Keywords: added value, classical test theory, CTT, diagnostic value, empirical Bayes, item removal, overlapping items, ReliaVAR plots, simulation, subscores

Richard A. Feinberg

National Board of Medical Examiners, Philadelphia, PA

Richard Feinberg is a Senior Psychometrician with NBME, where he leads and oversees the data analysis and score reporting activities for large-scale high-stakes licensure and credentialing examinations. He is also an Assistant Professor at the Philadelphia College of Osteopathic Medicine, Philadelphia, PA, where he teaches a course on Research Methods and Statistics.

His research interests include psychometric applications in the fields of educational and psychological testing.

He earned a PhD in Research Methodology and Evaluation from the University of Delaware, Newark, DE.

Howard Wainer

Retired

Howard Wainer is an American statistician, past principal research scientist at the Educational Testing Service, adjunct professor of statistics at the Wharton School of the University of Pennsylvania, and author, known for his contributions in the fields of statistics, psychometrics, and statistical graphics.