Digital Module 25: Testlet Models

5 (3 votes)

Recorded On: 11/15/2021

In this digital ITEMS module, Dr. Hong Jiao and Dr. Manqian Liao describe testlet response theory for the construction and evaluation of new measures and scales. They start with an introduction to the needs of testlets when local item dependence is present and then introduce the basic testlet response models proposed from different theoretical frameworks built upon standard item response theory models. Furthermore, they introduce different methods for model parameter estimation and related software programs that implement them. Finally, they showcase further extensions of the extended testlet response models for more complex local item dependence issues in innovative assessment. The module is designed for students, researchers, and data scientists in various disciplines such as psychology, sociology, education, business, health and other social sciences in developing testlet-based assessment. It contains audio-narrated slides, sample data, syntax files, diagnostic quiz questions, data-based activities, curated resources, and a glossary.

Key words: Bayesian estimation, innovative assessment, item response theory, local item dependence, multi-part items, paired passages, testlets, testlet response theory

Hong Jiao

Professor

University of Maryland, College Park

Hong Jiao is currently a professor at the University of Maryland (UMD), College Park specializing in educational measurement and psychometrics in large-scale assessment. She received her doctoral degree from Florida State University. Prior to joining the faculty in Measurement, Statistics, and Evaluation at UMD, she worked as a psychometrician at Harcourt Assessment on different state assessment programs. Her methodological research is to improve the practice in educational and psychological assessment and develop solutions to emerging psychometric challenges. Many of these are due to the use of more complex innovative assessment formats. Two areas of her research include methodological research on local dependence due to the use of testlet and Bayesian model parameter estimation. Her methodological research has been recognized by a national award, academic work including numerous edited books, book chapters, refereed journal papers, and national and international invited and refereed presentations and different research grants and contracts on which she serve as PI or CO-PI. Hong Jiao proposed a multilevel testlet model for mixed-format tests that won the 2014 Bradley Hanson Award for Contributions to Educational Measurement by the National Council on Measurement in Education.

Contact Hong at hjiao@umd.edu

Manqian Liao

Psychometrician

Duolingo

Manqian Liao is a psychometrician at Duolingo where she conducts validity and fairness research on the Duolingo English Test. Among other things, she has worked on investigating the differential item functioning in the Duolingo English Test items and evaluating the efficiency of the item selection algorithm. Manqian received her Ph.D. degree in Measurement, Statistics and Evaluation from University of Maryland, College Park. Her research interest focuses on item response theory (IRT) and diagnostic classification models (DCM). Her dissertation is on modeling multiple problem-solving strategies and strategy shift with DCM.