Digital Module 28: Unusual Things that Usually Occur in a Credentialing Testing Program

Digital Module 28: Unusual Things that Usually Occur in a Credentialing Testing Program

Product not yet rated

Recorded On: 02/25/2022

In this digital ITEMS module, Drs. Richard Feinberg, Carol Morrison, and Mark R. Raymond discuss an overview of how credentialing testing programs operate and special considerations that need to be made when unusual things occur.  Formal graduate education in a measurement related field provides a solid foundation for professionals who work on credentialing examinations. Those foundational skills are then expanded and refined over time as practitioners encounter complex and nuanced challenges that were not covered by or go beyond the context described in textbooks. For instance, as most of us who work on operational testing programs are (sometimes) painfully aware, real data can be very messy. Often unanticipated situations arise that can create a range of problems from threats to score validity, to unexpected financial costs, and even longer-term reputational damage. In practice, solutions for these unanticipated situations are not always straightforward, often requiring a compromise between psychometric best practices, business resources, and needs of the customer. In this module we discuss some of these unusual challenges that usually occur in a credentialing program. First, we provide a high-level summary of the main components of the assessment lifecycle and the different roles within a testing organization. Next, we propose a framework for qualifying risk along with various considerations and potential actions for managing these challenges. Lastly, we integrate this information by presenting a few scenarios that can occur in practice that should help learners think though applicable team-based problem-solving and better align recommended action from a psychometric perspective given the context and magnitude of the challenge.


Keywords:Credential/Licensure Testing, Assessment Design, Assessment Challenges, Threats to Score Validity, Operational Psychometrics

Richard Feinberg

Senior Psychometrician

National Board of Medical Examiners

Contact Richard Feinberg via RFeinberg@nbme.org

Carol Morrison

Principal Psychometrician

National Board of Medical Examiners

Contact Carol Morrison: CMorrison@nbme.org

Mark R. Raymond

Director of Assessment Design and Delivery

National Conference of Bar Examiners

Contact Mark Raymond: MRaymond@ncbex.org

Key:

Complete
Failed
Available
Locked
DM28 (Unusual Things, Version 1.0)
Recorded 02/25/2022
Recorded 02/25/2022 Full interactive version of the digital module with all quiz questions and resource links.
DM28 VIDEO (Section 1, Version 1.0)
Open to view video.
Open to view video. Video version of the first content section of the module. This module is optimized for interaction. Video components are limited and only contain small snippets of each section.
DM28 VIDEO (Section 2, Version 1.0)
Open to view video.
Open to view video. Video version of the second content section of the module. This module is optimized for interaction. Video components are limited and only contain small snippets of each section.
DM28 VIDEO (Section 3, Version 1.0)
Open to view video.
Open to view video. Video version of the third content section of the module. This module is optimized for interaction. Video components are limited and only contain small snippets of each section.
DM28 VIDEO (Section 4, Version 1.0)
Open to view video.
Open to view video. Video version of the fourth content section of the module. This module is optimized for interaction. Video components are limited and only contain small snippets of each section.