International Journal of Selection and Assessment

Skip to Search

Skip to Navigation

Volume 27 Issue 3 (September 2019), Pages 217-295

Option weights should be determined empirically and not by experts when assessing knowledge with multiple‐choice items (pages 256-266)

Abstract Multiple‐choice tests are frequently used in personnel selection contexts to measure knowledge and abilities. Option weighting is an alternative multiple‐choice scoring procedure that awards partial credit for incomplete knowledge reflected in applicants’ distractor choices. We investigated whether option weights should be based on expert judgment or on empirical data when trying to outperform conventional number‐right scoring in terms of reliability and validity. To obtain generalizable results, we used repeated random sub‐sampling validation and found that empirical option weighting, but not expert option weighting, increased the reliability of a knowledge test. Neither option weighting procedure improved test validity. We recommend to improve the reliability of existing ability and knowledge tests used for personnel selection by computing and publishing empirical option weights.

Add This link

Bookmark and Share>