Same Test, Better Scores: Boosting the Reliability of Short Online Intelligence Recruitment Tests with Nested Logit Item Response Theory Models
Один и тот же тест, более высокие показатели: повышение надёжности кратких онлайн-тестов интеллекта для отбора персонала с помощью вложенных логит-моделей теории ответов на задания
2019-07-10
SCID: 54.1/kn5pt5eq
Discuss with AI
Item Response TheoryNested Logit ModelsOnline intelligence testingRecruitment testingReliability of ability estimates
Figures from the paper
Abstract (AI)
Assessing job applicants' general mental ability online poses psychometric challenges due to the necessity of having brief but accurate tests. Recent research (Myszkowski & Storme, 2018) suggests that recovering distractor information through Nested Logit Models (NLM; Suh & Bolt, 2010) increases the reliability of ability estimates in reasoning matrix-type tests. In the present research, we extended this result to a different context (online intelligence testing for recruitment) and in a larger sample ( N = 2949 job applicants). We found that the NLMs outperformed the Nominal Response Model (Bock, 1970) and provided significant reliability gains compared with their binary logistic counterparts. In line with previous research, the gain in reliability was especially obtained at low ability levels. Implications and practical recommendations are discussed.
Key Findings
1
Nested Logit Models improved the reliability of ability estimates in brief online reasoning tests used for job recruitment.
2
Nested Logit Models outperformed the Nominal Response Model and yielded significant reliability gains over corresponding binary logistic models.
3
Reliability improvements were particularly pronounced among applicants with low ability levels.
4
The findings replicated and extended prior evidence on recovering distractor information in matrix-type intelligence tests to a larger applicant sample of N = 2949.
5
The results support using Nested Logit Item Response Theory models to enhance the accuracy of short online intelligence recruitment tests.
Research Object
online intelligence recruitment tests for job applicants, specifically reasoning matrix-type tests
Research Subject
the reliability and accuracy of general mental ability estimates, including the benefits of Nested Logit Item Response Theory models and their performance across ability levels
Publication Details
Publication Date
2019-07-10
Journal
Publisher
ISSN
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest