The Effect of Aberrant Responses on Ability Estimation in Computer Adaptive Tests

GÖREN S., KARA H., ERDEM KARA B., KELECİOĞLU H.

Journal of Measurement and Evaluation in Education and Psychology, cilt.13, sa.3, ss.256-268, 2022 (ESCI) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 13 Sayı: 3
  • Basım Tarihi: 2022
  • Doi Numarası: 10.21031/epod.1067307
  • Dergi Adı: Journal of Measurement and Evaluation in Education and Psychology
  • Derginin Tarandığı İndeksler: Emerging Sources Citation Index (ESCI), TR DİZİN (ULAKBİM)
  • Sayfa Sayıları: ss.256-268
  • Anahtar Kelimeler: Computer adaptive tests (CAT), 3PL IRT model, 4PL IRT model, aberrant responses, early mistake
  • Anadolu Üniversitesi Adresli: Evet

Özet

In computer adaptive test (CAT), aberrant responses caused by some factors such as lucky guesses and carelessness errors may cause significant bias in ability estimation. Correct responses resulting from lucky guesses and false responses resulting from carelessness or anxiety may reveal aberrant responses and the impact of these types of aberrant responses may cause an erroneous estimation of the examinee's actual ability because they do not reflect the examinee's actual knowledge. In this study, the performances of regarding ability estimation were examined comparatively in the context of CAT simulations in case of aberrant responses.Under different conditions, twelve different CAT simulations were conducted with 10 replications for each of the conditions. Correlation, RMSE, bias, and mean absolute error (MAE) values were calculated and interpreted for each condition. Results generally indicated that the 4PL IRT model provided a more efficient and robust ability estimation than the 3PL IRT model and the 4PL model increased the precision and effectiveness of the CAT applications.