The effect of the correction of self-assessment-based chance success on psychometric characteristics of the test

Authors

DOI:

https://doi.org/10.14527/pegegog.2018.022

Abstract

This study examines the effect of self-assessment-based chance success on psychometric characteristics of the test. First, the data was cleared of chance success by means of correction-for-guessing formula and self-assessment, and then statistical analyses were conducted. Item discriminations showed an increase when the correction-for-guessing formula was used; and when self-assessment was used, they showed variability. Test validity increased when correction formula was used; and when self-assessment was used, a slight decrease was observed. Besides, this study examined the effect of correction for chance success upon corrected self-assessment based on IRT guessing parameter. It was observed that the data that were not corrected in accordance with chance scores had higher guessing parameters than those corrected in accordance with self-assessment. In addition, it was evident that the difference between the guessing parameters of the uncorrected data and the data cleared of chance scores by means of self-assessment was significant. It was also revealed that the correction of self-assessment-based chance success have an advantage over classical correction for guessing formula on psychometric characteristics of the test.

Downloads

Download data is not yet available.

References

Araz, G. (2001). Aynı davranışı ölçmeye yönelik kısa cevaplı, üç seçenekli ve beş seçenekli çoktan seçmeli testlerin madde ve test özelliklerinin şans başarısı ile birlikte incelenmesi. Unpublished master's thesis, Hacettepe Üniversitesi Sosyal Bilimler Enstitüsü, Ankara.
Atılgan, H., Kan, A., & Doğan, N. (2009). Çoktan seçmeli testler. In H. Atılgan (Ed), Eğitimde ölçme ve değerlendirme (pp.223-268). Ankara: Anı Yayıncılık.
Baykul, Y. (2010). Eğitimde ve psikolojide ölçme: klasik test teorisi ve uygulaması. Ankara: Pegem Akademi.
Ben-Simon, A., Budescu, D., & Nevo, B. (1997). A comparative study of measures of partial knowledge in multiple-choice tests. Applied Psychological Measurement, 21(1), 65-88.
Boud, D. & Falchikov, N. (1989). Quantitative studies of student self-assessment in higher education: a critical analysis of findings. Higher Education, 18(5), 529-549.
Budescu, D. & Bar-Hillel, M. (1993). To guess or not to guess: A decision-theoretic view of formula scoring. Journal of Educational Measurement, 30(4), 277-291.
Burton, R. F. (2001a). Quantifying the effects of chance in multiple choice and true/false tests: Question selection and guessing of answers. Assessment & Evaluation in Higher Education, 26(1), 41-50.
Burton, R. F. (2001b). Do item discrimination indices really help us to improve our tests?. Assessment and Evaluation in Higher Education, 26(3), 213-220.
Büyüköztürk, Ş., Çokluk, Ö., & Köklü, N. (2010). Sosyal bilimler için istatistik. Ankara: Pegem Akademi.
Byram, M. (1997). Teaching and assessing intercultural communicative competence. Clevedon: Multilingual Matters.
Çelen, Ü. (2002). Şans başarısı için düzeltme formülü kullanılacağına ilişkin yönergenin testin psikometrik özelliklerine etkisinin araştırılması. Unpublished master's thesis, Ankara Üniversitesi Eğitim Bilimleri Enstitüsü, Ankara.
Dochy, F. & McDowell, L. (1997). Assessment as a tool for learning. Studies in Educational Evaluation, 23, 279-298.
Doğan, N., Çetin, S., & Gelbal, S. (2007). The use of self-assessment in improving guessing. Paper presented at the The Jubilee International Scientific Conference under the heading Science, Education and Time As Our Concern, Plovdiv University, Smolyan, Bulgaristan.
Karasar, N. (2012). Bilimsel araştırma yöntemi. Ankara: Nobel Yayıncılık.
Kurz, T. B. (1999). A review of scoring algorithms for multiple-choice tests. Paper presented at the Southwest Educational Reaserch Association, Texas A M University, San Antonio, ABD.
Mehrens, W. A. & Lehmann, I. J. (1984). Measurement and evaluation in education and psychology. New York: Holt, Rinehart and Winston Publishing.
Şahhüseyinoğlu, D. B. (1998). Sayısal yetenek testlerinde seçenek sayısının test ve madde istatistikleri üzerindeki etkisinin şans başarısı ile birlikte incelenmesi. Unpublished master's thesis, Hacettepe Üniversitesi Sosyal Bilimler Enstitüsü, Ankara.
Telli, A. (1993). Şans başarısının madde türlerindeki madde ve test istatistiklerine etkisi. Unpublished master's thesis, Hacettepe Üniversitesi Sosyal Bilimler Enstitüsü, Ankara.
Turgut, M. F. (1971). Şans başarısının test puvanlarına etkisi. Ankara: Başnur Matbaası.

Downloads

Published

2018-04-29

How to Cite

Özdoğan, D., & Doğan, N. (2018). The effect of the correction of self-assessment-based chance success on psychometric characteristics of the test. Pegem Journal of Education and Instruction, 8(3), 567–598. https://doi.org/10.14527/pegegog.2018.022

Issue

Vol. 8 No. 3 (2018): Pegem Eğitim ve Öğretim Dergisi

Section

Article

License

Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.

NonCommercial — You may not use the material for commercial purposes.

No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.