Validity and reliability study of the Web Based Listening Comprehension Test

  • M. Bahaddin Acat Osmangazi University
  • Hilmi Demiral
  • Mehmet Fatih Kaya
Keywords: Listening comprehension, web based listening, listening comprehension test, Turkish instruction

Abstract

The purpose of this study is to examine validity and reliability of the web based test designed to measure listening skills of the 5th graders. The test's content validity supplied 27 of 36 listening objectives described in the curriculum for the 5th graders' Turkish course. After, the constructs were prepared for measuring if the students can gain the listening objectives. Firstly 56 items were prepared. Then two experts working on the fields of Turkish instruction and two experts working on the fields of measurement and evaluation were asked for their opinion about content validity. After making necessary corrections proposed by the experts, eventually, 41 items were left in the test and pilot studies were conducted. KR-20 and Cronbach Alpha coefficients were estimated both being as .77 in an attempt to determine internal consistency of the test. The reliability coefficient calculated by means of Spearman-Brown prophecy formula was found as .80. Such findings point out that the test is reliable. It was also observed during the study that students tend to have higher motivation and concentration level when they engage in web based systems. Therefore, it could be said that web-based environments could be used for measuring and evaluating language skills.

Downloads

Download data is not yet available.

References

Akbaba, B. (2009). Atatürk İlkeleri ve İnkılâp Tarihi öğretiminde çoklu ortam kullanımının akademik başarı ve tutumlara etkisi. Yayımlanmamış Doktora Tezi. Gazi Üniversitesi, Ankara.

Al-Amri, S. (2008). Computer-based testing vs. paper-based testing: A comprehensive approach to examining the comparability of testing modes. Essex Graduate Student Papers in Language & Linguistics, 10, 22-44.

Alderson, J. C. (2000). Technology in testing: The present and the future. System, 28, 593-603. DOI: https://doi.org/10.1016/S0346-251X(00)00040-3

Aybek, E. C. (2012). Kâğıt - kalem formu ve bilgisayar ortamında uygulanan genel yetenek testinin psikometrik özelliklerinin karşılaştırılması. Yayımlanmamış Yüksek Lisans Tezi. Ankara Üniversitesi, Ankara.

Bahar, M., Nartgün, Z., Durmuş, S. ve Bıçak, B. (2010). Ölçme ve değerlendirme teknikleri-Öğretmen el kitabı. Ankara: Pegem Akademi.

Boo, J. (1997). Computerized versus paper-and-pencil assessment of educational development: Score comparability and examinee preferences. Unpublished PhD Dissertation, University of Iowa.

Brown, J. D. (1997). Computers in language testing: Present research and some future directions. Language Learning & Technology, 1(1), 44-59. http://llt.msu.edu/vol1num1/brown/default.html internet adresinden 17.08.2011 tarihinde edinilmiştir.

Bugbee, A. (1996) The equivalence of paper-and-pencil and computer-based testing. Journal of Research on Computing in Education, 28(3), 282-300. DOI: https://doi.org/10.1080/08886504.1996.10782166

Büyüköztürk, Ş. (2010). Sosyal bilimler için veri analizi el kitabı. Ankara: Pegem Akademi.

Büyüköztürk, Ş., Kılıç Çakmak, E., Akgün, Ö. E., Karadeniz, Ş. ve Demirel, F. (2010). Bilimsel araştırma yöntemleri. Ankara: Pegem Akademi.

Chalhoub-Deville, M. (2001). Language testing and technology: past and future. Language learning & technology, 5(2), 95-98. http://llt.msu.edu/vol5num2/deville/default.html internet adresinden 15.06.2012 tarihinde edinilmiştir.

Chalhoub-Deville, M. ve Deville, C. (1999). Computer adaptive testing in second language contexts. Annual Review of Applied Linguistics, 19, 273-299. DOI: https://doi.org/10.1017/S0267190599190147

Chen, L. ve Zhang, R. (2011). Web-based CALL to listening comprehension. Current Issues in Education, 13(4). http://cie.asu.edu/ojs/index.php/cieatasu/article/view/286 internet adresinden 11.06.2012 tarihinde edinilmiştir.

Choi, I., Kim, K. ve Boo, J. (2003) Comparability of a paper-based language test and a computer-based language test. Language Testing, 20(3), 295-320. DOI: https://doi.org/10.1191/0265532203lt258oa

Çifçi, M. (2006). Türkçe öğretiminin sorunları. G. Gülsevin ve E. Boz, (Ed.), Türkçenin çağdaş sorunları içinde (77-134). Ankara: Gazi.

Ergün, E. (2002). Üniversite öğrencilerinin bilgisayar destekli ölçmeden elde ettikleri başarının kalem-kâğıt testi başarısı, bilgisayar kaygısı ve bilgisayar tecrübeleri açısından incelenmesi. Yayımlanmamış yüksek lisans tezi. Anadolu Üniversitesi, Eskişehir.

Erkuş, A. (2009). Psikometri üzerine yazılar. Ankara: Türk Psikologlar Derneği Yayınları.

Giri, R. A. (1998). Listening skills for young learners. Journal of Nelta, 3(1-2), 1-5.

Gül, İ. (1998). İlköğretim okullarında Türkçe öğretimi. Samsun: Sönmez Ofset.

Hoven, D. (1999). A model for listening and viewing comprehension in multimedia environments. Language Learning and Technology, 3(1), 88-103. http://llt.msu.edu/vol3num1/hoven/index.html internet adresinden 19.06.2012 tarihinde edinilmiştir.

Johnson, N. ve Green, S. (2004). On-line assessment: The impact of mode on students' performance. The British Educational Research Association Annual Conference, Manchester, UK.

Karasar, N. (2009). Bilimsel araştırma yöntemi. Ankara: Nobel Yayın Dağıtım.

KavaliauskienÄ—, G. (2008). Podcasting: a tool for improving listening skills. Teaching English with technology: A Journal for Teachers of English, 3(4), http://www.iatefl.org.pl/call/j_techie33.htm.

Mead, A. D., ve Drasgow, F. (1993). Equivalence of computerized and paper and-pencil cognitive ability tests: a meta-analysis. Psychological Bulletin, 114(3), 449'458. DOI: https://doi.org/10.1037/0033-2909.114.3.449

MEB (2005). İlköğretim Türkçe dersi öğretim programı ve kılavuzu (1-5.sınıflar). Ankara: MEB.

Oral, B. (2005). İnternet ve eğitim. A. Tarcan, (Ed.), İnternet ve toplum içinde (90-116). Ankara: Anı.

Özbay, M. (2010). Anlama teknikleri II: Dinleme eğitimi. Ankara: Öncü.

Özçelik, D. (2010). Test hazırlama kılavuzu. Ankara: Pegem Akademi.

Özçelik, D. (2011). Ölçme ve Değerlendirme. Ankara: Pegem Akademi.

Pala, O. (2005). Teaching listening and reading comprehension in Turkish using web-based materials. The Final Project, University of Oregon. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.124.8608&rep=rep1&type=pdf internet adresinden 11.06.2012 tarihinde edinilmiştir.

Pardo-Ballester, C. (2008). The construct validity of a web-based listening comprehension exam. C. A. Chapelle, Y. R. Chung ve J. Xu (Eds.), In Towards adaptive CALL: Natural language processing for diagnostic language assessment,(209-227). Ames, IA: Iowa State University.

Programme for International Student Assessment (2010). PISA computer-based assessment of student skills in science. OECD Publications.

Pomplun, M., Frey, S. ve Becker, D. (2002). The score equivalence of paper-and-pencil and computerized versions of a speeded test of reading comprehension. Educational and Psychological Measurement, 62(2), 337-354. DOI: https://doi.org/10.1177/0013164402062002009

Roever, C. (2001). Web-based language testing. Language Learning & Technology, 5(2), 84-94. http://llt.msu.edu/vol5num2/roever/default.html internet adresinden 06.06.2012 tarihinde edinilmiştir.

Sawaki, Y. (2001) Comparability of conventional and computerized tests of reading in a second language. Language Learning & Technology, 5(2), 38-59.

Shin, S. Y. (2008). Examining the construct validity of a web-based academic listening test: an investigation of the effects of response formats. The Spaan Fellowship Working Papers in Second or Foreign Language, 6, 95-129. English Language Institute. University of Michigan. http://www.cambridgemichigan.org/sites/default/files/resources/SpaanPapers/Spaan_V6_Shin.pdf internet adresinden 18.06.2012 tarihinde edinilmiştir.

Smith, B., Caputi, P. ve Rawstorne, P. (2000). Differentiating computer experience and attitudes toward computers: an empirical investigation. Computers in Human Behavior, 16, 59-81. DOI: https://doi.org/10.1016/S0747-5632(99)00052-7

Wang, X. (2006). Evaluating a web-based listening programme for Chinese University non-English majors. US-China Education Review, 3(5), 74-76.

Wang, Y. (2010). To give control to learners or not? A comparative study of two ways of teaching listening. English Language Teaching, 3(2), 162-174. DOI: https://doi.org/10.5539/elt.v3n2p162

Whitworth, B. (2001). Equivalency of paper-and-pencil tests and computer-administered tests. Unpublished PhD dissertation, University of North Texas.

Yangın, B. (1999). İlköğretimde etkili öğretme ve öğrenme öğretmen el kitabı modül 4, İlköğretimde Türkçe öğretimi. Ankara: Milli Eğitim Bakanlığı.

Published
2013-09-01
How to Cite
Acat, M. B., Demiral, H., & Kaya, M. F. (2013). Validity and reliability study of the Web Based Listening Comprehension Test. Pegem Eğitim Ve Öğretim Dergisi, 3(3), 01-10. https://doi.org/10.14527/C3S3M1