Higher-order thinking test of science for college students using multidimensional item response theory analysis

Authors

DOI:

https://doi.org/10.47750/pegegog.12.01.30

Keywords:

higher-order thinking skills, science, item response theory

Abstract

The purpose of this study was to construct a higher order thinking test of science for pre-service elementary school teachers. The test was created using the ADDIE model. The Analyze stage was carried out by identifying the needs and baseline of higher order thinking skills of students from the Department of Primary School Teacher Education in Yogyakarta. The Design stage involved the creation of test blueprints and question cards. The Develop stage involved validating the test’s content and construct validity. The content validity test was conducted using the Delphi technique with seven validators, whilst the construct validity test was conducted using item response theory and EFA. This study developed 77 questions, 73 multiple choice questions and four essay questions, all of which were determined to be valid in terms of content and constructions. The HOTS test’s content validity test resulted in a V-value of 0.879 (valid with high criteria) based on the average Aiken’s V index. Meanwhile, reliability analysis using the Cronbach’s Alpha coefficient revealed a score of 0.907 for the 77 test items based on the construct validity test. The discriminatory index (di) classified all items as good, whereas the difficulty index (bi) classified 63 items as good and 10 as poor. The ten items were revised, despite their high index of difference. All of the test questions are appropriate for students whose ability score (θ) ranged from -2.85 to 2.15.

Downloads

Download data is not yet available.

References

Abdullah, A. H., Ali, M., Liyana, N., & Abidin, Z. (2015). Analysis of students’ errors in solving higher order thinking skills (HOTS) problems for the topic of fraction. Asian Social Science, 11(21), 133–142. https://doi.org/10.5539/ass.v11n21p133

Alice Thomas, & Glenda Thorne. (2009). How To Increase Higher Level Thinking | Center for Development and Learning. The Center for Development And Learning. http://www.cdl.org/articles/how-to-increase-high-order-thinking/

Anderson, C. A., & Krathwohl, D. R. (2014). Bloom’s Taxonomy of Educational Objectives. Center for Innovation in Teaching and Learning, 1–2.

Anderson, L. W., Krathwohl, D. R., & Airasian, P. W. (2000). A Taxonomy for learning, teaching, and assessing: a revision of bloom’s taxonomy of educational objectives. http://digilib.uns.ac.id/pengguna.php?mn=showview&id=21967

Andriani, D. E. (2010). Mengembangkan Profesionalitas Guru Abad-21 Melalui Program Pembimbingan yang Efektif. Jurnal Manajemen Pendidikan, 6(2), 78–92. http://journal.uny.ac.id/index.php/jmp/article/view/3639%7B%25%7D0Ahttp://journal.uny.ac.id/index.php/jmp/article/download/3639/3112

Atmojo, I., Sajidan, P., Sunarno, W., & Ashadi, M. (2017). Profile of elementary school pre-service teacher based on high order thinking skills (HOTS) on natural science subject. Advances in Social Science, Education and Humanities Research (ASSEHR), 158, 501–504. https://doi.org/10.2991/ICTTE-17.2017.57

Branch, R. M. (2010). Instructional Design: The ADDIE Approach. Springer US. https://doi.org/10.1007/978-0-387-09506-6

Brookhart, S. M., Bookhart, S. M., & Brookhart, S. M. (2010). How to assess Higher-Order Thinking Skills in Your Classroom. ASCD. http://www.ascd.org/publications/books/109111.aspx

Darling-Hammond, L. (2006). Constructing 21st-Century Teacher Education. Journal of Teacher Education, 57(3), 300–314. https://doi.org/10.1177/0022487105285962

Dubas, J. M., & Toledo, S. A. (2016). Taking higher order thinking seriously: using marzano’s taxonomy in the economics classroom. International Review of Economics Education, 21, 12–20. https://doi.org/10.1016/j.iree.2015.10.005

Faizah, K. (2016). Miskonsepsi dalam pembelajaran IPA. Darussalam: Jurnal Pendidikan Komunikasi Dan Pemikiran Hukum Islam, 8(1), 113–125.

Fajriyah, K., & Agustini, F. (2018). Analisis Keterampilan BerpikirTingkat Tinggi Siswa SD Pilot ProjectKurikulum 2013 Kota Semarang. Elementary School: Jurnal Pendidikan Dan Pembelajaran Ke-SD-An, 5(1), 54–67. https://doi.org/10.31316/esjurnal.v5i1

Furr, R. M., & Bacharach, V. R. (2008). Psychometrics. An introduction. Sage.

Gradini, E., Firmansyah, F., & Noviani, J. (2018). Menakar kemampuan berpikir tingkat tinggi calon guru matematika melalui level HOTS Marzano. Eduma : Mathematics Education Learning and Teaching, 7(2), 41–48. https://doi.org/10.24235/eduma.v7i2.3357

Heong, Y. M., Othman, W. B., Yunos, J. B. M., Kiong, T. T., Hassan, R. Bin, & Mohamad, M. M. B. (2011). The Level of Marzano Higher Order Thinking Skillsamong Technical Education Students. International Journal of Social Science and Humanity, 1(2), 121–125. https://doi.org/10.7763/ijssh.2011.v1.20

Heong, Y. M., Sern, L. C., Kiong, T. T., & Binti Mohamad, M. M. (2016). The Role of Higher Order Thinking Skills in Green Skill Development. MATEC Web of Conferences, 70, 1–5. https://doi.org/10.1051/matecconf/20167005001

Kuntarto, E., Alirmansyah, A., & Kurniawan, A. R. (2019). Kemampuan mahasiswa PGSD dalam merancang dan melaksanakan pembelajaran berbasis high order of thinking skills. Jurnal Kiprah, 7(2), 107–116. https://doi.org/10.31629/KIPRAH.V7I2.1454

Lewy. (2011). Pengembangan Soal Untuk Mengukur Kemampuan Berpikir Tingkat Tinggi Pokok Bahasan Barisan dan Deret Bilangan di Kelas IX Akselerasi SMP Xaverius Maria Palembang. Jurnal Pendidikan Matematika, 5(1), 58–73. https://doi.org/10.22342/jpm.5.1.821

Maryani, I., Husna, N. N., Wangid, M. N., Mustadi, A., & Vahechart, R. (2018). Learning difficulties of the 5thgrade elementary school students in learning human and animal body organs. Jurnal Pendidikan IPA Indonesia, 7(1). https://doi.org/10.15294/jpii.v7i1.11269

Marzano, R. J. (1993). How classroom teachers approach the teaching of thinking. Theory Into Practice, 32(3), 154–160. https://doi.org/10.1080/00405849309543591

Marzano, R. J., & Kendall, J. S. (2006). The new taxonomy of educational objectives (2nd ed.). Corwin Press.

Mehrens, W. A., & Lehmann, I. J. (1991). Measurement and evaluation in education and psychology. Harcourt Brace College Publishers.

Pappas, C. (2013). 8 important characteristics of adult learners. Elearning Design and Development. https://elearningindustry.com/8-important-characteristics-of-adult-learners

Ramos, J. L. S., Dolipas, B. B., & Villamor, B. B. (2013). Higher Order Thinking Skills and Academic Performance in Physics of College Students : A Regression Analysis. International Journal of Innovative Interdisciplinary Research, 4(1), 48–60. https://doi.org/ISSN 1839‐9053

Surya, A., Sularmi, S., Istiyati, S., & Prakoso, R. F. (2018). Finding HOTS-base mathematical learning in elementary school students. Social, Humanities, and Educational Studies (SHEs): Conference Series, 1(1), 30–37. https://doi.org/10.20961/shes.v1i1.24308

Suto, I. (2013). 21 st Century skills : Ancient , ubiquitous , enigmatic ?

Toledo, S., & Dubas, J. M. (2016). Encouraging Higher-Order Thinking in General Chemistry by Scaffolding Student Learning Using Marzano’s Taxonomy. Journal of Chemical Education, 93(1), 64–69. https://doi.org/10.1021/acs.jchemed.5b00184

Trisdiono, H. (2013). Strategi pembelajaran abad 21.

Utomo, A. P., Narulita, E., & Shimizu, K. (n.d.). Diversification of reasoning science test items of TIMSS grade 8 based on higher-order thinking skills: a case study of Indonesian students.

Webb, C. (2002). The Motivational Enhancement Therapy and Cognitive Behavorial Therapy Supplement: 7 Sessions of Cognitive Behavioral Therapy for Adolescent Cannabis Users. US Department of Health and Human Services, Substance Abuse and Mental Health.

Widarjono, A. (2015). Analisis Multivariat Terapan dengan Program SPSS, AMOS, dan SMARTPLS, II. UPP STIM YKPN.

Wiyoko, T., & Aprizan, A. (2020). Analisis profil kemampuan kognitif mahasiswa PGSD pada mata kuliah ilmu alamiah dasar. IJIS Edu : Indonesian Journal of Integrated Science Education, 2(1), 2020. https://doi.org/10.29300/ijisedu.v2i1.2384

Yee, M. H., Othman, W., Md Yunos, J., Tee, T. K., Hassan, R., & Mohamad, M. M. (2011). The level if Marzano higher order thinking skills among technical education students. International Journal of Social Science and Humanity. http://merr.utm.my/id/eprint/1589

Yusuf, I., Widyaningsih, W., & Sebayang, R. B. (2018). Implementation of e-learning based-STEM on quantum physics subject to student HOTS ability. Journal of Turkish Science Education, 15(Special), 67–75. https://doi.org/10.12973/tused.10258a

Zulfiani, Z., Suwarna, I. P., & Sumantri, M. F. (2020). Science adaptive assessment tool: Kolb’s learning style profile and student’s higher order thinking skill level. Jurnal Pendidikan IPA Indonesia, 9(2), 194–207. https://doi.org/10.15294/JPII.V9I2.23840

Downloads

Published

2022-01-02

How to Cite

Maryani, I., Kun Prasetyo, Z., Wilujeng, I., & Purwanti, S. . (2022). Higher-order thinking test of science for college students using multidimensional item response theory analysis. Pegem Journal of Education and Instruction, 12(1), 292–300. https://doi.org/10.47750/pegegog.12.01.30

Issue

Section

Article