Can innovative learning affect student HOTS achievements?: A meta-analysis study


  • Duden Saepuzaman Universitas Pendidikan Indonesia
  • Heri Retnawati
  • Edi Istiyono
  • Haryanto haryanto



HOTS, innovative learning, meta-analysis


This study looks at innovative learnings’ effect on achieving students’ High Order Thinking Skills (HOTS). HOTS is the latest issue that students need to develop in the 21st Century. This research is a quantitative study with a meta-analysis approach to the mean with random-effects models. Generally, the steps involve formulating research problems meta-analysis to be performed, collecting studies, calculating the effect size, test of heterogeneity, data analysis (summary effect, forest plots, funnel plots, test the potential for publication bias), and concludes. The research sample of 42 research includes international journals and national, international, and nationwide proceedings and a thesis student. The analysis using random models (random-effect model) obtained a mean value of the effect size of aggregation or summary effect amounted to 77.37, standard error weighs the mean 12.36, the upper limit of 53.14, and the lower limit of 101.60. Analysis of the null hypothesis (Ho: actual effect ???? = ????) leads to rejecting Ho because the ???? value of the summary effect is 77.37 with the one-tailed p-value (0.00) smaller than the α value (0.05). Analysis of publication bias tests from Funnel Plot, Rank Correlation, Regression Method outputs, and “Trim and Fill” shows no potential publication bias regarding conclusions drawn. So the findings made based on the random effect model on the application of innovative learning affect HOTS students achievement are valid. The implication is that teachers or other practitioners can use innovative learning to improve students’ HOTS.


Download data is not yet available.


Akyol, Z., & Garrison, D. R. (2011). Understanding cognitive presence in an online and blended community of inquiry: Assessing outcomes and processes for deep approaches to learning. British Journal of Educational Technology, 42(2), 233–250.

Anazifa, R. D., & Djukri. (2017). Project- based learning and problem- based learning: Are they effective to improve student’s thinking skills? Jurnal Pendidikan IPA Indonesia, 6(2), 346–355.

Anderson, L. W., & Krathwohl, D. R. (2001). A revision of Bloom’s taxonomy of educational objectives. A Taxonomy for Learning, Teaching and Assessing. Longman, New York.

Apino, E., & Retnawati, H. (2017). Developing Instructional Design to Improve Mathematical Higher Order Thinking Skills of Students. Journal of Physics: Conference Series, 812(1), 12100.

Apino, Ezi, & Retnawati, H. (2016). Creative Problem Solving to Improve Students ’ Higher Order Thinking Skills in Mathematics Instructions. Proceeding of 3Rd International Conference on Research, Implementation and Education of Mathematics and Science, May, 339–346.

Bohan, H., & Bohan, S. (2020). Extending the Regular Curriculum Through Creative Problem Solving. The Arithmetic Teacher, 41(2), 83–87.

Brookhart, S. M. (2010). How to assess higher-order thinking skills in your classroom. ASCD.

Çoğaltay, N., & Karadağ, E. (2015). Introduction to meta-analysis. In Leadership and organizational outcomes (pp. 19–28). Springer.

Conklin, W. (2012). Higher-Order Thinking Skills to Develop 21st Century Learners. In Shell Education. Teacher Created Materials.

Crumb, L. N. (1983). Taxonomy Of Educational Objectives The Classification of Educational Goals. In Cataloging and Classification Quarterly (Vol. 3, Issue 1). Ann Arbor.

Dickersin, K., Chan, S., Chalmersx, T. C., Sacks, H. S., & Smith, H. (1987). Publication bias and clinical trials. Controlled Clinical Trials, 8(4), 343–353.

Djidu, H., & Jailani. (2016). Aktivitas Pembelajaran Matematika yang Dapat Melatih Kemampuan Berpikir Tingkat Tinggi Siswa. Seminar Nasional Matematika X Universitas Negeri Semarang, 314.

Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455–463.

Fisher, R. (2010). Thinking Skill Strategies.pdf. In D. J. Arthur & T. Cremin (Eds.), Learning to Teach in The Primary School (2nd ed., pp. 374–387). Routledge.

Freeman, P. R., Hedges, L. V., & Olkin, I. (1986). Statistical Methods for Meta-Analysis. In Biometrics (Vol. 42, Issue 2). Academic press.

Goethals, P. L. (2013). The Pursuit of Higher-Order Thinking in the Mathematics Classroom. United States Military Academy, West Point, NY.

Hadi, S., Retnawati, H., Munadi, S., Apino, E., & Wulandari, N. F. (2018). The difficulties of high school students in solving higher-order thinking skills problems. Problems of Education in the 21st Century, 76(4), 520.

Indiani, V., & Retnawati, H. (2017). Need Assessment To Develop Teaching Set For Improving Higher Order Thinking Skills of Junior High School Students In Learning Mathematics. Proceeding the International Conference on Education Innovation, 1(1), 848–853.

Jailani, & Retnawati, H. (2016). Keefektifan Pemanfaatan Perangkat Pembelajaran Berbasis Masalah untuk Meningkatkan HOTS dan Karakter Siswa. Jurnal Pendidikan Dan Pembelajaran, 23(2), 111–123.

King, F. J., Goodson, L., & Rohani, F. (2010). Higher order thinking skills: Definition, Teaching Strategies, Assessment.

Koza Çiftçi, & Yıldız, P. (2019). The effect of gender on algebra achievement: The meta-analysis of trends in international mathematics and science study (TIMSS). Turkish Journal of Computer and Mathematics Education, 10(3), 617–627.

Krulik, S., & Rudnick, J. A. (1999). Innovative tasks to improve critical and creative thinking skills. In D. L. V Stiff & F. R. Curcio (Eds.), Developing Mathematical Reasong in Grades K-12 (pp. 138–145). NCTM.

Limbach, B., & Waugh, W. (2010). Developing higher level thinking. Journal of Instructional Pedagogies, 3, 9.

Liu, X. (2014). Essentials of Science Classroom Assessment. In Essentials of Science Classroom Assessment. Sage Publications.

Loewen, A. C. (1995). Creative problem solving. Teaching Children Mathematics, 2(2), 96–99.

Miri, B., David, B. C., & Uri, Z. (2007). Purposely teaching for the promotion of higher-order thinking skills: A case of critical thinking. Research in Science Education, 37(4), 353–369.

Mohd Zin Mokhtar, Rohani Ahmad Tarmizi, Ahmad Fauzi Mohd Ayub, & Nawawi, M. D. H. (2013). Motivation and Performance In Learning Calculus Through Problem-Based Learning. International Journal of Asian Social Science, 3(9), 1999–2005.

Moseley, D., Elliott, J., Gregson, M., & Higgins, S. (2005). Thinking skills frameworks for use in education and training. British Educational Research Journal, 31(3), 367–390.

Orlich, D. C., Harder, R. J., Callahan, R. C., Trevisan, M. S., & Brown, A. H. (2010). Teaching Strategies-Guide to Effective Instruction. Wadsworth Engage Learning.

Pigott, T. (2012). Advances in meta-analysis. Springer Science & Business Media.

Presseisen, B. Z. (1988). Thinking skills: Meanings and models. In D. A. L. Costa (Ed.), Developing minds: A resource book for teaching thinking (pp. 43–48). ASCD.

Retnawati, H. (2018). Pengantar Analisis Meta. Parama Publishing.

Retnawati, H., Djidu, H., Kartianom, Apino, E., & Anazifa, R. D. (2018). Teachers’ knowledge about higher-order thinking skills and its learning strategy. Problems of Education in the 21st Century, 76(2), 215–230.

Rochani, S. (2016). Keefektifan pembelajaran matematika berbasis masalah dan penemuan terbimbing ditinjau dari hasil belajar kognitif kemampuan berpikir kreatif. Jurnal Riset Pendidikan Matematika, 3(2), 273.

Sumarmo, U., & Nishitani, I. (2010). High Level Mathematical Thinking: Experiments with High School and Under Graduate Students using Various Approaches and Strategies. In [Natural Sciences Edition] (Vol. 58, pp. 9–22).

Thomas, A., & Thorne, G. (2009). How to increase higher order thinking. In Metarie, LA: Center for Development and Learning. Center for Development and Learning.(accessed on 6.

Thompson, T. (2008). Mathematics teachers’ interpretation of higher-order thinking in Bloom’s taxonomy. International Electronic Journal of Mathematics Education, 3(2), 96–109.

Thompson, T. (2011). An Analysis of Higher-Order Thinking on Algebra I End-of Course Tests. International Journal for Mathematics Teaching and Learning, c, 1–36.

Turgut, S., & Turgut, I. G. (2018). The effects of cooperative learning on mathematics achievement in Turkey: A meta-analysis study. In International Journal of Instruction (Vol. 11, Issue 3, pp. 663–680).

Vidergor, H. E., & Krupnik-Gottlieb, M. (2015). High order thinking, problem based and project based learning in blended learning environments. In Applied Practice for Educators of Gifted and Able Learners (pp. 217–232). Brill Sense.




How to Cite

Saepuzaman, D., Retnawati, H. ., Istiyono, E., & haryanto, H. (2021). Can innovative learning affect student HOTS achievements?: A meta-analysis study. Pegem Journal of Education and Instruction, 11(4), 290–305.