A Literature Review Of Multimodal Semiotic Reasoning In Mathematics





reasoning, semiotic, multimodal, mathematics education


Multimodal semiotic reasoning is a process of drawing conclusions related to objects, symbols or signs through the integration of various resources originating from written answer texts, oral answers, gestures, facial expressions, pictures, concrete objects and artifacts that students use. This research is a literature study that focuses on research articles on multimodal semiotic reasoning using the subject of mathematics. The research was conducted by reviewing several articles that concentrated on research design, the type of participants, the focus of the research material and the role of multimodal semiotic reasoning with the subject of mathematics. A total of 22 articles were analyzed by searching and reviewing articles from trusted database journals, namely Scopus, Science Direct, and ERIC. The literature review of multimodal semiotic reasoning is explored based on 1) the type of research design used in research, 2) the type of participants selected in research, 3) the focus of the research material, and 4) the role of multimodal semiotic reasoning. Based on the results of the manuscript review, multimodal semiotic reasoning research shows a great influence on mathematics education. Research on multimodal semiotic reasoning shows that 1) descriptive research and case studies are the most commonly used designs; 2) the study participants were dominated by high school and elementary school students; 3) the focus of most research material is in the field of mathematics, especially geometry, 4) the role of multimodal semiotic reasoning in mathematics education is mostly shown in knowledge of mathematics.


Download data is not yet available.

Author Biographies

Toto, Universitas Negeri Malang

Mathematics Department

makbul, Universitas Negeri Malang

Mathematics Department

Dwi Agus, Universitas Negeri Malang

mechanical Engineering Department


Bartocci, C. (2013). “Reasoning well from badly drawn figures”: the birth of algebraic topology. Lettera Matematica, 1(1–2), 13–22. https://doi.org/10.1007/s40329-013-0010-4

Bergvall, I., & Dyrvold, A. (2021). A Model for Analysing Digital Mathematics Teaching Material from a Social Semiotic Perspective. Designs for Learning, 13(1), 1–7. https://doi.org/10.16993/dfl.167

Björklund Boistrup, L., & Gustafsson, L. (2014). Construing mathematics-containing activities in adults’ workplace competences: Analysis of institutional and multimodal aspects. Adults Learning Mathematics: An International Journal, 9(1), 7–23.

Campbell, C. (2019). Educating Semiosis: Foundational Concepts for an Ecological Edusemiotic. Studies in Philosophy and Education, 38(3), 291–317. https://doi.org/10.1007/s11217-018-9617-4

Chahine, I. C. (2013). The impact of using multiple modalities on students’ acquisition of fractional knowledge: An international study in embodied mathematics across semiotic cultures. Journal of Mathematical Behavior, 32(3), 434–449. https://doi.org/10.1016/j.jmathb.2013.04.004

Chalkiadaki, A. (2018). A systematic literature review of 21st century skills and competencies in primary education. International Journal of Instruction, 11(3), 1–16. https://doi.org/10.12973/iji.2018.1131a

Chen, C.-L., & Herbst, P. (2013). The interplay among gestures, discourse, and diagrams in students’ geometrical reasoning. Educational Studies in Mathematics, 83(2), 285–307. https://doi.org/10.1007/s10649-012-9454-2

Clark-Wilson, A., Robutti, O., & Thomas, M. (2020). Teaching with digital technology. ZDM - Mathematics Education, 52(7), 1223–1242. https://doi.org/10.1007/s11858-020-01196-0

Creswell, J. W. (2009). Third Edition Research Design: Qualitative, Quantitave, and Mixed Methods Approaches. In SAGE Publications, Inc. SAGE Publications, Inc. https://doi.org/10.2307/1523157

Crompton, H., Grant, M. R., & Shraim, K. Y. H. (2018). Technologies to enhance and extend children’s understanding of geometry: A configurative thematic synthesis of the literature. Educational Technology and Society, 21(1), 59–69.

Daher, W., & Abu Thabet, I. (2020). Social semiotics analysis of Palestinian mathematics textbooks for eighth grade. JRAMathEdu (Journal of Research and Advances in Mathematics Education), 5(1), 1–12. https://doi.org/10.23917/jramathedu.v5i1.8960

Danielsson, K., & Selander, S. (2021). Multimodal Texts in Disciplinary Education. In Multimodal Texts in Disciplinary Education. https://doi.org/10.1007/978-3-030-63960-0

Ernest, P., Skovsmose, O., Bendegem, J. P. van, Bicudo, M., Miarka, R., & Moeller, L. K. R. (2016). The Philosophy of Mathematics Education. ICME-13 Topical Surveys. In ICME-13 Topical Surveys. http://link.springer.com/10.1007/978-3-319-40569-8

Espeland, M., Smith, K., & Kvinge, Ø. (2018). Performing the Pre-Formed: Towards a Conceptual Framework for Understanding Teaching as Curricular Transformation. Designs for Learning, 10(1), 29–39. https://doi.org/10.16993/dfl.83

Fiantika, F. R., Maknun, C. L., Budayasa, I. K., & Lukito, A. (2018). Analysis of students’ spatial thinking in geometry: 3D object into 2D representation. Journal of Physics: Conference Series, 1013(1), 1–8. https://doi.org/10.1088/1742-6596/1013/1/012140

Freud, E., & Ganel, T. (2015). Visual control of action directed toward two-dimensional objects relies on holistic processing of object shape. Psychonomic Bulletin and Review, 22(5), 1377–1382. https://doi.org/10.3758/s13423-015-0803-x

Geesa, R. L., Izci, B., Song, H., & Chen, S. (2019). Exploring factors of home resources and attitudes towards mathematics in mathematics achievement in South Korea, Turkey, and the United States. Eurasia Journal of Mathematics, Science and Technology Education, 15(9). https://doi.org/10.29333/ejmste/108487

Hunt, T., Carper, J., Lasley, T., & Raisch, C. (2013). Trends in International Mathematics and Science Study (TIMSS). Encyclopedia of Educational Reform and Dissent, 562–569. https://doi.org/10.4135/9781412957403.n438

Jasinski, M.-A. (2012). Helping Children to Learn at Home: A Family Project to Support Young English-Language Learners. TESL Canada Journal, 29, 224. https://doi.org/10.18806/tesl.v29i0.1119

Jaynelle G. Domingo, Ibañez, E. D., Gener S. Subia, Pentang, J. T., M., A. G., Pascual, L. E., Mina, J. C., Tomas, A. V., & Liangco, M. M. (2021). Cognitive Skills Achievement in Mathematics of the Elementary Pre-Service Teachers Using Piaget’s Seven Logical Operations. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 12(4), 435–440. https://doi.org/10.17762/turcomat.v12i4.524

Karasavvvidis, I. (2019). Students’ use of digital video effects in multimodal compositions: an exploratory study. Journal of Visual Literacy, 38(3), 221–244. https://doi.org/10.1080/1051144X.2019.1611697

Kjällander, S, Mannila, L., Åkerfeldt, A., & Heintz, F. (2021). Elementary students’ first approach to computational thinking and programming. Education Sciences, 11(2), 1–15. https://doi.org/10.3390/educsci11020080

Kjällander, Susanne, Mannila, L., Åkerfeldt, A., & Heintz, F. (2021). Elementary students’ first approach to computational thinking and programming. Education Sciences, 11(2), 1–15. https://doi.org/10.3390/educsci11020080

Knain, E., Fredlund, T., & Furberg, A. (2021). Exploring Student Reasoning and Representation Construction in School Science Through the Lenses of Social Semiotics and Interaction Analysis. Research in Science Education, 51(1), 93–111. https://doi.org/10.1007/s11165-020-09975-1

Leduc, K., Conway, L., Gomez-Garibello, C., & Talwar, V. (2018). The influence of participant role, gender, and age in elementary and high-school children’s moral justifications of cyberbullying behaviors. Computers in Human Behavior, 83, 215–220. https://doi.org/10.1016/j.chb.2018.01.044

Magnani, L. (2006). Multimodal abduction: External semiotic anchors and hybrid representations. Logic Journal of the IGPL, 14(2), 107–136. https://doi.org/10.1093/jigpal/jzk009

Magnani, Lorenzo. (2015). The eco-cognitive model of abduction ’Aπαγσγη now: Naturalizing the logic of abduction. Journal of Applied Logic, 13(3), 285–315. https://doi.org/10.1016/j.jal.2015.04.003

Magnani, Lorenzo. (2016). The eco-cognitive model of abduction II: Irrelevance and implausibility exculpated. Journal of Applied Logic, 15, 94–129. https://doi.org/10.1016/j.jal.2016.02.001

Marshall, D., & Conana, H. (2021). Multimodality and New Materialism in Science Learning: Exploring Insights from an Introductory Physics Lesson. Education as Change, 25. https://doi.org/10.25159/1947-9417/8848

Martin, M. O., & Davier, M. Von. (2022). TIMSS 2023 Assessment Frameworks TIMSS 2023 Assessment Frameworks.

Moore-Russo, D., & Viglietti, J. M. (2012). Using the K 5 Connected Cognition Diagram to analyze teachers’ communication and understanding of regions in three-dimensional space. Journal of Mathematical Behavior, 31(2), 235–251. https://doi.org/10.1016/j.jmathb.2011.12.001

Moschkovich, J. N. (2015). Academic literacy in mathematics for English Learners. Journal of Mathematical Behavior, 40, 43–62. https://doi.org/10.1016/j.jmathb.2015.01.005

Mullis, I. V. ., Martin, M. O., Foy, P., & Hopper, M. (2015). Timss 2015 International Results in Mathematics. IEA: TIMSS & PIRLS International Study Center, 1–971. http://timss2015.org/timss-2015/science/student-achievement/distribution-of-science-achievement/

National Council of Teachers of Mathematics. (2000). Principles and Standards for school Mathematics. In The National Council of Teachers of Mathematics, Inc. 1906 Association Drive, Reston, VA 20191-9988 www.nctm.org (Vol. 59).

Nordin, A. K., & Björklund Boistrup, L. (2018). A framework for identifying mathematical arguments as supported claims created in day-to-day classroom interactions. Journal of Mathematical Behavior, 51(April), 15–27. https://doi.org/10.1016/j.jmathb.2018.06.005

Oti, A., & Crilly, N. (2021). Immersive 3D sketching tools: Implications for visual thinking and communication. Computers and Graphics (Pergamon), 94(April), 111–123. https://doi.org/10.1016/j.cag.2020.10.007

Paul Heacock (Ed.). (2009). Cambridge Academic Content Dictionary. Cambridge University Press. https://archive.org/details/cambridgeacademi00camb

Phelps, C., & Sperry, L. L. (2020). Children and the COVID-19 Pandemic. Psychological Trauma: Theory, Research, Practice, and Policy, 12, 73–75. https://doi.org/10.1037/tra0000861

Pratika, Y., Salahudin, S., Riyanto, D. W. U., & Ambarwati, T. (2021). Analysis of Pay Later Payment System on Online Shopping in Indonesia. Journal of Economics, Business, & Accountancy Ventura, 23(3), 329–339. https://doi.org/10.14414/jebav.v23i3.2343

Ramadhany, N. (2021). Analysis of Students Mathematical Reasoning Abilities on Number Topics. Psychology and Education Journal, 58(1), 4750–4755. https://doi.org/10.17762/pae.v58i1.1634

Ratna, H., Roemintoyo, R., & Usodo, B. (2020). The Role of Adversity Quotient in the Field of Education: A Review of the Literature on Educational Development. International Journal of Educational Methodology, 6(3), 507–515. https://doi.org/10.12973/ijem.6.3.507

Romero-Rodríguez, S., Moreno-Morilla, C., & García-Jiménez, E. (2021). The construction of cultural identities in migrant children: An approach based on collaborative ethnography. Revista de Investigacion Educativa, 39(2), 483–501. https://doi.org/10.6018/RIE.441411

Saens-Ludlow, A., & Gert Kadunz. (2015). Semiotic as a tools for learning mathematics. In Acta Universitatis Agriculturae et Silviculturae Mendelianae Brunensis (Vol. 53, Issue 9). http://publications.lib.chalmers.se/records/fulltext/245180/245180.pdf%0Ahttps://hdl.handle.net/20.500.12380/245180%0Ahttp://dx.doi.org/10.1016/j.jsames.2011.03.003%0Ahttps://doi.org/10.1016/j.gr.2017.08.001%0Ahttp://dx.doi.org/10.1016/j.precamres.2014.12

Saleh, M., Prahmana, R. C. I., Isa, M., & Murni. (2018). Improving the reasoning ability of elementary school student through the Indonesian realistic mathematics education. Journal on Mathematics Education, 9(1), 41–53. https://doi.org/10.22342/jme.9.1.5049.41-54

Sdrolia, C. (2018). The semiotic impulse : experimenting with Peirce ’ s diagrammatic love. Journal of Aesthetics & Culture, 10(3). https://doi.org/10.1080/20004214.2018.1435151

Seeger, F. (2011). On meaning making in mathematics education: Social, emotional, semiotic. Educational Studies in Mathematics, 77(2–3), 207–226. https://doi.org/10.1007/s10649-010-9279-9

Sonesson, G. (2019). On Mimicry, Signs and Other Meaning-Making Acts. Further Studies in Iconicity. Biosemiotics, 12(1), 99–114. https://doi.org/10.1007/s12304-018-9340-0

Stein, A., & Maier, E. (1995). Structuring collaborative information-seeking dialogues. Knowledge-Based Systems, 8(2–3), 82–93. https://doi.org/10.1016/0950-7051(95)98370-L

Suryaningrum, C. W., & Ningtyas, Y. D. W. K. (2019). Multiple representations in semiotic reasoning. Journal of Physics: Conference Series, 1315(1). https://doi.org/10.1088/1742-6596/1315/1/012064

Suryaningrum, Christine Wulandari, Dwi, Y., Kusuma, W., Susanto, H., & Irfan, M. (2020). Machine Translated by Google PENILAIAN SEMIOTIS MUNCUL DALAM KONSTRUKSI PROPERTI DARI SEBUAH PERSEGIATAN : STUDI KUOTIEN KESALAHAN Machine Translated by Google. 11(1), 95–110.

Suryaningrum, Christine Wulandari, Purwanto, Subanji, Susanto, H., Ningtyas, Y. D. W. K., & Irfan, M. (2020). Semiotic reasoning emerges in constructing properties of a rectangle: A study of adversity quotient. Journal on Mathematics Education, 11(1), 95–110. https://doi.org/10.22342/jme.11.1.9766.95-110

Szkudlarek, T. (2011). Semiotics of Identity: Politics and Education. Studies in Philosophy and Education, 30(2), 113–125. https://doi.org/10.1007/s11217-011-9225-z

Turgut, M. (2021). Reinventing Geometric Linear Transformations in a Dynamic Geometry Environment: Multimodal Analysis of Student Reasoning. International Journal of Science and Mathematics Education. https://doi.org/10.1007/s10763-021-10185-y

Tytler, R., Prain, V., Aranda, G., Ferguson, J., & Gorur, R. (2020). Drawing to reason and learn in science. Journal of Research in Science Teaching, 57(2), 209–231. https://doi.org/10.1002/tea.21590

Ubah, I., & Bansilal, S. (2019). The use of semiotic representations in reasoning about similar triangles in Euclidean geometry. Pythagoras, 40(1), 1–10. https://doi.org/10.4102/PYTHAGORAS.V40I1.480

Vassilyev, S. N. (2021). Erratum to: Abductive Reasoning in Explanation Problems of an Observed Effect (Doklady Mathematics, (2020), 102, 1, (337-341), 10.1134/S1064562420040195). Doklady Mathematics, 103(1), 67. https://doi.org/10.1134/S1064562421330012

Wilkie, K. J. (2019). Investigating secondary students’ generalization, graphing, and construction of figural patterns for making sense of quadratic functions. Journal of Mathematical Behavior, 54(February 2018), 0–1. https://doi.org/10.1016/j.jmathb.2019.01.005

Winsløw, C. (2019). Nordic Research in Mathematics Education. In Nordic Research in Mathematics Education. https://doi.org/10.1163/9789087907839

Xu, L, Ferguson, J., & Tytler, R. (2021). Student Reasoning About the Lever Principle Through Multimodal Representations: a Socio-Semiotic Approach. International Journal of Science and Mathematics Education, 19(6), 1167–1186. https://doi.org/10.1007/s10763-020-10102-9

Xu, L, van Driel, J., & Healy, R. (2021). A multi-layered framework for analyzing primary students’ multimodal reasoning in science. Education Sciences, 11(12). https://doi.org/10.3390/educsci11120758

Xu, Lihua, Ferguson, J., & Tytler, R. (2021). Student Reasoning About the Lever Principle Through Multimodal Representations: a Socio-Semiotic Approach. International Journal of Science and Mathematics Education, 19(6), 1167–1186. https://doi.org/10.1007/s10763-020-10102-9

Yuliono, T., Sarwanto, S., & Rintayati, P. (2018). The Promising Roles of Augmented Reality in Educational Setting: A Review of the Literature. International Journal of Educational Methodology, 4(3), 125–132. https://doi.org/10.12973/ijem.4.3.125




How to Cite

Ekowati, D. W., nusantara, toto, muksar, makbul, & Sudjimat, D. agus. (2024). A Literature Review Of Multimodal Semiotic Reasoning In Mathematics : . Pegem Journal of Education and Instruction, 14(2), 261–274. https://doi.org/10.47750/pegegog.14.02.30