Comparison of decision trees used in data mining
The purpose of this study is to compare decision trees obtained by data mining algorithms used in various areas in recent years according to different criteria. In the study, similar and different aspects of the decision trees obtained by different methods for classifying the students as successful and unsuccessful in terms of science literacy were revealed with the help of 12 independent variables included in the PISA 2015 student survey. Data collected across Turkey, from a total of 5895 students in the age group of 15, were analyzed in Java-based Weka software, which has an open source code. As a result of the analysis, it was found that the most successful algorithms in terms of correct classification rate were respectively Logistic Model, Hoeffding Tree, J.48, REPTree and Random Tree. In addition, regarding the decision trees obtained by different learning algorithms, variables that have been effective in the classification were found to be different. According to the results, it was concluded that independent variables found to be effective in the classification of the students for the decision trees obtained by different algorithms differed from each other and it was suggested to report the finding of more than one algorithm instead of those of only one algorithm.
Almuniri, I., & Said, A. M. (2017). School’s performance evaluation based on data mining. International Journal of Engineering and Information Systems, 1 (9), 56–62.
Aydın, S. (2007). Veri madenciliği ve Anadolu Üniversitesi uzaktan eğitim sisteminde bir uygulama. Unpublished doctorate dissertation, Anadolu Üniversitesi, Eskişehir.
Bakker, R. (2016). A comparison of decision trees for ingredient classification. Bachelor thesis, University of Amsterdam, Amsterdam.
Barros, R. C., Carvalho, A. C. P. L. F. de, & Freitas, A. A. (2015). Automatic design of decision-tree ınduction algorithms. Heidelberg, NY: SpringerBriefs in Computer Science. DOI: https://doi.org/10.1007/978-3-319-14231-9
Büyüköztürk, Ş., Kılıç Çakmak, E., Akgün, Ö., Karadeniz, Ş., & Demirel, F. (2016). Eğitimde bilimsel araştırma yöntemleri. Ankara: Pegem Akademi Yayıncılık. DOI: https://doi.org/10.14527/9789944919289
Carletta, J. (1993). Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics, 22 (2), 249–254.
Cinaroglu, S. (2016). Comparison of performance of decision tree algorithms and random forest: An application on OECD countries health expenditures. International Journal of Computer Applications, 138 (1), 37–41. DOI: https://doi.org/10.5120/ijca2016908704
Dietterich, T. (2000). An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning, 40 (2), 139–157. DOI: https://doi.org/10.1023/A:1007607513941
Doreswamy, H. K. (2012). Performance evaluation of predictive classifiers for knowledge discovery from engineering materials data sets. CIIT International Journal of Artificial Intelligent Systems and Machine Learning, 3 (3), 162–168.
Fayyad, U., Piatetsky-Shapiro, G., & Smyth, P. (1996). The KDD process for extracting useful knowledge from volumes of data. Communications of the ACM, 39 (11), 27–34. https://doi.org/10.1145/240455.240464 DOI: https://doi.org/10.1145/240455.240464
Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education. New York: McGraw-Hill.
Han, J., & Kamber, M. (2006). Data mining concepts and techniques. San Francisco, CA: Morgan Kaufmann Publishers.
Hossin, M., & Sulaiman, M. N. (2015). A review on evaluation metrics for data classification evaluations. International Journal of Data Mining & Knowledge Management Process, 5(2), 01–11. https://doi.org/10.5121/ijdkp.2015.5201 DOI: https://doi.org/10.5121/ijdkp.2015.5201
Hssina, B., Abdelkarim, M., Ezzikouri, H., & Erritali, M. (2014). A comparative study of decision tree ID3 and C4.5. International Journal of Advanced Computer Science and Applications, 4 (2), 13–19. https://doi.org/10.14569/specialissue.2014.040203 DOI: https://doi.org/10.14569/SpecialIssue.2014.040203
Huang, S., & Fang, N. (2013). Predicting student academic performance in an engineering dynamics course: A comparison of four types of predictive mathematical models. Computers and Education, 61 (1), 133–145. https://doi.org/10.1016/j.compedu.2012.08.015 DOI: https://doi.org/10.1016/j.compedu.2012.08.015
Imielinski, T., & Mannila, H. (1996). A database perspective on knowledge discovery. Communications of the ACM, 39 (11), 373–408. DOI: https://doi.org/10.1145/240455.240472
Kiranmai, B., & Damodaram, A. (2014). A review on evaluation measures for data mining tasks. International Journal Of Engineering And Computer Science, 3 (7), 7217–7220.
Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, 2, 1137–1145. https://doi.org/10.1067/mod.2000.109031
Kusiak, A. (2001). Data analysis: Models and algorithms. Proceedings of the SPIE Conference on Intelligent Systems and Advanced Manufacturing, In P.E. Orban and G.K. Knopf (Eds), SPIE (pp. 1-9), Boston: MA. DOI: https://doi.org/10.1117/12.417237
Kuyucu, Y. E. (2012). Lojistik regresyon analizi (LRA), yapay sinir ağları (YSA) ve sınıflandırma ve regresyon ağaçları (C&RT) yöntemlerinin karşılaştırılması ve tıp alanında bir uygulama. Unpublished master’s thesis, Gaziosmanpaşa Üniversitesi, Sağlık Bilimleri Enstitüsü, Tokat.
Larose, D. T. (2005). Discovering knowledge in data: An introduction to data mining. New Jersey: John Wiley & Sons. DOI: https://doi.org/10.1002/0471687545
Liaw, A., & Wiener, M. (2002). Classification and regression by random forest. R News, 2 (3), 18–22.
Lv, S., Kim, H., Zheng, B., & Jin, H. (2018). A review of data mining with big data towards its applications in the electronics industry. Applied Sciences, 7 (582), 2–34. https://doi.org/10.3390/app8040582 DOI: https://doi.org/10.3390/app8040582
Lykourentzou, I., Giannoukos, I., Mpardis, G., Nikolopoulos, V., & Loumos, V. (2009). Early and dynamic student achievement prediction in e-learning courses using neural networks. Journal of the American Society for Information Science and Technology, 60 (2), 372–380. DOI: https://doi.org/10.1002/asi.20970
Maimon, O., & Rokach, L. (2005). Data mining and knowledge discovery handbook. Secaucus, NJ: Springer-Verlag Inc. DOI: https://doi.org/10.1007/b107408
Mease, D., & Wyner, A. (2008). Evidence contrary to the statistical view of boosting: A rejoinder to responses. Journal of Machine Learning Research, 9, 195–201.
MEB. (2016). PISA 2015 Ulusal Raporu. Millî Eğitim Bakanlığı, Ölçme, Değerlendirme ve Sınav Hizmetleri Genel Müdürlüğü, Ankara.
Mohan, V. (2013). Decision trees: A comparison of various algorithms for building decision trees. Retrieved July 23, 2019, from https://pdfs.semanticscholar.org/3399/c175beca3ab4843d67f91bb28f564099d0bb.pdf
Neelamegam, S., & Ramaraj, E. (2013). Classification algorithm in data mining: An Overview. International Journal of P2P Network Trends and Technology (IJPTT), 3 (5), 1–5.
Podgorelec, V., Kokol, P., Stiglic, B., & Rozman, I. (2002). Decision trees: An overview and their use in medicine. Journal of Medical Systems, 26 (5), 445-463. DOI: https://doi.org/10.1023/A:1016409317640
Rastogi, R., & Shim, K. (2000). PUBLIC: A decision tree classifier that integrates building and pruning. Data Mining and Knowledge Discovery, 4 (4), 315–344. DOI: https://doi.org/10.1023/A:1009887311454
Romero, C., & Ventura, S. (2013). Data mining in education. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 3 (1), 12–27. https://doi.org/10.1002/widm.1075 DOI: https://doi.org/10.1002/widm.1075
Sieber, J. E. (2008). Data mining: Knowledge discovery for human research ethics. Journal of Empirical Research on Human Research Ethics, 3 (3), 1–2. https://doi.org/10.1525/jer.2008.3.3.1 DOI: https://doi.org/10.1525/jer.2008.3.3.1
Sokolova, M., & Lapalme, G. (2009). A systematic analysis of performance measures for classification tasks. Information Processing and Management, 45 (4), 427–437. https://doi.org/10.1016/j.ipm.2009.03.002 DOI: https://doi.org/10.1016/j.ipm.2009.03.002
Strobl, C., Malley, J., & Tutz, G. (2009). An introduction to recursive partitioning: Rationale, application, and characteristics of classification and regression trees, bagging, and random forests. Psychological Methods, 14 (4), 323–348. https://doi.org/10.1037/a0016973. DOI: https://doi.org/10.1037/a0016973
Svetnik, V., Liaw, A., Tong, C., & Wang, T. (2004). Application of breiman’s random forest to modeling structure-activity relationships of pharmaceutical molecules. In F. Roli, J. Kittler, & T. Windeatt, (Eds.). Multiple Classifier Systems (pp. 1-35), Berlin, Heidelberg: Springer. DOI: https://doi.org/10.1007/978-3-540-25966-4_33
Tan, P.N., Steinbach, M., & Kumar, V. (2005). Introduction to data mining. Boston, USA: Addison-Wesley Longman Publishing Co.
Thuraisingham, B. (2003). Web data mining and applications in business ıntelligence and counter terrorism. USA: CRC Press LLC, Boca Raton, FL. DOI: https://doi.org/10.1201/9780203499511
Tiwari, M., Jha, M. B., & Yadav, O. (2012). Performance analysis of data mining algorithms in weka. IOSR Journal of Computer Engineering (IOSRJCE), 6 (3), 32–41. DOI: https://doi.org/10.9790/0661-0633241
Vaus, D. de. (2001). Research design in social research. London: Sage Publications.
Vihinen, M. (2012). How to evaluate performance of prediction methods? Measures and their interpretation in variation effect analysis. BMC Genomics, 13 (4), 1-10. https://doi.org/10.1186/1471-2164-13-S4-S2 DOI: https://doi.org/10.1186/1471-2164-13-S4-S2
Weiss, S. M., & Kulikowski, C. A. (1991). Computer systems that learn: Classification and prediction methods from statistics, neural nets, machine learning, and expert systems. San Mateo, CA: Morgan Kaufmann.
Witten, I. H., & Frank, E. (2005). Data mining: Practical machine learning tools and techniques. San Francisco, CA: Morgan Kaufmann Publishers.
Witten, I. H., Frank, E., & Hall, M. (2016). Data mining: Practical machine learning tools and techniques. USA: Morgan Kaufmann Publications.
Wu, X., Kumar, V., Ross, Q. J., Ghosh, J., Yang, Q., Motoda, H., … Steinberg, D. (2008). Top 10 algorithms in data mining. Knowledge and Information Systems, 14 (1), 1-37. https://doi.org/10.1007/s10115-007-0114-2 DOI: https://doi.org/10.1007/s10115-007-0114-2
Copyright (c) 2019 Gökhan Aksu, Nuri Doğan (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
I accept that the owner of PEGEGOG, the editor, associate editors, reviewers and the scientific committee cannot be hold responsible regarding the scope, the findings, the discussion and conclusion of the manuscript submitted.
I declare to the editorship of PEGEGOG that the manuscript is original and has not been published anywhere else or is not under evaluation process for any other journal.
I approve that I grant PEGEGOG as the sole and exclusive right and license to publish for the full legal term of copyright of my manuscript concurring with article 5846 / 22-23-25 while I retain copyright in the work.