Developing Classroom Assessment Tool using Learning Management System-based Computerized Adaptive Test in Vocational High Schools

Samsul Hadi; Haryanto Haryanto; Muh. Asriadi AM; Marlina Marlina; Abdul Rahim

doi:10.23887/jere.v6i1.35630

Authors

Samsul Hadi Universitas Negeri Yogyakarta http://orcid.org/0000-0003-3437-2542
Haryanto Haryanto Universitas Negeri Yogyakarta
Muh. Asriadi AM Universitas Negeri Yogyakarta http://orcid.org/0000-0003-4370-3550
Marlina Marlina Universitas Negeri Yogyakarta http://orcid.org/0000-0002-1363-9941
Abdul Rahim Universitas Negeri Yogyakarta http://orcid.org/0000-0002-6425-3600

DOI:

https://doi.org/10.23887/jere.v6i1.35630

Keywords:

Adaptive, Learning Management System, Classroom Assessment

Abstract

Computers have taken on a large role in education, including testing and evaluation. Traditional tests that aren't comprehensive and don't distinguish between students' beginning talents lead to measurement findings that aren't representative of their true abilities. This study aims to develop and test assessment tool eligibility class that is used as an LMS-based adaptive. This type of research includes development research. The respondents of this study were experts who assessed the validation and students of SMK Electrical Power Installation Engineering Expertise Competencies. The data analysis technique used item response theory, classical test theory and descriptive statistics. Item analysis using the Rasch Model showed 10 items were not fit and the remaining 50 items were fit. Classical test theory analysis items with less validity there are 0 items, moderate there are 55 items, and high there are 5 items, with an Alpha reliability of 0.934. The attitude questionnaire developed consists of 8 items. There are 0 items with less validity, 6 items being moderate, and 2 items high, with an Alpha reliability of 0.731. The developed observation guide contains 16 observations. Items with less validity have 0 items, while there are 15 items, and 1 item high. The use of CAT-assisted learning management systems on research tools greatly facilitates teachers in conducting research accurately and practically.

Author Biographies

Samsul Hadi, Universitas Negeri Yogyakarta

Department of Electrical Engineering Education

Haryanto Haryanto, Universitas Negeri Yogyakarta

Department of Electrical Engineering Education

Muh. Asriadi AM, Universitas Negeri Yogyakarta

Department of Education Research and Evaluation

Marlina Marlina, Universitas Negeri Yogyakarta

Department of Education Research and Evaluation

Abdul Rahim, Universitas Negeri Yogyakarta

Department of Education Research and Evaluation

References

Abbasi, S., Moeini, M., Shahriari, M., Ebrahimi, M., & Khoozani, E. K. (2018). Designing and manufacturing of educational multimedia software for preventing coronary artery disease and its effects on modifying the risk factors in patients with coronary artery disease. Electronic Journal of General Medicine, 15(3), 1–6. https://doi.org/10.29333/ejgm/85942.

Abidin, A. Z., Istiyono, E., Fadilah, N., & Dwandaru, W. S. B. (2019). A computerized adaptive test for measuring the physics critical thinking skills in high school students. International Journal of Evaluation and Research in Education (IJERE), 8(3), 376. https://doi.org/10.11591/ijere.v8i3.19642.

Asriadi, M., & Istiyono, E. (2020). Exploration of Creative Thinking Skills of Students in Physics Learning. Journal of Educational Science and Technology (EST), 6(2), 151–158. https://doi.org/10.26858/est.v6i2.12737.

Aybek, E. C., & Demirtasli, R. N. (2017). Computerized Adaptive Test (CAT) Applications and Item Response Theory Models for Polytomous Items. International Journal Of Research In Education And Science, 3(2), 475–475. https://doi.org/10.21890/ijres.327907.

Batubara, H. H. (2017). engembangan Situs E-Learning Dengan Moodle Versi 3.1 Sebagai Media Pembelajaran Pada Program Studi Pendidikan Guru Madrasah Ibtidaiyah. AL-BIDAYAH: Jurnal Pendidikan Dasar Islam, 9(1), 1–10. https://doi.org/10.14421/al-bidayah.v9i1.116.

Biney, I. K. (2020). Experiences of adult learners on using the Sakai Learning Management System for learning in Ghana. Journal of Adult and Continuing Education, 26(2), 262–282. https://doi.org/10.1177/1477971419864372.

Boonprasom, C., & Sintanakul, K. (2020). The Development of Collaborative Learning Management System Using Problem-Based on Cloud Learning to Enhance Critical Thinking. 2020 7th International Conference on Technical Education (ICTechEd7), 13–18. https://doi.org/10.1109/ICTechEd749582.2020.9101249.

Bradley, V. M. (2020). Learning Management System (LMS) Use with Online Instruction. International Journal of Technology in Education, 4(1), 68–92. https://doi.org/10.46328/ijte.36.

Chaw, L. Y., & Tang, C. M. (2018). What Makes Learning Management Systems Effective for Learning? Journal of Educational Technology Systems, 47(2), 152–169. https://doi.org/10.1177/0047239518795828.

Codish, D., Rabin, E., & Ravid, G. (2019). User behavior pattern detection in unstructured processes – a learning management system case study. Interactive Learning Environments, 27(5–6), 699–725. https://doi.org/10.1080/10494820.2019.1610456.

Copriady, J., Iswandari, S. N., Noer, A. M., & Albeta, S. W. (2020). Pengembangan E-Modul Berbasis Moodle Pada Materi Hidrokarbon. Edusains, 12(1), 81–88. https://doi.org/10.15408/es.v12i1.11503.

Devi, M., Annamalai, M. A. R., & Veeramuthu, S. P. (2020). Literature Education and Industrial Revolution 4.0. Universal Journal of Educational Research, 8(3), 1027–1036. https://doi.org/10.13189/ujer.2020.080337.

Dewanti, S. S., Ayriza, Y., & Setiawati, F. A. (2020). The Application of Item Response Theory for Development of a Students’ Attitude Scale Toward Mathematics. The New Educational Review, 60(2), 100–122. https://doi.org/10.15804/tner.2020.60.2.09.

Dewanti, S. S., Hadi, S., Nu’man, N., & Ibrahim. (2021). The Application of Item Response Theory in Analysis of Characteristics of Mathematical Literacy Test Items. İlköğretim Online, 20(1), 1226–1237. https://doi.org/10.17051/ilkonline.2021.01.119.

Fajrianthi, F., Hendriani, W., & Septarini, B. G. (2016). Pengembangan Tes Berpikir Kritis Dengan Pendekatan Item Response Theory. Jurnal Penelitian Dan Evaluasi Pendidikan, 20(1), 45–55. https://doi.org/10.21831/pep.v20i1.6304.

Fearnley, M. R., & Amora, J. T. (2020). Learning Management System Adoption in Higher Education Using the Extended Technology Acceptance Model. IAFOR Journal of Education, 8(2), 89–106. https://doi.org/10.22492/ije.8.2.05.

Febliza, A., & Okatariani, O. (2020). The Development of Online Learning Media by Using Moodle for General Chemistry Subject. Journal of Educational Science and Technology (EST), 6(1), 40. https://doi.org/10.26858/est.v6i1.12339.

Gustiani, S. (2019). Research And Development (R&D) Method As A Model Design In Educational Research And Its Alternatives. Holistics, 11(2). https://jurnal.polsri.ac.id/index.php/holistic/article/view/1849.

Hadi, S., Retnawati, H., Munadi, S., Apino, E., & Wulandari, N. F. (2018). The Difficulties of High School Students in Solving Higher-Order Thinking Skills Problems. Problems of Education in the 21st Century, 76(4), 520–532. https://doi.org/10.33225/pec/18.76.520.

Hardyanto, R. H., & Surjono, H. D. (2016). Pengembangan dan Implementasi E-Learning Menggunakan Moodle dan Vicon Untuk Pelajaran Pemrograman Web di SMK. Jurnal Pendidikan Vokasi, 6(1), 43. https://doi.org/10.21831/jpv.v6i1.6675.

Haryanto, H. (2013). Pengembangan Computerized Adaptive Testing (Cat) dengan Algoritma Logika Fuzzy. Jurnal Penelitian Dan Evaluasi Pendidikan, 15(1), 47–70. https://doi.org/10.21831/pep.v15i1.1087.

Hu, X., Ng, J., Tsang, K. K. Y., & Chu, S. K. W. (2020). Integrating Mobile Learning to Learning Management System in Community College. Community College Journal of Research and Practice, 44(10), 722–737. https://doi.org/10.1080/10668926.2019.1640146.

Hussin, A. A. (2018). Education 4.0 Made Simple: Ideas For Teaching. International Journal of Education and Literacy Studies, 6(3), 92–98. https://doi.org/10.7575/aiac.ijels.v.6n.3p.92.

Istiyono, E., Dwandaru, W. B., & Rahayu, F. (2018). The Developing Of Creative Thinking Skills Test Based On Modern Test Theory In Physics Of Senior High Schools. Jurnal Cakrawala Pendidikan, 37(2). https://doi.org/10.21831/cp.v37i2.19233.

Jana, S., & Khatun, S. (2021). Towards Selecting Effective Open Source Learning Management System/Software (OSLMS) for Higher Education Domain. Higher Education for the Future, 8(1), 47–70. https://doi.org/10.1177/2347631120970083.

Kezer, F. (2021). The Effect of Item Pools of Different Strengths on the Test Results of Computerized-Adaptive Testing. International Journal of Assessment Tools in Education, 8(1), 145–155. https://doi.org/10.21449/ijate.735155.

Lase, D. (2019). Pendidikan di Era Revolusi Industri 4.0. Sundermann: Jurnal Ilmiah Teologi, Pendidikan, Sains, Humaniora Dan Kebudayaan, 12(4), 28–43. https://doi.org/10.36588/sundermann.v1i1.18.

Liao, Y., Loures, E. R., Deschamps, F., Brezinski, G., & Venâncio, A. (2018). The impact of the fourth industrial revolution: A cross-country/region comparison. Production, 28(1), 1–18. https://doi.org/10.1590/0103-6513.20180061.

Liu, Q., & Geertshuis, S. (2021). Professional identity and the adoption of learning management systems. Studies in Higher Education, 46(3), 624–637. https://doi.org/10.1080/03075079.2019.1647413.

Ngafeeson, M. N., & Gautam, Y. (2021). Learning Management System Adoption: A Theory of Planned Behavior Approach. International Journal of Web-Based Learning and Teaching Technologies, 16(1), 27–42. https://doi.org/10.4018/IJWLTT.2021010104.

Noreen, S. (2020). Implementation of Learning Management System: A Way Ahead on the Digital Journey in Distance Learning. Open Praxis, 12(3), 329–342. https://doi.org/10.5944/openpraxis.12.3.1086.

Putra, E. A., Sudiana, R., & Pamungkas, A. S. (2020). Pengembangan Smartphone Learning Management System (S-LMS) Sebagai Media Pembelajaran Matematika di SMA. Kreano, Jurnal Matematika Kreatif-Inovatif, 11(1), 36–45. https://doi.org/10.15294/kreano.v11i1.21014.

Retnawati, H. (2017). The Dif Identification in Constructed Response Items Using Partial Credit Model. International Journal of Assessment Tools in Education, 5(1), 73–89. https://doi.org/10.21449/ijate.347956.

Rizal, R., Rusdiana, D., Setiawan, W., & Siahaan, P. (2020). Students Perception of Learning Management System Supported Smartphone: Satisfaction Analysis in Online Physics Learning. Jurnal Pendidikan IPA Indonesia, 9(4), 600–610. https://doi.org/10.15294/jpii.v9i4.25363.

Rudibyani, R. B., Perdana, R., & Elisanti, E. (2020). Development of Problem-Solving-Based Knowledge Assessment Instrument in Electrochemistry. International Journal of Instruction, 13(4), 957–974. https://doi.org/10.29333/iji.2020.13458a.

Sahin, A., Hurtado Grooscors, H. A., & Góngora-Cortés, J. J. (2018). Review of Fast Test: A Platform for Adaptive Testing. Measurement: Interdisciplinary Research and Perspectives, 16(4), 256–263. https://doi.org/10.1080/15366367.2018.1492867.

Sahin, D. I., & Shelley, D. M. (2020). Educational Practices during the COVID-19 Viral Outbreak: International Perspectives. USA: ISTES Organization Monument.

Sahi̇N, M. D., & Gelbal, P. D. S. (2020). Development of a Multidimensional Computerized Adaptive Test based on the Bifactor Model. International Journal of Assessment Tools in Education, 7(3), 323–342. https://doi.org/10.21449/ijate.707199.

Van der Linden, W. J. (2018). Handbook of Item Response Theory. Boca Raton London: CRC Press Taylor & Francis Group.

Wise, S. L., Kingsbury, G. G., & Webb, N. L. (2015). Evaluating Content Alignment in Computerized Adaptive Testing. Educational Measurement: Issues and Practice, 34(4), 41–48. https://doi.org/10.1111/emip.12094.

Yunis, R., & Telaumbanua, K. (2017). engembangan E-Learning Berbasiskan LMS untuk Sekolah, Studi Kasus SMA/SMK di Sumatera Utara. Jurnal Nasional Teknik Elektro Dan Teknologi Informasi (JNTETI), 6(1), 33–36. https://doi.org/10.22146/jnteti.v6i1.291.