Analyzing User Experience and Satisfaction in the B-Block Game-Based Assessment

Lailatul Husniah; Ali Sofyan Kholimi; Umi Laili Yuhana; Eko Mulyanto Yuniarno; Mauridhi Hery Purnomo

doi:10.23887/janapati.v14i1.92784

Authors

Lailatul Husniah Institut Teknologi Sepuluh Nopember
Ali Sofyan Kholimi Universitas Muhammadiyah Malang
Umi Laili Yuhana Institut Teknologi Sepuluh Nopember
Eko Mulyanto Yuniarno Institut Teknologi Sepuluh Nopember
Mauridhi Hery Purnomo Institut Teknologi Sepuluh Nopember

DOI:

https://doi.org/10.23887/janapati.v14i1.92784

Keywords:

game-based assessment, education, mathematic, user experience, user satisfaction

Abstract

Game-based assessment (GBA) has developed as an innovative education method, including learning basic arithmetic operations. This study aims to analyze user experience and satisfaction using B-Block, an assessment-based game for basic arithmetic operations. The study involved 94 junior high school students with an age distribution of 12-13 years old and varying levels of gaming experience. The research used descriptive statistical analysis, validity and reliability test, Pearson correlation test, and multiple linear regression to identify factors influencing user satisfaction and continuance usage intention. The analysis showed that B-Block has good usability and educational benefits, with user satisfaction being the most dominant aspect. Validity and reliability tests confirmed that most variables were valid and reliable (Cronbach's Alpha > 0.7), except Errors, which had lower reliability (α = 0.632). Pearson correlation shows that Perceived Usefulness has a strong relationship with satisfaction (r = 0.784), while user satisfaction contributes significantly to continuance intention (r = 0.694). Multiple linear regression revealed that perceived usability and perceived usefulness were the main factors influencing user satisfaction, while confirmation and satisfaction had the most effect on continuance intention. The findings confirm that the gameplay's usability and perceived usefulness are key in increasing user satisfaction while matching the experience with initial expectations, and user satisfaction contributes to continued use.

References

M. J. Gomez, J. A. Ruiperez-Valiente, and F. J. G. C. Clemente, “A Systematic Literature Review of Game-Based Assessment Studies: Trends and Challenges,” IEEE Trans. Learn. Technol., vol. 16, no. 4, pp. 500–515, 2023, doi: 10.1109/TLT.2022.3226661.

K. Kiili and H. Ketamo, “Evaluating Cognitive and Affective Outcomes of a Digital Game-Based Math Test,” IEEE Trans. Learn. Technol., vol. 11, no. 2, pp. 255–263, 2018, doi: 10.1109/TLT.2017.2687458.

M. Ninaus, K. Kiili, J. McMullen, and K. Moeller, “Assessing fraction knowledge by a digital game,” Comput. Human Behav., vol. 70, pp. 197–206, 2017, doi: 10.1016/j.chb.2017.01.004.

K. J. M. Kiili, K. Devlin, A. Perttula, P. Tuomi, and A. Lindstedt, “Using video games to combine learning and assessment in mathematics education,” Int. J. Serious Games, vol. 2, no. 4, pp. 37–55, 2015, doi: 10.17083/ijsg.v2i4.98.

F. Leutner, S. C. Codreanu, J. Liff, and N. Mondragon, “The potential of game- and video-based assessments for social attributes: examples from practice,” J. Manag. Psychol., vol. 36, no. 7, pp. 533–547, 2021, doi: 10.1108/JMP-01-2020-0023.

M. N. Azzmi.H., U. L. Yuhana, N. Sulistyani, and L. Husniah, “Analyzing the Quality of Game-based Assessment Design in Basic Arithmetic Operations,” Kinet. Game Technol. Inf. Syst. Comput. Network, Comput. Electron. Control, vol. 4, no. 1, 2023, doi: 10.22219/kinetik.v8i1.1599.

J. Hautala, R. Heikkilä, L. Nieminen, V. Rantanen, J. M. Latvala, and U. Richardson, “Identification of Reading Difficulties by a Digital Game-Based Assessment Technology,” J. Educ. Comput. Res., vol. 58, no. 5, pp. 1003–1028, 2020, doi: 10.1177/0735633120905309.

V. Shute et al., “Maximizing learning without sacrificing the fun: Stealth assessment, adaptivity and learning supports in educational games,” J. Comput. Assist. Learn., vol. 37, no. 1, pp. 127–141, 2021, doi: 10.1111/jcal.12473.

M. Cutumisu, D. B. Chin, and D. L. Schwartz, “A digital game-based assessment of middle-school and college students’ choices to seek critical feedback and to revise,” Br. J. Educ. Technol., vol. 50, no. 6, pp. 2977–3003, 2019.

V. Vallejo et al., “Evaluation of a novel Serious Game based assessment tool for patients with Alzheimer’s disease,” PLoS One, vol. 12, no. 5, p. e0175999, 2017.

Y. L. Chien et al., “Game-Based Social Interaction Platform for Cognitive Assessment of Autism Using Eye Tracking,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 31, pp. 749–758, 2023, doi: 10.1109/TNSRE.2022.3232369.

M. Zapata-Caceres, E. Martin-Barroso, and M. Roman-Gonzalez, “Collaborative Game-Based Environment and Assessment Tool for Learning Computational Thinking in Primary School: A Case Study,” IEEE Trans. Learn. Technol., vol. 14, no. 5, pp. 576–589, 2021, doi: 10.1109/TLT.2021.3111108.

M. Cutumisu, K. P. Blair, D. B. Chin, and D. L. Schwartz, “Posterlet : A Game-Based Assessment of Children ’ s Choices to Seek Feedback and to Revise Choice-Based Assessments,” J. Learn. Anal., vol. 2, pp. 49–71, 2015.

A. Leonardou, M. Rigou, and J. Garofalakis, “Techniques to Motivate Learner Improvement in Game-Based Assessment,” Information, vol. 11, no. 4, p. 176, 2020.

T. Jordaan, M. Havenga, and B. Bunt, “Mathematical game-based learning: education students’ collaboration and on-line experiences during disrupted Covid-19 circumtances,” World Trans. Eng. Technol. Educ., vol. 19, no. 3, pp. 263–270, 2021.

J. L. G. Sánchez, F. L. G. Vela, F. M. Simarro, and N. Padilla-Zea, “Playability: Analysing user experience in video games,” Behav. Inf. Technol., vol. 31, no. 10, pp. 1033–1054, 2012, doi: 10.1080/0144929X.2012.710648.

N. Adiftya Falah, E. Dwi Wahyuni, and V. Rahmayanti Setyaning Nastiti, “Analisis Kepuasan Pengguna E-Learning Menggunakan System Usability Scale (SUS) (Studi Kasus: MA Muhammadiyah 1 Malang),” Repositor, vol. 5, no. 3, pp. 815–824, 2023.

Ramadiani, R. Atan, M. H. Selamat, R. Abdullah, N. C. Pa, and Azainil, “User difficulties in e-learning system,” Proceeding - 2016 2nd Int. Conf. Sci. Inf. Technol. ICSITech 2016 Inf. Sci. Green Soc. Environ., no. October, pp. 158–162, 2017, doi: 10.1109/ICSITech.2016.7852626.

Sholiq, R. A. Auda, A. P. Subriadi, A. Tjahyanto, and A. D. Wulandari, “Measuring software quality with usability, efficiency, and portability characteristics,” IOP Conf. Ser. Earth Environ. Sci., vol. 704, no. 1, 2021, doi: 10.1088/1755-1315/704/1/012039.

M. E. Eltahir, S. Al-qatawneh, N. Al-ramahi, and N. Alsalhi, “THE PERSPECTIVE OF STUDENTS AND FACULTY MEMBERS ON THE EFFICIENCY AND USABILITY OF E-LEARNING COURSES AT AJMAN UNIVERSITY: A CASE STUDY,” J. Technol. Sci. Educ., vol. 9, no. 3, pp. 388–403, 2019.

T. Cahyani Naila, D. Wahyuni, and V. Rahmayanti, “EVALUASI USABILITY DAN REKOMENDASI PERBAIKAN WEBSITE SIP BRO MENGGUNAKAN METODE SUS DAN THINK ALOUD,” vol. 9, no. 2, 2023.

M. Pendidikan, D. A. N. Kebudayaan, and R. Indonesia, “Peraturan Menteri Pendidikan dan Kebudayaan Republik Indonesia Nomor 37 Tahun 2018 Tentang Perubahan Atas Peraturan Menteri Pendidikan dan Kebudayaan Nomor 24 Tahun 2016 Tentang Kompetensi Inti dan Kompetensi Dasar Pelajaran Pada Kurikulum 2013 Pada Pendi.” Kemdikbud RI, 2018.

E. Erdfelder, F. FAul, A. Buchner, and A. G. Lang, “Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses,” Behav. Res. Methods, vol. 41, no. 4, pp. 1149–1160, 2009, doi: 10.3758/BRM.41.4.1149.

D. Lakens, “Sample Size Justification,” Collabra Psychol., vol. 8, no. 1, 2022, doi: 10.1525/collabra.33267.

A. Ramadhan, A. N. Hidayanto, G. A. Salsabila, I. Wulandari, J. A. Jaury, and N. N. Anjani, The effect of usability on the intention to use the e-learning system in a sustainable way: A case study at Universitas Indonesia, vol. 27, no. 2. Springer US, 2022.

R. Sharma and M. Gopal, “A Markov Game-Adaptive Fuzzy Controller for Robot Manipulators,” vol. 16, no. 1, pp. 171–186, 2008.