Statistical measurement of behavioral effects based on multimodal data

Suyan Tan; Yunyi Zhao; Jinjun Wang; Jia Fang; Suyan Tan; Yunyi Zhao; Jinjun Wang; Jia Fang

doi:10.3934/NAR.2024027

National Accounting Review

2024, Volume 6, Issue 4: 573-589. doi: 10.3934/NAR.2024027

Previous Article Next Article

Research article

Statistical measurement of behavioral effects based on multimodal data

Suyan Tan ¹,
Yunyi Zhao ¹,
Jinjun Wang ^{1
,
,},
Jia Fang ^{2
,
,}

1.
School of Foreign Studies, Guangzhou University, Guangzhou, Guangdong 510006, China
2.
School of Humanities and Economics, Changzhou Open University, Changzhou 213001, China

Received: 24 October 2024 Revised: 27 December 2024 Accepted: 29 December 2024 Published: 31 December 2024
JEL Codes: C83, C90, I21

The application of multimodal data is particularly important in accurately assessing behavioral effects and optimizing the decision-making process. This type of data provides more comprehensive and in-depth insights by integrating information from different sources and formats. Comprehensive data support not only enhances the science and accuracy of decision-making but also significantly improves the quality of behavioral effectiveness assessment. This study first describes the practical significance and theoretical value of multimodal data in behavioral effect assessment. Subsequently, the types of multimodal data involved and the construction methods of data sets are introduced. In order to demonstrate the role of multimodal data in behavioral effect assessment, the teaching effect of English classroom presentations at a comprehensive university in China is taken as a case study, and the effect of the target behavior was statistically measured based on multimodal data such as students' classroom behavioral videos, images, questionnaires, interviews, and assessment data. The results of the case study show that AI+ demonstrates significant advantages in behavioral effect assessment, which is more objective and effectively avoids the limitations of subjectivity in traditional assessment methods. At the same time, multimodal data helps optimize behavioral effects. For example, the presentations made at the beginning of the class show significant advantages in teaching effect compared with the presentation made before the end of the class, which provides data support and optimization direction for the implementation of teaching activities.
- multimodal data,
- statistical measurement,
- behavioral effect,
- case study,
- English classroom presentation
Citation: Suyan Tan, Yunyi Zhao, Jinjun Wang, Jia Fang. Statistical measurement of behavioral effects based on multimodal data[J]. National Accounting Review, 2024, 6(4): 573-589. doi: 10.3934/NAR.2024027

Related Papers:

Abstract

The application of multimodal data is particularly important in accurately assessing behavioral effects and optimizing the decision-making process. This type of data provides more comprehensive and in-depth insights by integrating information from different sources and formats. Comprehensive data support not only enhances the science and accuracy of decision-making but also significantly improves the quality of behavioral effectiveness assessment. This study first describes the practical significance and theoretical value of multimodal data in behavioral effect assessment. Subsequently, the types of multimodal data involved and the construction methods of data sets are introduced. In order to demonstrate the role of multimodal data in behavioral effect assessment, the teaching effect of English classroom presentations at a comprehensive university in China is taken as a case study, and the effect of the target behavior was statistically measured based on multimodal data such as students' classroom behavioral videos, images, questionnaires, interviews, and assessment data. The results of the case study show that AI+ demonstrates significant advantages in behavioral effect assessment, which is more objective and effectively avoids the limitations of subjectivity in traditional assessment methods. At the same time, multimodal data helps optimize behavioral effects. For example, the presentations made at the beginning of the class show significant advantages in teaching effect compared with the presentation made before the end of the class, which provides data support and optimization direction for the implementation of teaching activities.

References

[1]	Aboulola O, Khayyat M, Al-Harbi B, et al. (2021) Multimodal feature-assisted continuous driver behavior analysis and solving for edge-enabled Internet of connected vehicles using deep learning. Appl Sci 11: 10462. https://doi.org/10.3390/app112110462 doi: 10.3390/app112110462
[2]	Albert CCY, Sun Y, Guang L, et al. (2022) Identifying and monitoring students' classroom learning behavior based on multisource information. Mob Inf Sys 8: 9903342. https://doi.org/10.1155/2022/9903342 doi: 10.1155/2022/9903342
[3]	Alkhomsan MN, Hossain MA, Rahman SKMM, et al. (2017) Situation awareness in ambient assisted living for smart healthcare. IEEE Access 5: 20716–20725. https://doi.org/10.1109/ACCESS.2017.2731363 doi: 10.1109/ACCESS.2017.2731363
[4]	Chandra BS, Sastry CS, Jana S (2018) Robust heartbeat detection from multimodal data via CNN-based generalizable information fusion. IEEE Trans Biomed Eng 66: 710–717. https://doi.org/10.1109/TBME.2018.2854899 doi: 10.1109/TBME.2018.2854899
[5]	Chen H, Guan J (2022) Teacher-student behavior recognition in classroom teaching based on improved YOLO-v4 and Internet of Things technology. Electronics 11: 3998. https://doi.org/10.3390/electronics11233998 doi: 10.3390/electronics11233998
[6]	Chew SH, Yi JJ, Zhang JS, et al. (2017) Education and anomalies in decision making: Experimental evidence from Chinese adult twins. J Risk Uncertain 53: 163–200. https://doi.org/10.1007/s11166-016-9246-7 doi: 10.1007/s11166-016-9246-7
[7]	Choi J (2018) Effects of collocational competence and small group cooperative learning on Korean university students' English presentations. Linguist Assoc Korea J 26: 267–285.
[8]	Cortes C, Manterola C (2020) Behavioral alterations associated with levetiracetam in pediatric epilepsy. Epilepsy Behav 112: 107472. https://doi.org/10.1016/j.yebeh.2020.107472 doi: 10.1016/j.yebeh.2020.107472
[9]	Trinh H, Edge D, Ring L, et al. (2016) Thinking Outside the Box: Co-planning Scientific Presentations with Virtual Agents. In: Traum, D., Swartout, W., Khooshabeh, P., Kopp, S., Scherer, S., Leuski, A. (eds) Intelligent Virtual Agents. IVA 2016. Lecture Notes in Computer Science, 10011: 306–316, Springer, Cham. https://doi.org/10.1007/978-3-319-47665-0_27
[10]	Han MH (2024) The effect of peer feedback on Korean college students' English presentation skills. Linguist Assoc Korea J 5: 73–92.
[11]	Halma E, de Louw AJA, Halma E (2014) Behavioral side-effects of levetiracetam in children with epilepsy: A systematic review. Seizure-Eur J Epilep 23: 685–691. https://doi.org/10.1016/j.seizure.2014.06.004 doi: 10.1016/j.seizure.2014.06.004
[12]	Hartono H, Mujiyanto J, Fitriati SW, et al. (2023) English presentation self-efficacy development of Indonesian ESP students: the effects of individual versus group presentation tasks. Int J Lang Educ 7: 361–376. https://doi.org/10.26858/ijole.v7i3.34442 doi: 10.26858/ijole.v7i3.34442
[13]	Ho YG, Kim HS, Lee SH (2021) The effect of children's private education experiences, parental perception of the necessity of private education, and conflict between family members on children's behavioral problems in China. The Journal of Korea Open Association for Early Childhood Education 26: 271–294.
[14]	Ignatious HA, El-Sayed H, Khan MA, et al. (2023) A generic framework for enhancing autonomous driving accuracy through multimodal data fusion. Appl Sci 13: 10749. https://doi.org/10.3390/app131910749 doi: 10.3390/app131910749
[15]	Jeynes WH (2019) A Meta-Analysis on the Relationship Between Character Education and Student Achievement and Behavioral Outcomes. Educ Urban Soc 51: 33–71. https://doi.org/10.1177/0013124517747681 doi: 10.1177/0013124517747681
[16]	Kim SH, Lee BI (2019) A Study on Effects of Universal Positive Behavior Support in Inclusive Classrooms on the Prosocial Behaviors of General Young Children. J Behav Anal Support 6: 49–80. https://doi.org/10.22874/kaba.2019.6.2.49 doi: 10.22874/kaba.2019.6.2.49
[17]	Lee JY (2018) An analysis of the presentation materials for effective presentation education of foreign undergraduate students. Urimalgeul: The Korean Language and Literature 79: 57–86.
[18]	Li L, Liu M, Sun L, et al. (2022) ET-YOLOv5s: Toward Deep Identification of Students' in-Class Behaviors. IEEE Access 5: 44200–44211. https://doi.org/10.1109/ACCESS.2022.3169586 doi: 10.1109/ACCESS.2022.3169586
[19]	Lim SH (2014) The Effects of Emotional Education Activities on 4-year-old Children's Emotional Intelligence and Their Behavioral Problems: Focusing on Use of Picture Books. Journal of Children's Literature and Education 15: 335–355.
[20]	Lin J, Li J, Chen J (2021) An analysis of English classroom behavior by intelligent image recognition in IoT. Int J Syst Assur Eng Manag 13: 1063–1071. https://doi.org/10.1007/s13198-021-01327-0 doi: 10.1007/s13198-021-01327-0
[21]	Liu Q et al. (2024) YOLOv8n_BT: research on classroom learning behavior recognition algorithm based on improved YOLOv8n. IEEE Access 12: 36391–36403. https://doi.org/10.1109/ACCESS.2024.3373536 doi: 10.1109/ACCESS.2024.3373536
[22]	Meyer C, Romero NB, Evangelista T, et al. (2024) IMPatienT: An integrated web application to digitize, process and explore multimodal patient data. J Neuromuscul Dis 11: 855–870. https://doi.org/10.3233/JND-230085 doi: 10.3233/JND-230085
[23]	Oresti B, Claudia B, Jaehun B, et al. (2016) Human behavior analysis by means of multimodal context mining. Sensors 16: 1264. https://doi.org/10.3390/s16081264 doi: 10.3390/s16081264
[24]	Ren QM (2023) The Formulation and Verification of the Scale of Multidimensional Evaluation for Student Engagement in College English Classroom. Shandong Foreign Language Teaching 43: 58–66. https://doi.org/10.16482/j.sdwy37-1026.2022-04-006 doi: 10.16482/j.sdwy37-1026.2022-04-006
[25]	Smoliński A, Forczmański P, Nowosielski A (2024) Processing and integration of multimodal image data supporting the detection of behaviors related to reduced concentration level of motor vehicle users. Electronics 13: 2457. https://doi.org/10.3390/electronics13132457 doi: 10.3390/electronics13132457
[26]	Stapa M, Murad NA, Ahmad N (2014) Engineering technical oral presentation: voices of the stakeholder. Procedia Soc Behav Sci 118: 463–467. https://doi.org/10.1016/j.sbspro.2014.02.063 doi: 10.1016/j.sbspro.2014.02.063
[27]	Park B (2012) The effect of peer feedback on improving English presentation skills. Studies in British and American Language and Literature 105: 193–216.
[28]	Park ED (2016) A study on the effect of the usage of mobile presentation authoring tool on presentation skill in PBL course. J Humanit Soc Sci 7: 605–624.
[29]	Takahashi K, Gu W, Ota K, et al. (2024) An academic presentation support system utilizing structural elements. IEICE Trans Inf Syst 6: 486–494. https://doi.org/10.1587/transinf.2023IHP0006 doi: 10.1587/transinf.2023IHP0006
[30]	Wang C, Zhang M, Shi F, et al. (2022) A hybrid multimodal data-fusion-based method for identifying gambling websites. Electronics 11: 2489. https://doi.org/10.3390/electronics11162489 doi: 10.3390/electronics11162489
[31]	Wang S, Zheng K, Kong W, et al. (2023) Multimodal data fusion based on IGERNNC algorithm for detecting pathogenic brain regions and genes in Alzheimer's disease. Brief Bioinform 24: 1–14. https://doi.org/10.1093/bib/bbac515 doi: 10.1093/bib/bbac515
[32]	Watts TW, Nguyen T, Carr RC, et al. (2021). Examining the Effects of Changes in Classroom Quality on Within-Child Changes in Achievement and Behavioral Outcomes. Child Dev 92: E439–E456. https://doi.org/10.1111/cdev.13552 doi: 10.1111/cdev.13552
[33]	Wu L (2016) A study of college students' classroom learning behavior. Educational Teaching Forum 11: 50–51.
[34]	Xiao J, Jiang Z, Wang L, et al. (2023) What can multimodal data tell us about online synchronous training: Learning outcomes and engagement of in-service teachers. Front Psychol 13: 1092848. https://doi.org/10.3389/fpsyg.2022.1092848 doi: 10.3389/fpsyg.2022.1092848
[35]	Xie L, Feng X, Zhang C, et al. (2022) Identification of urban functional areas based on the multimodal deep learning fusion of high-resolution remote sensing images and social perception data. Buildings 12: 556. https://doi.org/10.3390/buildings12050556 doi: 10.3390/buildings12050556
[36]	Zhang KF, Yeh SC, Wu EHK, et al. (2024) Fusion of multi-task neurophysiological data to enhance the detection of attention-deficit/hyperactivity disorder. IEEE J Transl Eng Health Med 12: 668–674. https://doi.org/10.1109/JTEHM.2024.3435553 doi: 10.1109/JTEHM.2024.3435553
[37]	Zhao X, Fei F (2022) Investigation on the design of anthropomorphic oral presentation assistant training system. Mobile Information Systems 2022: 3719010. https://doi.org/10.1155/2022/3719010 doi: 10.1155/2022/3719010
[38]	Zheng ZK, Staubitz JE, Weitlauf AS, et al. (2021) Predictive multimodal framework to alert caregivers of problem behaviors for children with ASD (PreMAC). Sensors 21: 370. https://doi.org/10.3390/s21020370 doi: 10.3390/s21020370
[39]	Zhou J, Ran F, Li G, et al. (2022) Classroom learning status assessment based on deep learning. Math Probl Eng 4: 7049458. https://doi.org/10.1155/2022/7049458 doi: 10.1155/2022/7049458

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)