Impact of Multimodal Instructions for Tool Manipulation Skills on Performance and User Experience in an Immersive Environment - Université d'Évry Access content directly
Conference Papers Year : 2024

Impact of Multimodal Instructions for Tool Manipulation Skills on Performance and User Experience in an Immersive Environment

Abstract

With the mentoring model, a mentee can learn technical skills under the supervision of more experienced peers who demonstrate their knowledge through several communication modalities. Supporting the mentoring model within shared immersive training simulators holds promise in enhancing mentor-mentee interactions and learning outcomes in a safe environment. However, efficient communication within these spaces remains an open issue. This work presents a user study that explores the combination of communication modalities (verbal-visual, verbal-haptic, visual-haptic, and verbal-visual-haptic) to convey instructions to learners on the amplitude of movements to perform during a tool-handling task in an immersive environment. The study aims to examine the impact of the four modality combinations on performance (speed and accuracy of movement replication), mental workload, and participants’ user experience. The results show that participants achieved higher accuracy with the visual-haptic and verbal-visual-haptic conditions. Moreover, they performed the movements faster, and their movement trajectories were closer to the reference trajectories in the visual-haptic condition. Finally, the most preferred verbal-visual-haptic combination enhanced the users’ sense of presence, co-presence, social presence, modalities is the best suited for enhancing learners’ performance. Adding the verbal modality can also improve the user experience in the immersive learning environment. These findings contribute to improving the design of immersive collaborative systems and pave the way for exploring novel avenues of research into the efficacy of multimodal communication for enhancing the mentoring-based acquisition of technical skills in VR. These tools hold promise for diverse applications, including medical simulation.
Fichier principal
Vignette du fichier
IEEEVR2024_CS_MBH_FL_SO_AC_Impact_of_Multimodal_Instructions_for_Tool_Manipulation_Skills_on_Performance_and_User_Experience_in_an_Immersive_Environment.pdf (30.77 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04505824 , version 1 (15-03-2024)

Identifiers

Cite

Cassandre Simon, Manel Boukli Hacene, Flavien Lebrun, Samir Otmane, Amine Chellali. Impact of Multimodal Instructions for Tool Manipulation Skills on Performance and User Experience in an Immersive Environment. 31st IEEE Conference On Virtual Reality And 3d User Interfaces (VR 2024), Mar 2024, Orlando, FL, United States. pp.670--680, ⟨10.1109/VR58804.2024.00087⟩. ⟨hal-04505824⟩
104 View
44 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More