Virtual reality (VR) technology has been increasingly employed in human-robot interaction (HRI) research to enhance the immersion and realism of the interaction. However, the integration of VR into HRI also introduces new challenges, such as latency, mismatch between virtual and real environments and potential adverse effects on human users. Despite these challenges, the use of VR in HRI has the potential to provide numerous benefits, including improved communication, increased safety and enhanced training and education. Yet, little research has been done by scholars to review the state of the art of VR applications in human-robot interaction. To bridge the gap, this paper provides an overview of the challenges and benefits of using VR in HRI, as well as current research in the field and future directions for development. It has been found that robots are getting more personalized, interactive and engaging than ever; and with the popularization of virtual reality innovations, we might be able to foresee the wide adoption of VR in controlling robots to fulfill various tasks of hospitals, schools and factories. Still, there are several challenges, such as the need for more advanced VR technologies to provide more realistic and immersive experiences, the development of more human-like robot models to improve social interactions and the need for better methods of evaluating the effectiveness of VR in human-robot interaction.
Citation: Yu Lei, Zhi Su, Chao Cheng. Virtual reality in human-robot interaction: Challenges and benefits[J]. Electronic Research Archive, 2023, 31(5): 2374-2408. doi: 10.3934/era.2023121
Virtual reality (VR) technology has been increasingly employed in human-robot interaction (HRI) research to enhance the immersion and realism of the interaction. However, the integration of VR into HRI also introduces new challenges, such as latency, mismatch between virtual and real environments and potential adverse effects on human users. Despite these challenges, the use of VR in HRI has the potential to provide numerous benefits, including improved communication, increased safety and enhanced training and education. Yet, little research has been done by scholars to review the state of the art of VR applications in human-robot interaction. To bridge the gap, this paper provides an overview of the challenges and benefits of using VR in HRI, as well as current research in the field and future directions for development. It has been found that robots are getting more personalized, interactive and engaging than ever; and with the popularization of virtual reality innovations, we might be able to foresee the wide adoption of VR in controlling robots to fulfill various tasks of hospitals, schools and factories. Still, there are several challenges, such as the need for more advanced VR technologies to provide more realistic and immersive experiences, the development of more human-like robot models to improve social interactions and the need for better methods of evaluating the effectiveness of VR in human-robot interaction.
| [1] |
T. Dardona, S. Eslamian, L. A. Reisner, A. Pandya, Remote presence: Development and usability evaluation of a head-mounted display for camera control on the da vinci surgical system, Robotics, 8 (2019), 31. https://doi.org/10.3390/robotics8020031 doi: 10.3390/robotics8020031
|
| [2] |
L. Shi, C. Copot, S. Vanlanduit, Gazeemd: Detecting visual intention in gaze-based human-robot interaction, Robotics, 10 (2021), 68. https://doi.org/10.3390/robotics10020068 doi: 10.3390/robotics10020068
|
| [3] |
Y. P. Su, X. Q. Chen, T. Zhou, C. Pretty, G. Chase, Mixed-reality-enhanced human–robot interaction with an imitation-based mapping approach for intuitive teleoperation of a robotic arm-hand system, Appl. Sci., 12 (2022), 4740. https://doi.org/10.3390/app12094740 doi: 10.3390/app12094740
|
| [4] |
S. Mugisha, V. K. Guda, C. Chevallereau, M. Zoppi, R. Molfino, D. Chabalt, Improving haptic response for contextual human robot interaction, Sensors, 22 (2022), 2040. https://doi.org/10.3390/s22052040 doi: 10.3390/s22052040
|
| [5] |
D. Ververidis, S. Nikolopoulos, I. Kompatsiaris, A review of collaborative virtual reality systems for the architecture, engineering, and construction industry, Architecture, 2 (2022), 476–496. https://doi.org/10.3390/architecture2030027 doi: 10.3390/architecture2030027
|
| [6] |
M. Serras, L. García-Sardiña, B. Simões, H. Álvarez, J. Arambarri, Dialogue enhanced extended reality: Interactive system for the operator 4.0, Appl. Sci., 10 (2020), 3960. https://doi.org/10.3390/app10113960 doi: 10.3390/app10113960
|
| [7] | D. Uchechukwu, A. Siddique, A. Maksatbek, I. Afanasyev, ROS-based integration of smart space and a mobile robot as the internet of robotic things, in 2019 25th Conference of Open Innovations Association (FRUCT), IEEE, Helsinki, Finland, (2019), 339–345. https://doi.org/10.23919/FRUCT48121.2019.8981532 |
| [8] |
A. Gallala, A. A. Kumar, B. Hichri, P. Plapper, Digital Twin for human–robot interactions by means of industry 4.0 enabling technologies, Sensors, 22 (2022), 4950. https://doi.org/10.3390/s22134950 doi: 10.3390/s22134950
|
| [9] |
T. B. Sheridan, Human–robot interaction: status and challenges, Hum. Factors, 58 (2016), 525–532. https://doi.org/10.1177/0018720816644364 doi: 10.1177/0018720816644364
|
| [10] | M. Harbers, J. de Greeff, I. Kruijff-Korbayová, M. A. Neerincx, K. V. Hindriks, Exploring the ethical landscape of robot-assisted search and rescue, in A World with Robots: International Conference on Robot Ethics: ICRE 2015, Springer, (2017), 93–107. https://doi.org/10.1007/978-3-319-46667-5_7 |
| [11] | S. A. Akgun, M. Ghafurian, M. Crowley, K. Dautenhahn, Using affect as a communication modality to improve human-robot communication in robot-assisted search and rescue scenarios, arXiv preprint, (2022), arXiv: 2208.09580. https://doi.org/10.48550/arXiv.2208.09580 |
| [12] |
Y. Qian, S. Han, G. Aguirre-Ollinger, C. Fu, H. Yu, Design, modelling, and control of a reconfigurable rotary series elastic actuator with nonlinear stiffness for assistive robots, Mechatronics, 86 (2022), 102872. https://doi.org/10.1016/j.mechatronics.2022.102872 doi: 10.1016/j.mechatronics.2022.102872
|
| [13] | C. Getson, G. Nejat, The robot screener will see you now: A socially assistive robot for COVID-19 screening in long-term care homes, in 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), IEEE, Napoli, Italy, (2022), 672–677. https://doi.org/10.1109/RO-MAN53752.2022.9900620 |
| [14] |
F. Wang, C. Li, S. Niu, P. Wang, H. Wu, B. Li, Design and analysis of a spherical robot with rolling and jumping modes for deep space exploration, Machines, 10 (2022), 126. https://doi.org/10.3390/machines10020126 doi: 10.3390/machines10020126
|
| [15] | F. Gul, S. Mir, I. Mir, Coordinated multi-robot exploration: Hybrid stochastic optimization approach, in AIAA SCITECH 2022 Forum, (2022), 1414. https://doi.org/10.2514/6.2022-1414 |
| [16] | M. Liu, Design and development of assistive robotic arm, 2022. Available from: https://hdl.handle.net/10356/158578. |
| [17] |
R. A. Shveda, A. Rajappan, T. F. Yap, Z. Liu, M. D. Bell, B. Jumet, et al., A wearable textile-based pneumatic energy harvesting system for assistive robotics, Sci. Adv., 8 (2022), 34. https://doi.org/10.1126/sciadv.abo2418 doi: 10.1126/sciadv.abo2418
|
| [18] | D. A. Kumar, K. C. Rath, K. Muduli, F. Ajesh, Design and modeling of virtual robot for industrial application in smart manufacturing assembly line, in ntelligent Systems: Proceedings of ICMIB 2021, Springer, Singapore, (2022), 471–483. https://doi.org/10.1007/978-981-19-0901-6_42 |
| [19] | C. S. Crawford, M. Andujar, J. E. Gilbert, Neurophysiological heat maps for human-robot interaction evaluation, in Proceedings of 2017 AAAI Fall Symposium Series: Artificial Intelligence for Human-Robot Interaction AAAI Technical Report FS-17-01, AAAI, Arlington, USA, (2017), 90–93. |
| [20] | R. R. Murphy, V. B. M. Gandudi, J. Adams, Applications of robots for COVID-19 response, Robotics, arXiv preprint, (2020), arXiv: 2008.06976. https://doi.org/10.48550/arXiv.2008.06976 |
| [21] | T. Manglani, R. Rani, R. Kaushik, P. K. Singh, Recent trends and challenges of diverless vehicles in real world application, in 2022 International Conference on Sustainable Computing and Data Communication Systems (ICSCDS), IEEE, Erode, India, (2022), 803–806. https://doi.org/10.1109/ICSCDS53736.2022.9760886 |
| [22] |
R. Monterubbianesi, V. Tosco, F. Vitiello, G. Orilisiet, F. Fraccastoro, A. Putignano, et al., Augmented, Virtual and Mixed Reality in Dentistry: A narrative review on the existing platforms and future challenges, Appl. Sci., 12 (2022), 877. https://doi.org/10.3390/app12020877 doi: 10.3390/app12020877
|
| [23] |
C. Antonya, S. Butnariu, Preservation of cultural heritage using virtual reality technologies and haptic feedback: A prototype and case study on antique carpentry tools, Appl. Sci., 12 (2022), 8002. https://doi.org/10.3390/app12168002 doi: 10.3390/app12168002
|
| [24] | L. G. Akhmaeva, D. V. Dolgopolov, A. I. Eremeeva, Demographic features of interconnection between VR and gaming experience on consumer market, in Proceedings of the International Scientific Conference "Smart Nations: Global Trends In The Digital Economy", Springer, (2022), 205–212. https://doi.org/10.1007/978-3-030-94870-2_27 |
| [25] |
D. J. Thomas, Augmented reality in surgery: The computer-aided medicine revolution, Int. J. Surg., 36 (2016), 25. https://doi.org/10.1016/j.ijsu.2016.10.003 doi: 10.1016/j.ijsu.2016.10.003
|
| [26] |
S. Y. Lee, J. Y. Cha, J. W. Yoo, M. Nazareno, Y. S. Cho, S. Y. Joo, et al., Effect of the application of virtual reality on pain reduction and cerebral blood flow in robot-assisted gait training in burn patients, J. Clin. Med., 11 (2022), 3762. https://doi.org/10.3390/jcm11133762 doi: 10.3390/jcm11133762
|
| [27] | L. Freina, M. Ott, A literature review on immersive virtual reality in education: state of the art and perspectives, eLSE, 1 (2015), 133–141. |
| [28] | S. Kavanagh, A. Luxton-Reilly, B. Wuensche, B. Plimmer, A systematic review of virtual reality in education, Themes Sci. Technol. Educ., 10 (2017), 85–119. |
| [29] |
L. Jensen, F. Konradsen, A review of the use of virtual reality head-mounted displays in education and training, Educ. Inf. Technol., 23 (2018), 1515–1529. https://doi.org/10.1007/s10639-017-9676-0 doi: 10.1007/s10639-017-9676-0
|
| [30] |
G. Riva, Virtual reality for health care: the status of research, Cyberpsychol. Behav., 5 (2002), 219–225. https://doi.org/10.1089/109493102760147213 doi: 10.1089/109493102760147213
|
| [31] |
A. J. Snoswell, C. L. Snoswell, Immersive virtual reality in health care: systematic review of technology and disease states, JMIR Biomed. Eng., 4 (2019), e15025. https://doi.org/10.2196/15025 doi: 10.2196/15025
|
| [32] | W. Thabet, M. F. Shiratuddin, D. Bowman, Virtual reality in construction: a review, in Engineering computational technology, Civil-Comp press, Edinburgh, UK, (2002), 25–52. https://doi.org/10.4203/csets.8.2 |
| [33] | M. Bassanino, K. C. Wu, J. Yao, F. Khosrowshahi, T. Fernando, J. Skjærbæk, The impact of immersive virtual reality on visualisation for a design review in construction, in 2010 14th International Conference Information Visualisation, IEEE, London, UK, (2010), 585–589. https://doi.org/10.1109/IV.2010.85 |
| [34] |
C. P. Ortet, A. I. Veloso, L. V. Costa, Cycling through 360° virtual reality tourism for senior citizens: Empirical analysis of an assistive technology, Sensors, 22 (2022), 6169. https://doi.org/10.3390/s22166169 doi: 10.3390/s22166169
|
| [35] |
D. Yu, X. Li, F. H. Y. Lai, The effect of virtual reality on executive function in older adults with mild cognitive impairment: a systematic review and meta-analysis, Aging Mental Health, 1 (2022), 1–11. https://doi.org/11.10.1080/13607863.2022.2076202 doi: 10.1080/13607863.2022.2076202
|
| [36] |
H. B. Abdessalem, M. Cuesta, S. Belleville, C. Frasson, Virtual reality zoo therapy: An interactive relaxing system for Alzheimer's disease, J. Exp. Neurol., 3 (2022), 15–19. https://doi.org/10.1007/978-3-030-78775-2_12. doi: 10.1007/978-3-030-78775-2_12.
|
| [37] | A. Schweidler, "Mixed Reality Bike"–Development and Evaluation of a Virtual Reality Cycling Simulator: Wien, Ph.D thesis, E193-Institut für Visual Computing and Human-Centered Technology, 2022. |
| [38] | R. Radoeva, E. Petkov, T. Kalushkov, D. Valcheva, G. Shipkovenski, Overview on hardware characteristics of virtual reality systems, in 2022 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA), IEEE, Ankara, Turkey, (2022), 01–05. https://doi.org/10.1109/HORA55278.2022.9799932 |
| [39] |
V. Z. Pérez, J. C. Yepes, J. F. Vargas, J. C. Franco, N. I. Escobar, L. Betancur, et al., Virtual reality game for physical and emotional rehabilitation of landmine victims, Sensors, 22 (2022), 5602. https://doi.org/10.3390/s22155602 doi: 10.3390/s22155602
|
| [40] |
A. Kolk, M. Saard, A. Roštšinskaja, K. Sepp, C. Kööp, Power of combined modern technology: Multitouch-multiuser tabletops and virtual reality platforms (PowerVR) in social communication skills training for children with neurological disorders: A pilot study, Child Neuropsychol., 1 (2022), 10. https://doi.org/10.1080/21622965.2022.2066532 doi: 10.1080/21622965.2022.2066532
|
| [41] |
G. C. Burdea, Virtual rehabilitation–benefits and challenges, Methods Inf. Med., 42 (2003), 519–523. https://doi.org/10.1055/s-0038-1634378 doi: 10.1055/s-0038-1634378
|
| [42] |
S. F. Hardon, A. Kooijmans, R. Horeman, M. van der Elst, A. L. Bloemendaal, T. Horeman, Validation of the portable virtual reality training system for robotic surgery (PoLaRS): a randomized controlled trial, Surg. Endosc., 36 (2022), 5282–5292. https://doi.org/10.1007/s00464-021-08906-z doi: 10.1007/s00464-021-08906-z
|
| [43] | B. Seeliger, J. W. Collins, F. Porpiglia, J. Marescaux, The role of virtual reality, telesurgery, and teleproctoring in robotic surgery, in Robotic Urologic Surgery, Springer, (2022), 61–77. https://doi.org/10.1007/978-3-031-00363-9_8 |
| [44] |
K. Sevcenko, I. Lindgren, The effects of virtual reality training in stroke and Parkinson's disease rehabilitation: a systematic review and a perspective on usability, Eur. Rev. Aging Phys. Act., 19 (2022), 1–16. https://doi.org/10.1186/s11556-022-00283-3 doi: 10.1186/s11556-021-00281-x
|
| [45] |
N. Puodžiūnienė, E. Narvyda, Standards for transition from 2D drawing to model based definition in mechanical engineering, Archives, 27 (2021), 351–354. https://doi.org/10.5755/j02.mech.25777 doi: 10.5755/j02.mech.25777
|
| [46] |
E. Touloupaki, T. Theodosio, Performance simulation integrated in parametric 3D modeling as a method for early stage design optimization—A review, Energies, 10 (2017), 637. https://doi.org/10.3390/en10050637 doi: 10.3390/en10050637
|
| [47] |
S. B. Tomlinson, B. K. Hendricks, A. Cohen-Gadol, Immersive three-dimensional modeling and virtual reality for enhanced visualization of operative neurosurgical anatom, World Neurosurg., 131 (2019), 313–320. https://doi.org/10.1016/j.wneu.2019.06.081 doi: 10.1016/j.wneu.2019.06.081
|
| [48] |
S. T. Puente, L. Más, F. Torres, F. A. Candelas, Virtualization of robotic hands using mobile devices, Robotics, 8 (2019), 81. https://doi.org/10.3390/robotics80300818:81. doi: 10.3390/robotics8030081
|
| [49] |
S. L. Pizzagalli, V. Kuts, T. Otto, User-centered design for Human-Robot Collaboration systems, IOP Conf. Ser.: Mater. Sci. Eng., 1140 (2021), 012011. https://doi.org/10.1088/1757-899X/1140/1/012011 doi: 10.1088/1757-899X/1140/1/012011
|
| [50] | K. Pasanen, J. Pesonen, J. Murphy, J. Heinonen, J. Mikkonen, Comparing tablet and virtual reality glasses for watching nature tourism videos, in Information and Communication Technologies in Tourism, Springer, (2019), 120–131. 10.1007/978-3-030-05940-8_10. |
| [51] |
H. Çetin, The impact of ICT patents on OECD countries' banks risk indicators and discussion on the use of robotic communication and smart glasses in the banking sector, Turk. Stud. Econ. Finance Polit., 15 (2020), 799–816. https://doi.org/10.29228/TurkishStudies.42916 doi: 10.29228/TurkishStudies.42916
|
| [52] | T. Hachaj, Head motion–based robot's controlling system using virtual reality glasses, in Image Processing and Communications: Techniques, Algorithms and Applications 11, Springer, (2019), 6–13. https://doi.org/10.1007/978-3-030-31254-1_2 |
| [53] | Q. C. Ihemedu-Steinke, R. Erbach, P. Halady, G. Meixner, M. Weber, Virtual reality driving simulator based on head-mounted displays, in Automotive User Interfaces, Springer, (2017), 401–428. https://doi.org/10.1007/978-3-319-49448-7_15 |
| [54] |
A. El hafidy, T. Rachad, A. Idri, A. Zellou, Gamified mobile applications for improving driving behavior: A systematic mapping study, Mobile Inf. Syst., 2021 (2021), 6677075. https://doi.org/10.1155/2021/6677075 doi: 10.1155/2021/6677075
|
| [55] | J. Wade, D. Bian, L. Zhang, A. Swanson, M. Sarkar, Z. Warren, et al., Design of a virtual reality driving environment to assess performance of teenagers with ASD, in Universal Access in Human-Computer Interaction. Universal Access to Information and Knowledge, Springer, (2014), 466–474. https://doi.org/10.1007/978-3-319-07440-5_43 |
| [56] |
J. K. Muguro, P. W. Laksono, Y. Sasatake, K. Matsushita, M. Sasaki, User monitoring in autonomous driving system using gamified task: A case for VR/AR in-car gaming, Multimodal Technol. Interact., 5 (2021), 40. https://doi.org/10.3390/mti5080040 doi: 10.3390/mti5080040
|
| [57] | A. M. Nascimento, A. C. M. Queiroz, L. F. Vismari, J. N. Bailenson, P. S. Cugnasca, J. B. C. Junior, et al., The role of virtual reality in autonomous vehicles' safety, in 2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR), IEEE, San Diego, USA, (2019), 50–507. https://doi.org/10.1109/AIVR46125.2019.00017 |
| [58] |
L. Morra, F. Lamberti, F. G. Pratticó, S. L. Rosa, P. Montuschi, Building trust in autonomous vehicles: Role of virtual reality driving simulators in HMI design, IEEE Trans. Veh. Technol., 68 (2019), 9438–9450. https://doi.org/10.1109/TVT.2019.2933601 doi: 10.1109/TVT.2019.2933601
|
| [59] | J. Helgath, P. Braun, A. Pritschet, M. Schubert, P. Böhm, D. Isemann, Investigating the effect of different autonomy levels on user acceptance and user experience in self-driving cars with a VR driving simulator, in Design, User Experience, and Usability: Users, Contexts and Case Studies, Springer, (2018), 247–256. https://doi.org/10.1007/978-3-319-91806-8_19 |
| [60] |
A. D. Pieterse, B. P. Hierck, P. G. M. de Jong, J. Kroese, L. N. A. Willems, M. E. J. Reinders, Design and implementation of "AugMedicine: Lung Cases, " an augmented reality application for the medical curriculum on the presentation of dyspnea, Front. Virtual Reality, 1 (2020), 577534. https://doi.org/10.3389/frvir.2020.577534 doi: 10.3389/frvir.2020.577534
|
| [61] | C. Chinnock, Virtual reality in surgery and medicine, Hosp. Technol. Ser., 13 (1994), 1–48. |
| [62] |
C. Pensieri, M. Pennacchini, Overview: Virtual reality in medicine, J. Virtual Worlds Res., 7 (2014). https://doi.org/10.4101/jvwr.v7i1.6364 doi: 10.4101/jvwr.v7i1.6364
|
| [63] |
Y. Kim, H. Kim, Y. O. Kim, Virtual reality and augmented reality in plastic surgery: A review, Arch. Plast. Surg., 44 (2017), 179–187. https://doi.org/10.5999/aps.2017.44.3.179 doi: 10.5999/aps.2017.44.3.179
|
| [64] |
B. Fiani, R. Jenkins, I. Siddiqi, A. Khan, A. Taylor, Socioeconomic impact of COVID-19 on spinal instrumentation companies in the era of decreased elective surgery, Cureus, 12 (2020), 9776. https://doi.org/10.7759/cureus.9776 doi: 10.7759/cureus.9776
|
| [65] |
T. Morimoto, T. Kobayashi, H. Hirata, K. Otani, M. Sugimoto, M. Tsukamoto, et al., XR (extended reality: virtual reality, augmented reality, mixed reality) technology in spine medicine: status quo and quo vadis, J. Clin. Med., 11 (2022), 470. https://doi.org/10.3390/jcm1102047067 doi: 10.3390/jcm11020470
|
| [66] |
T. Morimoto, Hirata H., M. Ueno, N. Fukumori, T. Sakai, M. Sugimoto, et al., Digital transformation will change medical education and rehabilitation in spine surgery, Medicina (Kaunas), 58(2022), 508. doi: 10.3390/medicina58040508 doi: 10.3390/medicina58040508
|
| [67] |
J. D. Bric, D. C. Lumbard, M. J. Frelich, J. C. Gould, Current state of virtual reality simulation in robotic surgery training: a review, Surg. Endosc., 30 (2016), 2169–2178. https://doi.org/10.1007/s00464-015-4517-y doi: 10.1007/s00464-015-4517-y
|
| [68] |
A. Moglia, V. Ferrari, L. Morelli, M. Ferrari, F. Mosca, A. Cuschieri, A systematic review of virtual reality simulators for robot-assisted surgery, Eur. Urol., 69 (2016), 1065–1080. https://doi.org/10.1016/j.eururo.2015.09.021 doi: 10.1016/j.eururo.2015.09.021
|
| [69] |
G. I. Lee, M. R. Lee, Can a virtual reality surgical simulation training provide a self-driven and mentor-free skills learning? Investigation of the practical influence of the performance metrics from the virtual reality robotic surgery simulator on the skill learning and associated cognitive workloads, Surg. Endosc., 32 (2018), 62–72. https://doi.org/10.1007/s00464-017-5634-6 doi: 10.1007/s00464-017-5634-6
|
| [70] | E. A. L. Lee, K. W. Wong, A review of using virtual reality for learning, in Transactions on Edutainment I, Springer, (2008), 231–241. https://doi.org/10.1007/978-3-540-69744-2_18 |
| [71] |
D. Allcoat, A. von Mühlenen, Learning in virtual reality: Effects on performance, emotion and engagement, Res. Learn. Technol., 26 (2018), 2140. https://doi.org/10.25304/rlt.v26.2140 doi: 10.25304/rlt.v26.2140
|
| [72] |
A. L. Butt, S. Kardong-Edgren, A. Ellertson, Using game-based virtual reality with haptics for skill acquisition, Clin. Simul. Nurs., 16 (2018), 25–32. https://doi.org/10.1016/j.ecns.2017.09.010 doi: 10.1016/j.ecns.2017.09.010
|
| [73] |
V. Román-Ibáñez, F. A. Pujol-López, H. Mora-Mora, M. L. Pertegal-Felices, A. Jimeno-Morenilla, A low-cost immersive virtual reality system for teaching robotic manipulators programming, Sustainability, 10 (2018), 1102. https://doi.org/10.3390/su10041102 doi: 10.3390/su10041102
|
| [74] |
K. Eliahu, J. Liounakos, M. Y. Wang, Applications for augmented and virtual reality in robot-assisted spine surgery, Curr. Rob. Rep., 3 (2022), 33–37. https://doi.org/10.1007/s43154-022-00073-w doi: 10.1007/s43154-022-00073-w
|
| [75] |
C. M. Lo, J. H. Wang, H. W. Wang, Virtual reality human–robot interaction technology acceptance model for learning direct current and alternating current, J. Supercomput., 78 (2022), 15314–15337. https://doi.org/10.1007/s11227-022-04455-x doi: 10.1007/s11227-022-04455-x
|
| [76] | Y. L. Chen, C. C. Hsu, C. Y. Lin, H. H. Hsu, Personalized English language learning through a robot and virtual reality platform, in Proceedings of Society for Information Technology & Teacher Education International Conference, AACE, San Diego, United States, (2022), 1612–1615. |
| [77] |
M. Soori, B. Arezoo, M. Habibi, Dimensional and geometrical errors of three-axis CNC milling machines in a virtual machining system, Comput.-Aided Des., 45 (2013), 1306–1313. https://doi.org/10.1016/j.cad.2013.06.002 doi: 10.1016/j.cad.2013.06.002
|
| [78] |
Y. Altintas, C. Brecher, M. Weck, S. Witt, Virtual machine tool, CIRP Ann., 54 (2005), 115–138. https://doi.org/10.1016/S0007-8506(07)60022-5 doi: 10.1016/S0007-8506(07)60022-5
|
| [79] |
C. F. Cheung, W. B. Lee, A framework of a virtual machining and inspection system for diamond turning of precision optics, J. Mater. Process. Technol., 119 (2001), 27–40. https://doi.org/10.1016/S0924-0136(01)00893-7 doi: 10.1016/S0924-0136(01)00893-7
|
| [80] |
H. Narita, K. Shirase, H. Wakamatsu, E. Arai, Pre-process evaluation of end milling operation using virtual machining simulator, Jpn. Soc. Mech. Eng. Int. J. Ser. C, 43 (2000), 492–497. https://doi.org/10.1299/jsmec.43.492 doi: 10.1299/jsmec.43.492
|
| [81] |
N. Slavkovic, S. Zivanovic, B. Kokotovic, Z. Dimic, M. Milutinovic, Simulation of compensated tool path through virtual robot machining model, J. Braz. Soc. Mech. Sci. Eng., 42 (2020), 1–17. https://doi.org/10.1007/s40430-020-02461-9 doi: 10.1007/s40430-019-2074-3
|
| [82] | J. Nilsson, M. Ericsson, F. Danielsson, Virtual machine vision in computer aided robotics, in 2009 IEEE Conference on Emerging Technologies & Factory Automation, IEEE, Palma de Mallorca, Spain, (2009), 1–8. https://doi.org/10.1109/ETFA.2009.5347003 |
| [83] |
L. Chen, F. Zhang, W. Zhan, M. Gan, L. Sun, Optimization of virtual and real registration technology based on augmented reality in a surgical navigation system, BioMed. Eng. OnLine, 19 (2020), 1. https://doi.org/10.1186/s12938-019-0745-z doi: 10.1186/s12938-019-0745-z
|
| [84] |
R. Wen, C. B. Chng, C. K. Chui, Augmented reality guidance with multimodality imaging data and depth-perceived interaction for robot-assisted surgery, Robotics, 6 (2017), 13. https://doi.org/10.3390/robotics6020013 doi: 10.3390/robotics6020013
|
| [85] |
A. K. Lefor, S. A. Heredia Pérez, A. Shimizu, H. C. Lin, J. Witowski, Development and validation of a virtual reality simulator for robot-assisted minimally invasive liver surgery training, J. Clin. Med., 11 (2022), 4145. https://doi.org/10.3390/jcm11144145 doi: 10.3390/jcm11144145
|
| [86] | P. Shi, Study on a novel virtual reality interventional training system for robot-assisted interventional surgery, Ph.D thesis, Kagawa University, 2022. |
| [87] |
A. J. Berges, S. S. Vedula, A. Malpani, C. C. G. Chen, Virtual reality simulation has weak correlation with overall trainee robot-assisted laparoscopic hysterectomy performance, J. Minimally Invasive Gynecol., 29 (2022), 507–518. https://doi.org/10.1016/j.jmig.2021.12.002 doi: 10.1016/j.jmig.2021.12.002
|
| [88] |
B. Der, D. Sanford, R. Hakim, E. Vanstrum, J. H. Nguyen, A. J. Hung, Efficiency and accuracy of robotic surgical performance decayed among urologists during COVID-19 shutdown, J. Endourology, 35 (2021), 888–890. http://doi.org/10.1089/end.2020.0869 doi: 10.1089/end.2020.0869
|
| [89] |
M. O. Parvez, A. Öztüren, C. Cobanoglu, H. Arasli, K. K. Eluwole, Employees' perception of robots and robot-induced unemployment in hospitality industry under COVID-19 pandemic, Int. J. Hosp. Manag., 107 (2022), 103336. https://doi.org/10.1016/j.ijhm.2022.103336 doi: 10.1016/j.ijhm.2022.103336
|
| [90] | S. Alves, A. Quevedo, D. Chen, J. Morris, S. Radmard, Leveraging simulation and virtual reality for a long term care facility service robot during COVID-19, in Symposium on Virtual and Augmented Reality (SVR'21), ACM, New York, USA, (2021), 187–191. https://doi.org/10.1145/3488162.3488185 |
| [91] |
A. Fujihara, O. Ukimura, Virtual reality of three-dimensional surgical field for surgical planning and intraoperative management, World J. Urol., 40 (2022), 687–696. https://doi.org/10.1007/s00345-021-03841-z doi: 10.1007/s00345-021-03841-z
|
| [92] | J. Y. M. Villamizar, I. Ostan, D. A. E. Ortega, A. A. G. Siqueira, Remote control architecture for virtual reality application for ankle therapy, in XXVⅡ Brazilian Congress on Biomedical Engineering, Springer, (2022), 517–522. https://doi.org/10.1007/978-3-030-70601-2_80 |
| [93] |
M. Motaharifar, A. Norouzzadeh, P. Abdi, A. Iranfar, F. Lotfi, B. Moshiri, et al., Applications of haptic technology, virtual reality, and artificial intelligence in medical training during the COVID-19 pandemic, Front. Rob. AI, 8 (2021), 612949. https://doi.org/10.3389/frobt.2021.612949 doi: 10.3389/frobt.2021.612949
|
| [94] |
F. Sanfilippo, T. Blazauskas, G. Salvietti, I. Ramos, S. Vert, J. Radianti, et al., A Perspective Review on Integrating VR/AR with haptics into STEM education for multi-sensory learning, Robotics, 11 (2022), 41. https://doi.org/10.3390/robotics11020041 doi: 10.3390/robotics11020041
|
| [95] |
M. Shahab, A. Taheri, M. Mokhtari, A. Shariati, R. Heidari, A. Meghdari, et al., Utilizing social virtual reality robot (V2R) for music education to children with high-functioning autism, Educ. Inf. Technol., 27 (2022), 819–843. https://doi.org/10.1007/s10639-020-10392-0 doi: 10.1007/s10639-020-10392-0
|
| [96] |
G. Du, Y. Li, K. Su, C. Li, P. X. Liu, A mobile natural human-robot interaction method for virtual chinese acupuncture, IEEE Trans. Instrum. Meas., 72 (2023), 1–10. https://doi.org/10.1109/TIM.2022.3201202 doi: 10.1109/TIM.2022.3201202
|
| [97] |
Y. J. Kim, H. S. Nam, W. H. Lee, H. G. Seo, J. H. Leigh, B. M. Oh, et al., Vision-aided brain–machine interface training system for robotic arm control and clinical application on two patients with cervical spinal cord injury, Biomed. Eng. Online, 18 (2019), 1–21, https://doi.org/10.1186/s12938-019-0633-6 doi: 10.1186/s12938-018-0620-3
|
| [98] | D. Szafir, D. A. Szafir, Connecting human-robot interaction and data visualization, in Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction, ACM, Boulder, USA, (2021), 281–292, https://doi.org/10.1145/3434073.3444683 |
| [99] |
A. Naceri, A. Moscatelli, R. Chellali, Depth discrimination of constant angular size stimuli in action space: role of accommodation and convergence cues, Front. Hum. Neurosci., 9 (2015), 511. https://doi.org/10.3389/fnhum.2015.00511 doi: 10.3389/fnhum.2015.00511
|
| [100] |
O. Cohavi, S. L.Tzedek, Young and old users prefer immersive virtual reality over a social robot for short-term cognitive training, Int. J. Hum.-Comput. Stud., 161 (2022), 102775. https://doi.org/10.1016/j.ijhcs.2022.102775 doi: 10.1016/j.ijhcs.2022.102775
|
| [101] |
C. E. Agüero, N. Koenig, I. Chen, H. Boyer, S. Peters, J. Hsu, et al., Inside the virtual robotics challenge: Simulating real-time robotic disaster response, IEEE Trans. Autom. Sci. Eng., 12 (2015), 494–506. https://doi.org/10.1109/TASE.2014.2368997 doi: 10.1109/TASE.2014.2368997
|
| [102] |
P. Samuels, L. Haapasalo, Real and virtual robotics in mathematics education at the school–university transition, Int. J. Math. Educ. Sci. Technol., 43 (2012), 285–301. https://doi.org/10.1080/0020739X.2011.618548 doi: 10.1080/0020739X.2011.618548
|
| [103] | B. H. Alsoliman, Virtual robotics in education: The experience of eighth grade students in STEM, in Frontiers in Education, (2022), 1–12, https://doi.org/10.3389/feduc.2022.950766 |
| [104] |
S. B. I. Badia, P. A. Silva, D. Branco, A. Pinto, C. Carvalho, P. Menezes, et al., Virtual reality for safe testing and development in collaborative robotics: Challenges and perspectives, Electronics 2022, 11 (2022), 1726; https://doi.org/10.3390/electronics11111726 doi: 10.3390/electronics11111726
|
| [105] | G. Palmieri, C. Scoccia, D. Costa, M. Callegari, Development of a virtual reality application for the assessment of human-robot collaboration tasks, in Advances in Service and Industrial Robotics: RAAD 2022, Springer (2022), 597–604. https://doi.org/10.1007/978-3-031-04870-8_70 |
| [106] |
T. Togias, C. Gkournelos, P. Angelakis, G. Michalos, S. Makris, Virtual reality environment for industrial robot control and path design, Procedia CIRP, 100 (2021), 133–138. https://doi.org/10.1016/j.procir.2021.05.021 doi: 10.1016/j.procir.2021.05.021
|
| [107] | S. Tellex, E. Rosen, D. Whitney, E. Phillips, D. Ullman, Testing robot teleoperation using a virtual reality interface with ROS reality, in Proceedings of the 1st International Workshop on Virtual, Augmented, and Mixed Reality for HRI (VAM-HRI), (2018), 1–4. |
| [108] |
S. Papanastasiou, N. Kousi, P. Karagiannis, C. Gkournelos, A. Papavasileiou, K. Dimoulas, et al., Towards seamless human robot collaboration: integrating multimodal interaction, Int. J. Adv. Manuf. Technol., 105 (2019), 3881–3897. https://doi.org/10.1007/s00170-019-03790-3 doi: 10.1007/s00170-019-03790-3
|
| [109] | S Makris, Virtual reality for programming cooperating robots based on human motion mimicking, in Cooperating Robots for Flexible Manufacturing, Springer, (2021), 339–353. https://doi.org/10.1007/978-3-030-51591-1_18 |
| [110] |
A. C. Simões, A. Pinto, J. Santos, S. Pinheiro, D. Romero, Designing human-robot collaboration (HRC) workspaces in industrial settings: A systematic literature review, J. Manuf. Syst., 62 (2022), 28–43. https://doi.org/10.1016/j.jmsy.2021.11.007 doi: 10.1016/j.jmsy.2021.11.007
|
| [111] |
M. Dianatfar, J. Latokartano, M. Lanz, Review on existing VR/AR solutions in human–robot collaboration, Procedia CIRP, 97 (2021), 407–411. https://doi.org/10.1016/j.procir.2020.05.259 doi: 10.1016/j.procir.2020.05.259
|
| [112] |
P. K. BN, A. Balasubramanyam, A. K. Patil, Y. H. Chai, GazeGuide: An eye-gaze-guided active immersive UAV camera, Appl. Sci., 10 (2020), 1668. https://doi.org/10.3390/app10051668 doi: 10.3390/app10051668
|
| [113] | P. Higgins, G. Y. Kebe, A. Berlier, K. Darvish, D. Engel, F. Ferraro, et al., Towards making virtual human-robot interaction a reality, in Proc. of the 3rd International Workshop on Virtual, Augmented, and Mixed-Reality for Human-Robot Interactions, (2021). |
| [114] |
A. Bustamante, L. M. Belmonte, R. Morales, A. Pereira, A. Fernández-Caballero, Video processing from a virtual unmanned aerial vehicle: Comparing two approaches to using OpenCV in unity, Appl. Sci., 12 (2022), 5958. https://doi.org/10.3390/app12125958 doi: 10.3390/app12125958
|
| [115] |
S. Hjorth, D. Chrysostomou, C. I. Manufacturing, Human–robot collaboration in industrial environments: A literature review on non-destructive disassembly, Rob. Comput. Integr. Manuf., 73 (2022), 102208. https://doi.org/10.1016/j.rcim.2021.102208 doi: 10.1016/j.rcim.2021.102208
|
| [116] | M. Rana, Using virtual reality for simulation of underwater human-robot interaction, in UMTC Undergraduate Research Presentations and Papers, (2021). |
| [117] |
M. de la Cruz, G. Casañ, P. Sanz, R. Marin, Preliminary work on a virtual reality interface for the guidance of underwater robots, Robotics, 9 (2020), 81. https://doi.org/10.3390/robotics9040081 doi: 10.3390/robotics9040081
|
| [118] | M. Wonsick, T. Padır, Human-humanoid robot interaction through virtual reality interfaces, in 2021 IEEE Aerospace Conference, IEEE, Big Sky, USA, (2021), 1–7. https://doi.org/10.1109/AERO50100.2021.9438400 |
| [119] |
T. Inamura, Y. Mizuchi, SIGVerse: A cloud-based VR platform for research on multimodal human-robot interaction, Front. Rob. AI, 8 (2021), 549360. https://doi.org/10.3389/frobt.2021.549360 doi: 10.3389/frobt.2021.549360
|
| [120] |
E. Prati, V. Villani, M. Peruzzini, L. Sabattini, An approach based on VR to design industrial human-robot collaborative workstations, Appl. Sci., 11 (2021), 11773. https://doi.org/10.3390/app112411773 doi: 10.3390/app112411773
|
| [121] | T. Williams, D. Szafir, T. Chakraborti, H. Ben Amor, Virtual, augmented, and mixed reality for human-robot interaction (VAM-HRI), in Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, ACM, Cambridge, UK, (2020), 663–664. https://doi.org/10.1145/3371382.3374850 |
Figures(18)
Yu Lei, Zhi Su, Chao Cheng. Virtual reality in human-robot interaction: Challenges and benefits[J]. Electronic Research Archive, 2023, 31(5): 2374-2408. doi: 10.3934/era.2023121
DownLoad: