Platform for immersive training and activity evaluation based on virtual reality and artificial intelligence
As a result of V-TREV experiment a virtual reality system for the on-site training of workers in the industrial sector using VR technology has been designed and developed. Moreover, it has been evaluated in a real environment with the collaboration of a large chemical company.
This solution has a double purpose. First, to ease the training in technical processes by providing an extensible and flexible system. Secondly, the real-time evaluation of the trainee performance to provide insights to their supervisors on which parts of the process seem to be harder for employees or which ones must be modified.
This project takes advantage of the last VR headset technologies, which allow standalone six-degrees-of-freedom (6DOF) tracking (which means that the user can move, turn, bend, crouch and much more) with wireless goggles. The wireless communication feature is used to send data to an independent platform in which the evaluation process is conducted.
V-TREV comprises three clearly separated modules:
- The first module includes a virtual reality application running in a standalone virtual reality head mounted device (Oculus Quest) with wireless capabilities.
- The second module takes advantage of the wireless feature of the former module to build a platform for the evaluation of the data gathered by the application running in the first module. This module is executed in a cloud environment, creating a system which receives and processes data streams from the first module, can immediately feedback results if required, and stores the data.
- The third module consists of a dashboard web application for monitoring, evaluating and presenting the data stored in the second module.
V-TREV makes use of innovative virtual reality technologies such as the Oculus Quest device and the last VR toolkits and frameworks as well as open-source software from the MIDIH reference architecture such as Apache Kafka.
Economical / Business impact
Technical operations in industrial factories are defined by extremely sensitive processes to produce highly specific products, involving several different types of machinery and hundreds, if not thousands, of steps that have to be meticulously followed with as few errors as possible to obtain maximum efficiency and economic profits.
Training in industrial factories is turning into an extensively complex task. Staff is given heavy, long, tedious, fuzzy documentation they have to study and make an abstraction of what is written there, and then try to apply it in a real scenario with little margin of error. This current training process produces stress, low self-confidence and other issues that can lead to mistakes that could be avoided.
V-TREV is IZERTIS’ proposal to solve this problem. Our solution is led by virtual reality along with data monitoring: virtual scenarios for training employees in a safe and comfortable environment, with interactive guided procedures that the trainees can repeat as much as they desire. These procedures can be extended, modified, or updated anytime it is required, creating a valuable tool for companies that embrace a new employee training paradigm.
As a result of the dissemination and exploitation activities performed within the project, 54 entities, including industries, services providers, public administration, academia and R&D centers from the metal, agri-food, textile, equipment and ITC sectors have been individually reached to inform them about V-TREV. 5 of them have shown their commercial interest in the application of V-TREV results in their training processes. Although due to the COVID-19 pandemic it has not been possible to reach as many potential customers for the technology developed during the experiment as expected, or even holding face-to-face demonstration meetings, the feedback received so far proves that this technology can make a big difference in the training process.
Information source: Midih
Virtual reality system