Google has developed a system to recognize the facial expressions of VR helmet users with eye-tracking cameras and a machine learning algorithm. A technology that could create more realistic avatars in virtual reality for more lifelike social interactions.
If virtual reality is struggling to become more democratic, it is partly because it is not sufficiently immersive at the moment. There are several reasons for this: The graphics are unrealistic, the stimulations of the senses are limited to hearing and seeing, and it is not yet possible to interact directly with the hands. Moreover, the avatars representing the players on the VR are not strong enough.
Certainly, some applications like Facebook Spaces offer the user many customization options to create avatars that look like them. However, the facial expressions of these avatars are not synchronized with those of the user. Even if the user laughs or gets angry, his face remains uninspired. This significantly reduces the feeling of immersion and the realism of social interactions in virtual reality.
Google wants to recognize expressions in VR through eye tracking and machine learning
However, Google may have found the solution to the problem. The giant of Moutain View has just filed a patent for a system to recognize the different facial expressions of a VR helmet user. This system is based on gaze tracking cameras and an automatic learning algorithm (Machine Learning).
The cameras would make it possible to continuously monitor eye movements, and the algorithm would be trained to identify movements such as frowning, a wavy lip and much more. These different facial expressions can be associated with emotions such as surprise, joy or anger.
Google’s patent also describes how these expressions and emotions can be transferred in real time to the faces of avatars to make them more loyal to users. This would give avatars and users the same facial expression. Interactions in virtual reality would become much more natural, and the feeling of immersion and realism would be enhanced.