Google announced the GameFace project at last year's I/O developer conference, allowing players to control PC game characters through facial expressions; at this year's I/O conference, Google announced the open-source code to Android developers.
Developers can integrate GameFace accessibility features into their applications, allowing users to control the cursor through facial gestures or head movements. For example, users can open their mouths to move the cursor, or raise eyebrows to drag and drop.
The GameFace project uses the device's camera, and also calls the MediaPipe FaceLandmarksDetectionAPI to access the facial expression database to manipulate the cursor. TapTechNews provides a demo video below:
Google explained in the announcement: Through the device's camera, it can seamlessly track facial expressions and head movements, and translate them into intuitive personalized controls. Developers can now create applications that allow users to customize their experience through custom facial expressions, gesture size, cursor speed, etc.
While Gameface was initially designed for gamers, Google stated that it is also collaborating with Incluzza, a social enterprise in India focused on accessibility, to research how to extend it to other settings such as work, schools, and social gatherings.
Related reading:
《Google launches GameFace project, allowing players to control PC games through facial expressions》
2024 Google I/O Developer Conference Special