Summary
- Project Gameface by Google, an open-source hands-free “mouse” using facial recognition, is now available for developers to enhance the accessibility of their Android apps.
- It offers low-cost accessibility for gamers with disabilities, responding to facial movements like raising eyebrows to execute actions.
- This customizable technology supports up to 52 distinct gestures, enabling users to set their own thresholds for sensitivity to control facial commands.
At this week’s Google I/O, while the spotlight was on major AI innovations, Google didn’t miss updating us on its facial recognition technology. Called Project “Gameface,” which was originally unveiled at last year’s Google I/O conference, it’s essentially a hands-free gaming “mouse” that lets you move a computer cursor using just your head and face. This time around, the company has gone a step further by releasing more of the code to the public, which means developers can now use it to enhance Android apps with the same technology.
Project Gameface operates through the device’s camera and taps into MediaPipe’s Face Landmarks Detection API to track a range of facial expressions and move the cursor accordingly. The aim is to offer a low-cost solution to make gaming more accessible for individuals with disabilities, as per Google’s blog post. At its core, it’s an open-source, hands-free “mouse” that responds to head and facial movements. For example, by simply raising your eyebrows, you can execute actions like clicking and dragging or navigating back to the home screen, and so on.
5 biggest Google I/O announcements: Circle to Search, Search changes, and lots of AI
This ain’t your dad’s Google
Imagine using your facial expressions as a remote control, where each twitch or smile translates into a command. The company has now added a layer of customization to this technology, enabling support for up to 52 distinct gestures. Users can also fine-tune how expressive they need to be to trigger these actions, setting their own thresholds for each gesture to control the sensitivity of their facial commands.
Project Gameface took shape following the experiences of Lance Carr, a gamer who battles muscular dystrophy, a condition that progressively weakens muscles. Previously, Carr relied on a pricy head-tracking mouse for gaming, but a devastating house fire claimed all his gaming equipment. In collaboration with Google, Carr contributed to the development of Project Gameface, enabling him to once again engage in and stream video games.
Project Gameface won’t be just for gamers anymore
Currently, Project Gameface is offered solely as a developer tool, meaning it’s in the hands of this community to choose whether they put this accessibility feature into their mobile games for Android.
While Project Gameface is primarily an accessibility tool, its potential benefits extend far beyond that. Originally devised for the gaming community, Google is expanding its reach by collaborating with Incluzza, a social enterprise promoting accessibility across India. Together, they’re looking into ways Gameface can be utilized in diverse settings like offices, educational institutions, and social gatherings.
