Google IO wrapped up its Google I/O main keynote in the last 24 hours, which was all things AI. Mentioned 120 times throughout the keynote — well, 121 because there had to be one more AI — the entire keynote skipped the norms of the past and focused on detailing how AI is coming to every single Google product.
Generative AI is now woven into every part of your Google experience with Gemini. From accessibility and human features like AI-powered scam detection to advanced search, support for new multi-modal AI-powered glasses, and the ability to search via video, this is the most transformative Google I/O keynote in years.
Google I/O 2024: Everything announced so far
It’s all about AI this year — here’s everything Google’s talked about so far
There were many impressive announcements, but a few stood out as incredibly important for the AI-powered future we’re all aiming for. They represent the future promise of AI in a way that levels societal differences like income, visual impairments, location, and more. These are the Gemini AI features that I think will truly transform the lives of everyday users.
1 Project Astra
If there’s one feature you should be left thinking about, it’s Project Astra. We’ve put the best one first: if you’re visually impaired, Project Astra has the potential to change your life. My sister has a +12 prescription, and Project Astra feels like it’ll herald in a future of multi-modal smart glasses that could improve the lives of the vision impaired.
Sure, Google has made glasses before, and there was no mention of hardware specifically at I/O, but the glasses are real, so we’ll probably learn more soon. Either way, we’ll eventually get glasses powered by Project Astra and made by Google or through partnerships with partners like OPPO, Samsung, or Meta. Regardless of where they come from, having your AI-powered assistant see what you see and help you answer questions opens the door to so much more.
2 Ask Photos
Google Photos is getting a major AI upgrade, designed to help regular people with queries. Instead of manually searching your gallery or browsing your favorite images, you can ask Gemini — which now replaces Google Assistant on your phone — simple questions, and it will go off and find the answers.
It works by breaking down your prompt into actionable steps behind the scenes, automatically doing the multi-step prompt process you would have had to do otherwise. The demos showed searches like “What’s my car number plate?” and “How has my daughter’s swimming progressed?” but as always, the real groundbreaking use cases will become apparent once everyone starts using it.
3 Spam Detection during calls
One of the new Android features will have the biggest impact: Spam Detection during calls. With Gemini AI built into the foundation of every Android phone — it’s not just Pixel users who will get these features — it is an always-on AI assistant designed to prompt and help you when it thinks you need it.
A demo of the Spam Detection feature showed a simulated spam call. As the caller suggests moving money to a safe account, Gemini AI alerts the user with a big on-screen prompt, effectively making a scammer’s job much harder. I do not doubt that scams will still evolve and find ways to beat the AI, but there’s also no doubt that Gemini will keep learning.
Instead of scammers vs. you, it’s now scammers vs you and an AI that will consistently evolve. I know which side I would bet on.
4 Ask with Video
I love Circle to Search, and Google just took it to the next level by adding video. Instead of figuring out how to prompt Google to find your desired answer, you can point your phone’s camera at something and ask Gemini how to do something. Then it goes off and does everything for you, finding the answer to your question far faster than you can ever do it.
The demo shown was a record player where the tone arm on a record player had broken. The demo involved pointing the camera at the record player and saying this piece keeps sticking. Then Gemini surfaced the product name and model number, the name of the part that kept sticking, and examples of how to fix it.
It was a very slick demo, and I can’t wait to see how we can use Ask with Video. I used Circle to Search to find the real value while selling things during an apartment move, but even that required multiple prompts. I can’t wait to point at something and ask Gemini how much it is worth, where’s the best place to sell it, and whether it’s likely to have long-term collector value.
5 Let Google TalkBack to you
If you’re severely visually impaired, TalkBack is one of Android’s most valuable features, and it just got a major AI-powered upgrade. Although you may not be able to see the details on the screen, you shouldn’t be disadvantaged, so Google is using AI to assist you with everything you may need.
Perhaps you’re shopping for a new outfit. You may not be able to see the finer details of the photo/image of a particular item, and far too many sites do not properly use ALT tags in their images, so AI can help describe an item in more detail to you. It can likely do so much more, and if it has the full capability of Gemini AI behind it, there’s a strong chance it’ll change lives for the better, allowing the impaired to do more on their own with AI by their side.
Bonus: Google Workspace
Google announced a ton of new Google Workspace features, all powered by Gemini AI. Whether you’re a sole trader, small business owner, or even someone who values the power of Google Workspace in your daily life, Gemini AI will transform your everyday experience.
Google has implemented Gemini in a sidebar that’s effectively its answer to Microsoft’s CoPilot and is integrated into many Workspace apps. It lets you do everything from summarizing emails and answering questions to building incredibly complex automated workflows that will reduce the amount of manual data entry and assistance your business needs.
While it’s not just for businesses, the features are limited to Workspace accounts, Google’s paid version of Gmail. Some of the demos were designed with personal use cases in mind, such as summarizing all the quotes you received for a home improvement project and automatically aggregating your receipts in a single Google Drive folder. Others were solely designed for businesses, like creating an automated routine where Gemini will look for every email containing financial information and automatically add them to a spreadsheet.
For all businesses — large and small — Gemini for Google Workspace could be transformative in every business area. Many employees use valuable resources searching for answers to simple problems, and intranet/documentation platforms are worth billions of dollars for a reason. In the demo of future features for Workspace, Google showed how Gemini could learn from its communications across an entire team. Instead of asking multiple people whether something was approved yet, you can create a virtual teammate (powered by Gemini) that goes off, looks at all of its conversations, and can answer that question. It’s all in real-time so that anyone can jump in at any time, and it effectively makes surfacing information effortless.
Building AI responsibly
Google knew this would be the biggest Google I/O in years, and every announcement had a strong flair for global impact. Google didn’t focus on how AI would work for niche or edge examples; rather, it focused on showcasing the real-world use cases of AI in all of its demos.
The Google I/O keynote left me feeling one thing: hope. It is clearly a transformative time for Google, and we’re seeing what an AI-powered future could look like. Sure, Google has a history of not delivering to its full potential, so there’s no guarantee that it will this time, but these AI features have so much possibility that I truly hope Google delivers on all of its promises. Then, maybe we can all eventually let Google do all the Googling in every facet of life.
5 biggest Google I/O announcements: Circle to Search, Search changes, and lots of AI
This ain’t your dad’s Google
