At the Google I/O 2024 event, the internet giant announced that it will integrate AI into Android in three ways: introducing AI search, making Gemini the new AI assistant, and utilizing AI directly on the device.

Google aims to "redesign" Android with the Gemini Artificial Intelligence, describing this as a "once-in-30-years event."

This means more AI search tools, like Circle to Search, will be central to Android. This AI-based tool, which can recognize naturally circled objects and text in photos and on screens, will be enhanced to tackle more complex problems, such as graphs and formulas, later this year.

Gemini AI, currently available on the Google Pixel 8a, will become the foundation of AI for Android, bringing multimodal AI—technology for processing, analyzing, and learning from information and inputs from multiple sources and sensors—to the mobile operating system.

In practice, Gemini will work across various applications and tools to provide context-aware suggestions, answers, and prompts. One example is using AI in the Android Messages app to generate AI-created images for sharing in conversations. Another is the ability to answer questions about a YouTube video a person is watching and pull data from sources like PDFs to answer highly specific queries, such as a particular rule in a sport.

Additionally, Gemini can learn from all these interactions and use this information to predict what a person might want. For instance, knowing that a user is interested in tennis and engages in conversations about the sport, it could offer suggestions for finding nearby tennis courts.

The third aspect of integrating AI into Android is ensuring that much of the intelligent processing can be done on the phone, without requiring an internet connection. Thus, Gemini Nano provides a fundamental low-latency model for embedded AI processing, with multimodal capabilities—essentially allowing AI to better understand the context of what it is being asked and what is happening.

An example of this in practice is how Gemini can detect a fraudulent call attempting to scam a person out of their banking information and alert them before the fraud occurs. And because this processing happens on the phone, there are no concerns about a remote AI monitoring private conversations.

Similarly, AI technology can use its contextual understanding to help provide accurate descriptions of what a visually impaired person is looking at, whether in real life or online.

In short, Google intends to make AI-centered Android more useful and powerful when it comes to finding things and completing tasks. And with Gemini Nano bringing multimodal capabilities to Pixel devices later this year, we can expect the Google Pixel 9 series to be the first phones to launch with the redesigned Android.