The attack exploited a built-in feature of Gemini's Android voice assistant, specifically a tool within the Android Utilities agent that reads and processes incoming device notifications. Because this tool handles untrusted data from third-party apps, a crafted message could embed malicious instructions directly into the notification text. When Gemini read the poisoned notification, it would silently inject those commands into its own context, ready to act on them during a subsequent, completely innocent interaction with the user .
This meant that an attacker did not need physical access to the phone or any special permissions. A single message delivered through a standard messaging platform—WhatsApp, Slack, Signal, SMS, Instagram, or Messenger—could be enough to compromise the device .
Google had already learned from earlier research. When SafeBreach previously demonstrated how a malicious Google Calendar invite could hijack Gemini, Google responded by patching the system to block chained tool invocations and delayed tool invocation, two common prompt injection strategies. The patch prevented attackers from triggering a sequence of sensitive actions or holding off on an attack until the user was looking away .
SafeBreach's researcher Or Yair found a creative way around these new guardrails. The novel 'Fake Context Alignment' technique created a dual reality to fool the AI's security logic . It worked by presenting two different faces:
The trick relied on hidden or obfuscated commands. Attackers would embed malicious instructions within foreign-language text, muted hyperlinks, or other concealed prompt formats that a human might ignore but an AI would process. When the user later issued a normal, harmless voice command or typed a reply, Gemini's own authorization logic would mistakenly interpret that user action as approval for the sensitive, hidden tasks that had been planted earlier. By combining multiple obfuscation and timing techniques into what researchers called the "Ultimate Combo" payload, the team could bypass all of Google's latest mitigations with high reliability .
SafeBreach didn't just describe the theoretical risk. They demonstrated five concrete attack scenarios that showed how complete the hijacking could be .
1. Smart Home Control
Once Gemini was compromised, an attacker could remotely manipulate any connected Google Home device. This included opening connected windows, controlling boilers, and managing lighting systems, turning the AI assistant into a digital intruder with physical-world consequences .
2. Forced Zoom Calls with Covert Camera Streaming
Researchers demonstrated the ability to silently launch the Zoom app on the victim's device and initiate a call that would stream the phone's live camera feed. They accomplished this by using a 301 HTTP redirect from a domain that was approved by Google's Safe Browsing service, making the malicious connection appear legitimate to security checks. The user would have no visual indication that their camera was live .
3. Memory Poisoning Across the Google Ecosystem
Perhaps the most insidious attack was the ability to inject false information into Gemini's long-term memory. Because this memory syncs across a user's entire Google Workspace account, a single poisoned notification could corrupt the "remembered" information available to the assistant on the victim's tablet, computer, and smart speakers—potentially leading to future, misinformed actions by the AI across all devices .
4. Fake Trusted-Contact Messages
The attack could be weaponized for large-scale social engineering. Researchers were able to extract real sender names from the device's notification queue and fabricate messages that appeared to come from a trusted contact, such as a boss or family member. This required no prior knowledge of the victim's contacts and could fuel highly convincing phishing campaigns .
5. Scheduled Surveillance
To enable ongoing data exfiltration, researchers established a recurring task within the AI's context. This instructed Gemini to automatically read the user's recent messages every day, creating a persistent, self-sustaining surveillance channel without any further attacker interaction .
The research followed a responsible disclosure timeline through Google's Vulnerability Reward Program (VRP):
While this specific window was closed, the research highlights a fundamental tension in AI assistants: the more useful and context-aware they become by reading our notifications, calendars, and emails, the more untrusted data pipelines they must securely manage. SafeBreach's work serves as a critical blueprint for hardening the next generation of AI agents against a threat that requires nothing more than an invitation to listen.
Comments
0 comments