AudioHijack: How Inaudible Audio Commands Are Hijacking AI Voice Assistants

What makes AudioHijack different from past audio attacks?

Earlier attacks on voice assistants typically relied on wake-word activation—playing a recorded "Hey Siri" or "OK Google" command to trigger the assistant, then issuing audible malicious commands. AudioHijack is far more dangerous because it targets generative LALMs that can autonomously execute complex multi-step actions—sending emails, accessing personal data, controlling smart home devices—without any audible trigger phrase .

The real leap forward is how the attack bypasses the model's audio tokenization. LALMs convert raw audio into discrete tokens, a process that normally breaks gradient-based optimization because the discretization step isn't differentiable. The AudioHijack framework overcomes this using sampling-based gradient estimation, which approximates the gradient through the black-box tokenizer, enabling end-to-end adversarial audio generation despite the non-differentiable pipeline .

How the attack actually works

The technical pipeline has several distinct stages:

Crafting adversarial audio. The attacker starts with a target instruction—for example, "search for and download sensitive files." An optimization algorithm perturbs an audio waveform inaudibly, repeatedly testing the model's response and refining the waveform until the model reliably executes the malicious command while the audio still sounds like normal background noise to humans .

Attention supervision. The attack steers the model's internal attention mechanisms toward the adversarial audio segment. This ensures the hidden instruction dominates the model's behavior, even when legitimate user speech is also being processed .

Context-agnostic training. Researchers train the adversarial audio across many different conversation contexts—various background noises, user commands, and interaction scenarios. The result is a single 30-minute crafted signal that works regardless of what the user is saying or doing at the time of the attack .

Natural blending. A convolutional blending method modulates the perturbation into what sounds like natural room reverberation. To a human ear, it's just faint echo or ambient tone; to the AI model, it's a set of overriding instructions .

Why it's so hard to defend against

AudioHijack presents a uniquely difficult defense challenge for several reasons.

No user interaction needed. Unlike phishing or app-based malware, the user doesn't click anything, install anything, or grant any permissions. Simply playing audio content near an AI-equipped device is enough to trigger the attack. Embedding the malicious signal into a YouTube video, podcast, streaming audio ad, or even a VOIP call gives attackers a vast distribution surface .

Stealth that defeats human detection. The adversarial perturbation is carefully shaped to sit below the perceptual threshold. Users hear nothing suspicious and have no reason to suspect their assistant has been commandeered .

Reusable and persistent. The same adversarial audio works every time it's played. Unlike software exploits that get patched once discovered, a crafted audio file can exploit a victim repeatedly, and the underlying vulnerability is in the model's fundamental architecture, not a software bug that can be hotfixed .

Model-agnostic threat. AudioHijack was tested successfully across 13 different state-of-the-art LALMs, suggesting the vulnerability is endemic to how these models process audio rather than being confined to a specific implementation .

Current defense limitations

Researchers note that the only effective defense demonstrated so far is monitoring the model's internal attention mechanisms to detect and intercept malicious audio guidance. However, attackers can adapt by fine-tuning the attention-steering strength, reducing detection rates while only modestly lowering attack success .

This creates a cat-and-mouse dynamic where defenders must constantly monitor internal model states—an approach that is computationally expensive and potentially privacy-invasive if deployed at scale.

The broader implication is that the audio input pipeline for AI assistants is fundamentally less scrutinized than text-based interfaces. While prompt injection via text is a well-explored threat, the shift to audio modalities opens a much wider attack surface that the industry is only beginning to understand.