ByteDance's SeedRealtime: The First AI That Watches, Listens, and Speaks at the Same Time

How SeedRealtime differs from traditional voice assistants

Traditional voice assistants operate as a cascaded pipeline: audio goes through ASR (speech-to-text), the text passes through a vision model if needed, then through an LLM, and finally through TTS (text-to-speech). Each handoff adds latency, and context is lost at every boundary .

SeedRealtime replaces this pipeline with four key advantages:

• Unified end-to-end architecture — native fusion of audio, video, and text eliminates multi-module delay and information loss
• Audio-visual joint understanding — the model can resolve homophone ambiguity by using visual scene context (e.g., distinguishing "write" from "right" based on what it sees) and accurately understand temporal references in video
• Proactive interaction — continuously senses its environment and can proactively alert the user when the scene changes (e.g., a target object appears) and call external tools
• Natural conversational timing — senses the user's conversational state and pacing in real time, chiming in, pausing, and responding at the right moments, with high robustness to background noise and bystander chatter

The technical challenge it solved: conversational rhythm

The hardest problem SeedRealtime solved is "when to speak, when to listen" in a continuous multimodal stream. Traditional turn-based (half-duplex) assistants wait for a period of silence before responding — which means they either interrupt or sit silent .

SeedRealtime's full-duplex architecture allows it to:

• Sense the user's conversational state and pacing in real time
• Distinguish intentional pauses from hesitations
• Ignore bystander chatter and background noise to avoid false triggering
• Produce backchannel cues and interruptions at human-appropriate moments

End-to-end human evaluation results show that compared to cascaded models, SeedRealtime's audio-video dialogue rhythm problems were reduced by half. Issues like "being interrupted before finishing, delayed responses after speaking, or false triggering by background noise" decreased significantly, while the probability of a complete, smooth single dialogue increased notably .

How SeedRealtime builds on Seeduplex

SeedRealtime is the multimodal evolution of ByteDance's earlier full-duplex work. Seeduplex was launched on April 9, 2026 as ByteDance's first native full-duplex speech-only LLM — it could listen and speak simultaneously, moving beyond the previous half-duplex paradigm .

SeedRealtime takes Seeduplex's core breakthrough — native end-to-end full-duplex processing — and extends it to incorporate real-time visual understanding, enabling the model to react to what it sees in addition to what it hears .

Where SeedRealtime is deployed

SeedRealtime has been fully deployed in the Doubao (豆包) app — ByteDance's AI assistant — and is available to all users. Users update Doubao to the latest version and enter the video call interface through the "phone call" feature to experience real-time audio-video AI interaction .

ByteDance has demonstrated multiple use cases: identifying different people in a group conversation and tracking who is speaking; helping a foreign tourist understand a Chinese menu and the waiter's explanations in real time; continuously observing a museum exhibit and proactively alerting when a specific artifact appears; guiding a user through operating a coffee machine and correcting errors based on visual changes; monitoring page turns while reading a paper and automatically prompting when a target chapter appears .

ByteDance's broader AI portfolio in 2026

SeedRealtime is one piece of ByteDance's accelerating AI release cadence. The Seed team has shipped multiple models through mid-2025 to mid-2026:

These models, together with Seeduplex (April 9, 2026), form ByteDance's full-stack AI ecosystem spanning language, image, video, audio generation, and real-time multimodal interaction .