Google DeepMind SL2T: How the Pixel 11 Turns Sign Language into Text

How SL2T works

Training data: 100,000+ hours, 50+ sign languages

SL2T was trained on more than 100,000 hours of sign language video covering over 50 sign languages, with roughly one quarter of that data in ASL . By training jointly on multiple sign languages, dialects, and proficiency levels, Google's research found the model outperforms single-language systems — learning shared structural patterns that improve generalization . Notably, the training data intentionally excludes signers under 18, which may reduce accuracy for younger users .

On-device landmark tracking: Only coordinates leave your phone

The system is designed for privacy. The phone's camera runs MediaPipe Holistic, extracting 2D landmark coordinates for 130 key points (face, body, hands) frame-by-frame. The raw camera video is then discarded immediately and never leaves the device . Only these abstract geometric coordinates are sent to Google's servers for translation, and no logs of user inputs or outputs are retained without explicit user authorization . The 2D landmark representation does not track tongue landmarks or provide depth ordering, making it harder to distinguish between hand contacts and hovering — both of which are important for ASL .

Direct pose-to-text: Gloss-free, autoregressive translation

SL2T uses a transformer that conditions directly on the landmark coordinate sequence and generates English text autoregressively. Unlike earlier approaches, it does not first output intermediate glosses or hand-coded linguistic representations . The model translates rather than transcribes — meaning it produces natural English text, not a word-for-word mapping of ASL signs.

Benchmark performance: State-of-the-art but vendor-reported

On FLEURS-ASL, a zero-shot evaluation dataset of complex, abstract language recorded in studio conditions by Certified Deaf Interpreters, SL2T scores 70 BLEURT, which Google says is far above any previously reported result . Additional benchmark results include:

• 74 BLEURT on a one-handed variant of FLEURS-ASL
• 85 BLEURT on PRESTO-ASL, an AI-assistant-phrase dataset
• 64% exact-match accuracy on FSboard, a fingerspelling dataset collected from Deaf signers on mobile devices

It is important to note that these are vendor-reported figures; no independent evaluation has been published as of the launch date.

Governance and limitations: Built with guardrails

The AI Sign Language Advisory Committee

Google convened an external AI Sign Language Advisory Committee (AISLAC), which includes representatives from the National Association of the Deaf (NAD), RIT/NTID, DPAN, and the World Federation of the Deaf (WFD). AISLAC co-authored the Joint Impact Report that defines the model's operational boundaries and limitations .

Prohibited use contexts

SL2T is explicitly designed and evaluated only for low-stakes, informal situations such as text dictation, messaging, search queries, and one-on-one conversations (e.g., at a coffee shop or retailer). High-stakes or multi-party conversations in medical, legal, and academic settings are out of scope .

ADA disclaimer

Google states in its documentation that SL2T is not a substitute for professional interpretation services and should not be relied upon in contexts where communication errors could cause harm .

Known technical constraints

• Low-light degradation: Performance drops under poor lighting because MediaPipe Holistic landmarks become less reliable .
• Single-signer support: The model handles only one signer at a time; it cannot interpret multi-party signed conversations .
• 60-second clip limit: Individual translation segments are capped at roughly 60 seconds of signing .
• No under-18 training data: The model may be less accurate for younger users, as training data excludes signers under 18 .
• 2D representation limits: The landmark system does not track tongue articulation or provide depth ordering, making it hard to distinguish contact from hovering — both linguistically significant in ASL .

The role of Sam Sepah

Sam Sepah, a Deaf Googler and member of the Sign Language team, played a central role in SL2T's development. He is a co-author on the FSboard dataset publication used in the model's evaluation . Google's blog and the Joint Impact Report emphasize that the team worked closely with Deaf Googlers — including Sepah — throughout pre-release testing to uncover model behaviors and interface issues that benchmarks alone could not capture .