Google releases SL2T: Sign language translation stops waiting for the cloud

The keyboard learns to read hands

Google DeepMind introduced SL2T (Sign Language-to-Text), a model that translates sign language directly into text. The first implementation is rolling out on Pixel 11 phones, where users can sign directly into Gboard or Live Transcribe instead of typing. (I am relying on the public announcement, as the full blog post was blocked during verification.)

This isn't a studio experiment where an algorithm spends minutes processing a recorded video. SL2T is optimized to run locally on the phone's chip in real-time. It starts with American Sign Language (ASL) to English and, according to the announcement, goes straight into the hands of end users as a native system feature, not an isolated third-party app.

Input latency drops for Deaf users

For Deaf and hard of hearing users, this significantly speeds up device communication. Previous alternatives required either slow typing or relying on high-latency cloud translators that send video data to a server. Native local translation at the system keyboard level gives signing the same status as voice dictation.

The fact that Google is integrating the model directly into Gboard highlights their bet on on-device AI. They aren't making it a premium paywalled service, but a core input method, because in gesture translation, every millisecond of latency is far more noticeable than in text.

Lighting and local dialects limit the reality

The promise of perfect transcription from a phone camera will hit the wall of physics and sociolinguistics. Gestures captured in a moving bus under poor lighting are a much harder computer vision problem than clean audio dictation.

Furthermore, sign languages have a massive amount of local dialects and nuances that cannot be easily normalized. Starting with ASL makes sense due to training data volume, but scaling to smaller sign languages or their regional variants will be exponentially more expensive and slower than adding new text languages to a standard LLM. European users will likely be waiting a long time for local sign language support.

The language rollout pace will reveal the bottleneck

The proof of success won't be how well ASL works in a demo, but how quickly Google can add other languages. If the expansion stalls after the two or three largest sign languages, it will prove that gathering training data for gestures remains too big a bottleneck, even for DeepMind.

Lilith's verdict

They aren't running a charity; they are building a hardware moat. If someone gives you a keyboard that understands your hands without a server connection, you will forgive them for making you buy their specific brand of phone to get it.