Inside Google's New Local AI Ecosystem for Mac: Gallery, Gemma 4 12B, and Eloquent

The application is now available across macOS, iOS, and Android, but the macOS version brings a specific model curation strategy into focus . Unlike the open libraries of Ollama and LM Studio, which let users pull almost any compatible model, the macOS AI Edge Gallery currently exposes five Google-curated Gemma models . As reported by 9to5Mac, the available models include Gemma-4-12B-it, Gemma-4-E2B-it, Gemma-4-E4B-it, a Gemma-4 26B variant, and FunctionGemma-270M . This hand-picked selection is the heart of Google's strategy: a controlled, quality-assured environment .

The Engine: LiteRT-LM Performance

Under the hood, the ecosystem is powered by Google's LiteRT-LM inference engine. It supports CPU, GPU, and NPU backends across Linux, macOS, and Windows . The featured model for performance benchmarking remains Gemma-4-E2B (2.58 GB), and the official documentation provides a clear look at its capabilities on a MacBook Pro M4 :

• CPU: 901 tokens/sec, 42 MB memory, 1.1s latency to first token, 736 MB peak memory .
• GPU (Metal): 7,835 tokens/sec, 160 MB memory, 0.1s latency to first token, 1,623 MB peak memory .

The massive leap in speed with GPU acceleration highlights how tuned Google's stack is for Apple Silicon's Metal API, delivering a near-instantaneous, fluid user experience.

Gemma 4 12B: The Encoder-Free Multimodal Powerhouse

Released under the Apache 2.0 license, Gemma 4 12B is the star of this launch . Its architecture is its biggest differentiator. It is a dense, decoder-only transformer using the same advanced decoder structure as the much larger Gemma 4 31B Dense model .

The critical innovation is its encoder-free multimodal design. Most multimodal models use separate, bulky encoders for vision (like a ViT) and audio (like conformer layers) to translate data for the language model . Gemma 4 12B eliminates them entirely . Instead, it employs:

• Vision: A lightweight 35M-parameter embedder replaces the 27-layer vision transformer. Raw 48x48 pixel patches are projected directly into the LLM's embedding space with a single matrix multiplication, with spatial information attached via a factorized coordinate lookup .
• Audio: Raw audio signals are projected directly into the same embedding space as text tokens, without a separate audio encoder .

This allows the model to natively process text, images, audio, and video in a single unified flow . Google claims this architecture delivers "performance nearing our 26B MoE model with less than half the memory," all while running on consumer laptops with just 16 GB of unified memory .

Benchmarks bear out this confidence, showing the 12B model punching far above its weight class. On GPQA Diamond (graduate-level reasoning), it scores an impressive 78.8, placing it near the 26B variant. On academic-style multiple-choice benchmarks like MMLU Pro, it achieves 77.2%, and on the competitive math benchmark AIME 2026 it scores 77.5% . On LiveCodeBench for code generation, it reaches a score of 72.5%, demonstrating robust practical capabilities in agentic workflows and multi-step reasoning .

AI Edge Eloquent: Free, Offline, Polished Dictation

Rounding out the product trio is Google AI Edge Eloquent, a dictation app that positions itself as a direct, free alternative to paid transcription services . The app is powered by Gemma-based models and is designed to be completely offline-first .

It transcends simple transcription by acting as an automatic speech polisher. It "aggressively trims out" filler words like "um" and "uh," fixes grammar on the fly, and restructures raw, chaotic speech into coherent, professional text . This makes it more of a communication tool than a note-taking app. The key differentiator is the price tag: there is no subscription and no usage cap . The macOS version requires macOS 13.0 or later and an Apple M1 chip or later, though the App Store page notes some advanced, optional features may require cloud processing .

Strategy: The Walled Garden vs. the Open Library

This launch establishes two opposing philosophies for local AI. Google's strategy is a "walled garden" approach: a curated, Google-approved set of models, tightly integrated with branded, first-party apps (Gallery for exploration, Eloquent for dictation), and a unified inference engine (LiteRT-LM) with a CLI and Python API . The goal is to provide a seamless, consumer-grade experience that "just works" right out of the box.

This is in direct contrast to Ollama and LM Studio, which prioritize maximum flexibility and choice as open libraries where users can pull any compatible model . Notably, both Ollama and LM Studio already support the open-weight Gemma 4 12B model, so Google's model is not exclusive to its own stack .

Google's advantage lies in first-party optimization, where its own models are tuned specifically for its inference engine on Apple Silicon for better performance and lower memory usage. The trade-off for the user is clear: you get a more polished and integrated experience, but you cannot run models outside of Google's curated Gemma family. This positions Google to capture users who value reliability and ease of use over experimental freedom, creating a distinct fork in the road for local AI on the Mac.