Google HEIR: The Open-Source Compiler That Lets AI Run on Encrypted Data

What HEIR Does for Private AI

HEIR lowers the barrier to using fully homomorphic encryption (FHE) for machine learning inference. A developer can take a normal trained model, feed it through HEIR's compilation passes, and produce an executable that operates on ciphertexts. The server never sees the plaintext inputs or the model's outputs, providing cryptographically-secure privacy for user data . HEIR's multi-layer IR design handles tasks like ciphertext data layout ("packing"), parameter selection, and arithmetization automatically . The project aims to enable interoperability of FHE programs across different encryption schemes, compilers, and hardware accelerators .

The Four Demonstration Applications

Google shipped four demonstration applications compiled with HEIR to show practical viability :

• Deep learning recommender — private personalized recommendations without exposing user history (developed with Belfort Labs, LG, and NYU)
• Credit-card fraud detection — detect fraudulent transactions on encrypted financial data (with Niobium and hardshell.ai)
• Kitsune network intrusion detector — identify network attacks without revealing traffic patterns (also with Niobium)
• Voice hotword spotting — trigger-word detection (e.g., "Hey Google") on encrypted audio (with Belfort Labs)

All four demos ran on single-threaded CPU implementations . Google has not yet published latency figures that demonstrate real-time performance for most workloads .

Partners and Collaborators

Hardware accelerator partners — Google is working with specialized hardware vendors to accelerate FHE inference:

• Belfort — Google is Belfort's first paying customer for encrypted compute hardware
• Niobium — their BASALISC ASIC supports BGV and CKKS via the OpenFHE API
• Cornami
• Optalysys

HEIR generates backend code for Belfort's FPGA in Rust and Google's TPU through the jaxite Python library . The first FPGA backend for HEIR was presented at the FHE.org conference in 2026, evaluating Boolean vs. arithmetic pipelines for practical FHE acceleration, with work done by researchers from KU Leuven and Google .

Academic collaborators — Co-authors on the HEIR paper include researchers from Intel (Alexander Viand), Moreh (Jaeho Choi), KU Leuven, UC Santa Barbara, University of Michigan, Ann Arbor, and Tsinghua University . The broader academic partner list also includes Georgia Tech and Carnegie Mellon .

Current Practical Limitations

FHE remains fundamentally slow. Independent benchmarks from mid-2026 state that FHE computation is still roughly 1,000x to 10,000x slower than equivalent plaintext computation, depending on the scheme and operation . Algorithm improvements, compiler optimization, batching techniques, and dedicated hardware acceleration have collectively pushed FHE performance 1,000x to 10,000x faster than implementations from five years ago, but that still leaves a large gap versus unencrypted computation for most workloads .

Google's HEIR demos ran on single-threaded CPU, and the company has not yet published latency figures that demonstrate real-time performance for most workloads . Hardware acceleration from partners like Belfort and Niobium is expected to improve speed, but those benefits are not yet demonstrated in production . A research paper evaluating three leading FHE frameworks — including Google's HEIR — uncovered 21 bugs across various layers, including 16 logic errors leading to silent incorrect calculations and 5 crashes .

Globally, only three production systems using FHE technology are publicly documented as of mid-2026 , underscoring that HEIR is primarily a research and enabling platform rather than a mature deployment tool. On Hacker News, the announcement received 275 points and commenters generally described it as real progress but nowhere near production-speed for general AI inference yet .