Google Open-Sources HEIR: AI on Encrypted Data Without Decryption

30-Second Summary

Google released HEIR (Homomorphic Encryption Intermediate Representation), an open-source compiler that converts any pretrained AI model to run inference on encrypted data. The server performing the AI computation never sees the underlying data at any point. This makes private AI inference practical for the first time without requiring expert cryptographers, and it removes a genuine technical barrier for enterprises in healthcare, legal, finance, and government.

At a Glance

• Topic: AI Security and Privacy Infrastructure
• Company: Google
• Date: August 15, 2026
• Announcement: Open-source release of HEIR, a compiler toolchain enabling AI inference on homomorphic-encrypted data
• What Changed: Private AI inference, previously requiring specialist cryptographers, is now accessible via a Python-first toolchain any engineering team can adopt
• Why It Matters: Enterprises can run AI on their most sensitive data without it ever leaving an encrypted state, eliminating the vendor data exposure problem
• Who Should Care: Operators in healthcare, legal, finance, and government; anyone building AI products for regulated industries; security and compliance leads

Key Facts

• HEIR stands for Homomorphic Encryption Intermediate Representation
• The toolchain is built on MLIR (Multi-Level Intermediate Representation), the same compiler framework used in production ML systems
• It ships with a Python frontend: developers annotate which variables are secret, and HEIR compiles the model to operate on encrypted inputs
• Use cases confirmed by Google include recommendations and fraud detection
• The project is available now on GitHub under an open-source licence
• Google describes the goal as a "one-click solution" enabling non-experts to incorporate encrypted inference into production applications
• Fully homomorphic encryption (FHE) allows computation on ciphertext that produces an encrypted result, which when decrypted matches what would have been obtained by performing the same operations on plaintext

What Happened

Google published HEIR as an open-source compiler toolchain that bridges the gap between pretrained AI models and fully homomorphic encryption (FHE). Homomorphic encryption allows computation to happen directly on encrypted data, producing an encrypted result. When the user decrypts the result, it matches exactly what would have been produced by running the same computation on unencrypted data. The server running the computation sees nothing but ciphertext throughout.

Until now, building systems that used FHE required deep cryptography expertise. HEIR changes this by providing a compiler that takes an ordinary Python model, accepts type annotations marking which inputs are secret, and generates optimised FHE circuits automatically. The engineering team does not need to understand ciphertext algebra, parameter selection, or packing layouts. HEIR handles those layers internally.

The toolchain targets the gap between AI practitioners who can build models and cryptographers who understand how to run them securely. By automating the translation between a PyTorch or similar model and an FHE-compatible representation, HEIR makes private inference a deployment option rather than a specialised research project.

The GitHub repository is live. The project is described as active and intended for production adoption, not a research prototype.

Why It Matters

It removes the last genuine technical objection to AI in regulated data environments. Many enterprises have refused to put sensitive data through cloud AI services because the inference server necessarily processes plaintext data. Contractual data processing agreements help legally but do not change the technical reality. HEIR changes the technical reality.

Healthcare is the most immediate beneficiary. Medical records, diagnostic images, and patient histories sit behind strict data sovereignty rules in most jurisdictions. Running AI on that data has meant either accepting the privacy risk or building on-premises infrastructure. Private inference via HEIR introduces a third path: cloud inference on encrypted data.

Financial services fraud detection becomes architecturally simpler. Banks currently run fraud models on transaction data that flows in plaintext to the inference server. With HEIR, the transaction data stays encrypted throughout. The output, a fraud probability score, is decrypted by the bank. The inference provider sees neither the transaction nor the score.

Legal and professional services can engage AI on privileged material. Law firms have been particularly cautious about AI tools because of professional privilege obligations. A system that cryptographically guarantees the model host never sees the data addresses that concern in a way that a vendor's privacy policy cannot.

Open-source distribution accelerates adoption and trust. Because HEIR is open source, enterprises can audit the toolchain, run it on their own infrastructure, and verify the implementation rather than relying on a vendor's assurances. For regulated industries, auditability is often as important as the privacy guarantee itself.

The timing coincides with EU AI Act enforcement. As GPAI transparency and high-risk AI obligations begin landing across Europe in August 2026, tools that allow organisations to demonstrate they are not sharing sensitive data with AI providers become a compliance asset, not just a technical feature.

The David and Goliath View

Private inference is not a new idea. Homomorphic encryption has existed for decades and has been the subject of academic interest for just as long. What has always prevented production adoption is the computational overhead: FHE is orders of magnitude slower than plaintext computation, and building FHE-compatible inference pipelines required specialist expertise almost no engineering team possesses.

HEIR addresses the expertise problem directly. The performance problem is still real and will limit use cases initially to lower-latency-tolerant applications: fraud scoring on a transaction submitted for authorisation, a recommendation made at search time, a document classification that does not need to complete in milliseconds. But performance constraints shrink as hardware improves, and the architectural pattern HEIR enables, which is inference on encrypted inputs, becomes more broadly applicable over time.

For operators building AI programmes in professional services, healthcare, or financial services, the practical move is to track this toolchain and begin evaluating it against the specific use cases your clients have declined to pursue because of data sensitivity. Those blocked use cases represent real revenue. HEIR may be the technical unlock.

Where This Fits in the AI Stack

HEIR sits at the inference layer. It is not a training tool and does not change how models are built. It changes how they are served. A model trained on plaintext data can be compiled via HEIR and then served as a private inference endpoint. The model weights themselves may still be stored in plaintext on the server; what changes is that the input data the model processes stays encrypted throughout the computation.

This puts HEIR alongside other privacy-preserving inference approaches including federated learning (which keeps data distributed rather than centralised) and confidential computing (which uses hardware enclaves to protect data during computation). HEIR's advantage is that it requires no trusted hardware and no cooperation from the hosting infrastructure: the encryption guarantee is mathematical, not hardware-dependent.

Questions Operators Are Asking

Does the AI model itself remain private? HEIR focuses on input data privacy. The model weights are generally held by the inference provider in plaintext. If you are using a third-party model, that provider still knows what model you are running. If you want model privacy as well, that requires additional approaches. HEIR solves the data privacy problem, not the model IP problem.

What is the performance penalty? FHE adds significant computational overhead compared to plaintext inference. The exact penalty depends on the model architecture and the operations involved. Google's blog describes the goal as making private inference practical for real applications, suggesting current performance is acceptable for use cases that are not latency-critical. Expect the toolchain to be most useful initially for asynchronous or batch inference rather than interactive, sub-second applications.

Can we run this on our existing models? HEIR is designed to convert pretrained models. You annotate which inputs are secret, and the compiler handles the conversion. There are constraints on the operations supported within FHE circuits, meaning some model architectures will convert more cleanly than others. The Python frontend is the starting point for evaluation.

Is this production-ready? Google describes it as an active project intended for production adoption. As with any early-stage open-source toolchain, enterprise teams should evaluate it carefully against their specific requirements before treating it as production infrastructure. Given Google's track record with MLIR-based projects, the foundation is solid.

How does this relate to confidential computing and secure enclaves? Confidential computing (such as AWS Nitro Enclaves or Intel TDX) protects data during processing using trusted hardware. HEIR's FHE approach protects data through mathematics, with no trusted hardware requirement. Both provide strong guarantees; they are complementary rather than competing. HEIR's mathematical guarantee is stronger in threat models where you do not trust the hardware vendor. Confidential computing is typically faster today.

Citable Summary

Google released HEIR, an open-source compiler toolchain that converts pretrained AI models to run inference on fully encrypted data using homomorphic encryption. The server never decrypts the input. The toolchain ships with a Python frontend, making private inference accessible to engineering teams without cryptography expertise. The release addresses a long-standing barrier to AI adoption in regulated industries including healthcare, legal, and financial services, where data cannot be shared with third-party AI providers in plaintext. The project is available on GitHub.