Abstract
Fully Homomorphic Encryption allows a remote system to compute without receiving a client decryption key. Applying that property to an autoregressive language model requires more than an encrypted forward pass. Hidden state, KV memory, token selection and feedback must remain encrypted and usable across every token step. We describe a runtime contract that treats each producer-to-consumer handoff as part of the protected computation. We then show why the next consumer must determine how the current encrypted step finishes. This consumer-aware transition keeps model state usable across the complete supported response. It also removes avoidable work between operators: standalone layout conversions, intermediate memory traffic, unnecessary refreshes, GPU synchronization and client round trips between tokens.
- Protected unit
- One complete supported generation
- Private state
- Prompt, hidden state, KV memory and token choice
- Client owns
- The secret key and final answer release
- Evaluator receives
- Ciphertext, public model data and the evaluation keys required by the admitted operations
- Compiler plans
- Layout, capacity, key scope, owner and GPU placement
- Latency focus
- Remove avoidable boundary work without weakening the privacy contract
First, the basic idea
FHE changes what the server receives.
With ordinary inference, transport encryption protects the request on the network, but the server still receives readable data to run the model. With FHE, the client encrypts first. The server computes on ciphertext and returns ciphertext. The secret key stays with the client.
What came before
Encrypted computing grew one step at a time
Early systems could protect one kind of calculation. Later systems protected a complete circuit and then one model prediction. Each advance kept more connected work encrypted.
One private calculation
Early schemes let a service add or multiply encrypted values without first decrypting them.[1]
- Could protect
- One supported calculation
- Still missing
- A longer program with different calculations
A sequence of calculations
Bootstrapping made it possible to refresh noisy ciphertext and run longer encrypted programs.[2]
- Could protect
- A known circuit
- Still missing
- A practical model with long-lived memory
One model prediction
Systems such as CryptoNets joined model calculations into one encrypted prediction.[7]
- Could protect
- One fixed model pass
- Still missing
- A model that repeats and changes its state
More of a transformer
Research brought more transformer calculations into the encrypted model graph.[9]
- Could protect
- More of one transformer pass
- Still missing
- Memory, token choice and the next pass
A complete supported response
Mirror treats every handoff as part of the protected execution, including model memory, token choice and feedback into the next step.
- Protects
- A complete supported generation
- What changes
- The runtime checks every step in the loop
What the research shows
The field is solving a larger part of the response
Papers first showed that individual transformer functions could run under FHE. The next step was a complete model pass. Recent work adds the memory and feedback needed to generate one token after another.
Make transformer functions work on ciphertext.
Join the functions and run them efficiently on a GPU.
Keep model memory and token feedback encrypted.
Every token and memory update stays inside the protected path.
The literature survey includes them for comparison. Mirror follows the ciphertext-only FHE path.
Inspect the paper-by-paper evidence11 systems / 2022 to 2026
THE-X
Approximates GELU, softmax and LayerNorm to enable encrypted transformer inference.
Pretrained transformers on fixed downstream tasks.
Autoregressive state and a complete decode runtime.
Polynomial Transformers
Co-designs transformer operators and architectures around polynomial HE evaluation.
Language and vision transformer tasks.
Serving recurrence and production model state.
EncryptedLLM
Accelerates FHE on GPUs and studies activation approximations for an encrypted GPT-2 pass.
A benchmarked GPT-2 forward pass.
Token recurrence, KV lifecycle and serving integration.
Encryption-Friendly LLM Architecture
Changes the transformer architecture so its nonlinear work is less expensive under encryption.
Model and training co-design for encrypted language inference.
The model itself can reduce cryptographic work before runtime optimization begins.
THOR
Optimizes encrypted matrix multiplication and nonlinear evaluation without online interaction.
BERT-base inference on one GPU.
Generative recurrence and persistent token state.
MOAI
Uses consistent packing across layers to remove format conversions and many rotations.
BERT evaluation with an extension study on Llama 3 8B.
A complete stateful generation trace.
Cachemir
Introduces KV-aware packing and bootstrapping placement for autoregressive FHE inference.
KV-cached token generation on Llama 3 8B.
Brings persistent encrypted state into the measured generation path.
SOAL
Targets the normalization, activation and softmax operations that dominate secure transformer cost.
Faster nonlinear layers for private LLM inference.
Operator design and the boundary around each operator must be optimized together.
AEGIS
Co-places transformer and encryption work across multiple GPUs for long-sequence inference.
Multi-GPU encrypted transformer execution and memory distribution.
Shows that placement, scaling and encrypted memory are one systems problem.
EncFormer
Co-designs compatible FHE stages and lower-cost FHE-MPC boundaries.
GPT-2 and BERT-style inference with interactive nonlinear protocols.
Lower latency in exchange for online two-party interaction.
Bifrost
Delegates linear work to FHE while keeping nonlinear, control and KV transitions in a CPU TEE.
A hybrid transformer and LLM serving architecture.
Plaintext exists inside an attested CPU boundary.
The literature is converging on co-design. Model functions, ciphertext layouts, memory and GPU placement can no longer be optimized independently. Some papers exchange a stronger ciphertext-only boundary for interaction or trusted execution. Mirror's path here keeps the supported generation loop in FHE and asks whether every state transition remains encrypted.
What changes for an LLM
An LLM does not run once. It runs a loop.
To produce each new token, the model reads the tokens and memory it already has, runs another model pass, chooses the next token, updates its memory and repeats.
This means one encrypted model pass is not enough. The output from each step must remain encrypted and be ready to become the input to the next step.
Enc(tokeni) + Enc(KVi)
FHE.Eval(model, CTi)
Enc(tokeni+1) + Enc(KVi+1)
The server advances the token counter on a fixed schedule.
PRIVATEAn encrypted active mask freezes a finished sequence without revealing its stop decision.
Three problems appear in the loop
The model, the ciphertext and the GPU must agree
A calculation can be correct by itself and still break the complete request. Its encrypted output must preserve the model's meaning, contain the information the next encrypted step needs and fit the way the GPU runs it.
The encrypted path must preserve a declared model contract.
For an unchanged supported graph, Mirror tracks approximation against declared tolerances and a sufficient token-stability condition. An encryption-friendly replacement is a different model and requires separate task-level evaluation.
- The problem
- Different model steps need different kinds of math
- What must hold
- Every step must remain inside its declared numerical and control contract
- Mirror's response
- Compile each model region with the step that uses its output
Problem 1: the handoff
A protected step can still fail at the next step
Each model step may work on encrypted data, but its output may arrive in the wrong form for the next step. The system then has to stop, perform an expensive conversion or open the data.
The handoff is planned as part of the model.
Mirror performs the calculation, restores ciphertext capacity when needed and places the result where the next step can use it.
Problem 2: different kinds of math
An LLM does not use one kind of calculation
Matrix multiplication, private-value multiplication, functions such as softmax and exact choices such as token selection behave differently under FHE. One ciphertext format cannot run all of them efficiently.
Most model layers multiply private data by public weights.
Attention and feed-forward layers repeatedly apply known weight matrices to encrypted model state.
- Model math
- Q/K/V, O, gate, up, down and LM head
- FHE challenge
- Arrange encrypted values for repeated matrix work
- GPU approach
- Run compatible matrix operations together
- Required output
- Encrypted state prepared for the next step
Matrix regions
Keep related matrix calculations in one efficient encrypted layout.
W x Enc(X)
Function boundaries
Evaluate functions and restore ciphertext capacity before the next region.
Refreshf(Enc(X))
Token and memory state
Choose a token, detect the end and update memory without revealing the choice.
Enc(bit) → state
Ciphertext continuity
Every step must return an encrypted result that its next consumer can use. That result carries the value together with its format, remaining capacity, key version, owner and physical location. If any one of those is missing, the generation loop stalls or creates a plaintext boundary.
encrypted value + format + capacity + owner + location → next encrypted step
The model plan, cryptographic plan and GPU plan are compiled as one connected execution.
Inspect the runtime state contract
EncryptedTensor<domain, scheme, ring, layout, scale, level, epoch, owner, placement, error>
- Key ownership
- The SDK retains the client secret key. The evaluator receives the public key and operation-enabling evaluation keys required by the plan.
- Metadata
- Tenant, request, model, shape, layout, scale and key epoch are admission inputs. Carrying them does not authenticate the computation.
- Precision
- Each approximate operator declares scale, modulus level, range and error transfer.
- Correctness
- Evaluation keys do not bind the evaluator to this graph. Malicious-execution integrity requires a separate proof or verification mechanism.[21]
Mirror builds the model around connected encrypted work
Earlier research provides the encrypted building blocks. Mirror organizes those blocks around the complete generation loop and the next use of every result.
Run compatible matrix work together.
Keep encrypted state in a useful layout across related model layers instead of converting after each one.
Enc(h) x W → Enc(hW)
Do the function and prepare the next ciphertext together.
Evaluate the function, restore capacity when needed and place the result in the next step's format.
Enc(x) → Eval(f) + refresh → Enc'(f(x))
Keep model memory and token choice inside the encrypted loop.
The next token starts from encrypted state without asking the server or client to open an intermediate value.
Enc(token) + Enc(KV) → next step
current step → compute + prepare → next step can continue
Each stage must return an encrypted result already prepared for the next stage.
One complete token cycle
How one encrypted token becomes the next
The input starts as an exact encrypted token. The transformer works on approximate encrypted vectors. Token choice becomes exact again. Mirror compiles both conversions, the transformer and its persistent KV memory into one server-side loop. The runtime receives evaluation keys, but never the client secret key. Deployment admission verifies that both scheme-conversion bridges have implemented circuits and compatible runtime parameters.
The value changes shape and arithmetic domain, but it does not become plaintext
Green marks approximate real arithmetic. Ochre marks exact token and control state. Every arrow is a typed ciphertext handoff checked before execution.
sk stays here
REMOTE FHE RUNTIME receives ciphertext, public model weights and evaluation keys
An exact token ID or selector enters the step.
Encexact(ti)
Map the exact selector into approximate slots and apply the embedding in the same transition.
Bridge(ti, E) → Encreal(hi)
K and V stay in approximate, head-aligned pages and feed later attention steps directly.
Encreal(hi, KVi) → Encreal(logitsi, KVi+1)
A supported comparison and multiplexer circuit turns approximate scores into exact encrypted state.
Encexact(ti+1, activei+1)
The server runs to a public token budget. An encrypted active bit selects the new or frozen state without changing server control flow.[22]
statei+1 = MUX(active, new, frozen)
At the fixed budget or a declared client checkpoint, the client receives ciphertext and releases the answer locally.
N public steps → client decrypts
The secret key never enters the runtime.
Dense layers and KV remain in the real-valued domain.
No reload and repack cycle before every attention step.
If 2εlogits < top1 - top2, approximation cannot change the top token.
Five controls keep the loop usable
Mirror compiles a bounded plan for each handoff before runtime, then checks the same plan as every token advances.
Compare producer output with consumer input across domain, scheme, ring, layout, scale, level, key epoch, owner, placement and error.
compatible(Sout, Sin)
Use the ciphertext directly or insert one declared transition. Admission requires every cast to preserve the encrypted path.
Propagate encoding, polynomial and rounding error through each consumer. Give every output a maximum accepted error.
εnext ≤ Lfεin + εop + εround
Reject a plan when its static bound exceeds the declared tolerance. Token parity is claimed only when a certified error and margin bound establishes the sufficient condition.
Track approximate scale and modulus levels, plus exact-plane noise capacity, across the full dependent chain. Place a scheme-specific bootstrap at a shared cut only when downstream depth requires it.
capacityin - cost(chain) ≥ capacityminimum
Co-schedule refresh with layout and placement work. Bootstrap restores ciphertext capacity; the separate numerical error budget still carries forward.
Derive ciphertext size from ring, modulus limbs and component count, then run liveness analysis over buffers, active KV pages, weight tiles, evaluation keys and workspaces.
peak HBM = live CT + active KV + weights + eval keys + workspace
Reuse dead buffers, keep the active KV window resident, page only encrypted state and load only the required rotation, multiplication and bootstrap keys. Reject a plan that exceeds its bound.
Compile adjacent producer and consumer stages together. Fold public transforms, bridge plus embedding, refresh, repack and placement where their schemas allow it.
Ttoken = TFHE work + Tboundary
Reduce launches, synchronizations, HBM writes, repeated packing and client callbacks while preserving the required cryptographic work.
Every ciphertext carries a contract for its next consumer
Shape alone is not enough. The schema records the FHE domain and scheme, packing, scale, remaining capacity, key epoch, owner, placement and accepted numerical error.
Encrypted embedding selection → QKV projection
The selected token is an exact bit vector or one-hot value. The embedding and hidden state are approximate real-valued tensors. Mirror fuses the admitted Boolean-to-real bridge with the public embedding calculation, then emits projection-native hidden-state tiles.
- Arithmetic
- Boolean or exact → approximate real
- Value
- Hidden vector
- Layout
- Tiles consumed by QKV
- Lifetime
- One immediate consumer
- Placement
- Active GPU working set
- Latency effect
- No standalone materialize and repack pass
K/V projection → encrypted KV cache → attention
K and V are private derived state. They persist across future tokens, so Mirror writes them into owner-scoped, head-aligned encrypted pages at the scale and level expected by attention. The active window can then be consumed without rebuilding the cache layout.
- Arithmetic
- Approximate real throughout
- Value
- Persistent K and V state
- Layout
- Layer, head, page, position, dimension
- Lifetime
- Reused by future token steps
- Placement
- Paged encrypted cache
- Latency effect
- Avoid repeated append, reload and repack work
Encexact(b), b ∈ {0,1}
Bridgeexact→real(b, E)
Encreal(h, KV)
Boolean correctness and floating-point error are different contracts
The exact plane tracks whether ciphertext noise remains inside the Boolean correctness margin. The approximate plane tracks numerical error against the next consumer's tolerance. Refresh restores computational capacity while numerical model error remains tracked as a separate contract.
marginbool > marginminimum
|Convertexact→real(b) - b| ≤ δbridge
εemb ≤ ||E|| δbridge + εencoding
εKV,next ≤ εKV,current + εtransition
εout < toleranceconsumer
2εlogits < logittop1 - logittop2
Compiler contract curves: symbolic bounds evaluated for each admitted plan.
Prepare once for the work that comes next
FHE arithmetic is expensive, but disconnected implementations add another cost between calculations. They write intermediate ciphertexts, synchronize the GPU, reload state, change layouts, move evaluation keys and refresh more often than the full path requires. An autoregressive model repeats those boundaries across layers and again for every token.
The next consumer should determine how the current step finishes
Mirror compiles the producer and consumer together. Public transforms are folded into adjacent work, refresh is placed from the remaining depth of the whole chain, and the result is written directly in the format and location its next consumer expects.
Every boundary starts another preparation sequence
The boundary becomes part of the latency of every layer and token.
One physical transition prepares the next encrypted step
The next step starts without a standalone conversion or plaintext handoff.
Fewer GPU launches and synchronizations
Related encrypted work becomes one scheduled region instead of a chain of isolated kernels.
Less intermediate memory traffic
The result can enter the consumer's buffer directly instead of being written, loaded and rearranged.
Refresh only when the chain needs it
Remaining ciphertext capacity is planned across the full path, avoiding refresh after every nonlinear source operation.
No client round trip between tokens
Encrypted selection, feedback and KV state remain on the server, so generation does not stop for client-side decryption.
Ttoken = Σ useful FHE work + Σ boundary overhead
Mirror preserves the required FHE work while reducing the second term: operations created only because adjacent encrypted stages were implemented separately. That boundary overhead recurs across layers and generated tokens.
How the Mirror layers work together
No single FHE call creates a protected response. The SDK, compiler, model runtime and GPU runner each have one job, and every layer must pass usable ciphertext to the next.
client-owned request
approved model path
model regions + handoffs
next-step-ready ciphertext
declared boundary record
From one path to larger decoders
Scaling changes the schedule, not the privacy contract
A larger model adds matrix work, a longer KV cache, more refresh points and greater pressure on GPU memory. Current research reduces those costs at different layers. Every optimization must still return ciphertext that the next step can consume directly.
Use encryption-friendly functions
Polynomial activations and redesigned normalization reduce the expensive nonlinear graph.
Plan around the KV cache
KV-aware packing and refresh placement keep repeated token state inside the encrypted path.
Place crypto with the model
Multi-GPU execution must distribute ciphertext, keys and transformer work as one topology.
Keep selection and feedback data-oblivious
Exact token choice, active-mask updates and the next embedding handoff stay encrypted while the server follows a public schedule.
Problem 3: encrypted memory is large
Encrypted model state is much larger than ordinary model state
Hidden state and the KV cache expand into many ciphertext components, metadata and evaluation keys. The runtime must decide what stays in GPU memory, what moves out and what the next layer will need before execution begins. Deployment admission records the model, batch, context, cryptographic parameters and computed peak bytes before execution.
Group encrypted state by how the next layer will use it.
Related KV entries share request-scoped encrypted pages. The active attention window stays in GPU memory while older encrypted pages and keys needed by the active layer move through planned storage tiers.
- GPU memory
- Required output: computed peak bytes for this model, batch and context
- Key scope
- Keys limited to the layer and function
- Refresh group
- A compatible group of ciphertexts
- Output order
- Already ordered for the next layer
How Mirror checks the plan
The compiler admits only paths that stay encrypted
Every encrypted value carries more than model data. It also records its FHE scheme, format, remaining capacity, key version, owner and physical location. Mirror approves a path only when every handoff declares how that information reaches the next step.
EncryptedTensor<scheme, ring, layout, scale, level, epoch, owner>
Where Mirror fits
A normal model request gains an encrypted execution path
The application still makes a model request. The Mirror SDK encrypts it, the runtime checks and schedules the encrypted graph, and the GPU runner computes without the client key. The SDK decrypts only the final result.
The precise security claim
What FHE hides, and what remains visible
FHE protects the content of the prompt, intermediate model state and supported output. It does not automatically hide request size, timing, the model being used or whether the server performed the correct calculation. Those need separate controls.
A network observer sees encrypted messages, not the prompt.
The prompt and response stay encrypted. Request size, timing and destination remain visible unless a separate traffic-hiding control protects them.
Mirror's contribution
Mirror turns FHE from a protected calculation into a complete encrypted model path
The cryptographic primitive makes computation on ciphertext possible. Mirror connects that primitive to the way an LLM actually runs: repeated model passes, persistent KV state, exact token decisions, GPU resource limits and final client release.
The application retains the secret key and sends ciphertext with owner-scoped request metadata.
Schema, error, capacity, ownership and peak memory are checked before admission.
Hidden state, KV memory, token selection and feedback remain encrypted across supported steps.
The runtime returns ciphertext. It never receives the secret key needed to release the answer.
For an admitted model path, a provider can serve useful inference without taking possession of the customer's readable prompt, derived model state or generated content. The application keeps its model-call workflow. Mirror owns the encrypted execution boundary between request and release.
Encrypted inference is not complete when one layer can process ciphertext. It is complete when the response can.
Explore Encrypted Token FactorySources and further reading
- Rivest, Adleman and Dertouzos, On Data Banks and Privacy Homomorphisms, 1978.
- Gentry, A Fully Homomorphic Encryption Scheme, 2009.
- Brakerski, Gentry and Vaikuntanathan, Fully Homomorphic Encryption without Bootstrapping, 2011.
- Fan and Vercauteren, Somewhat Practical Fully Homomorphic Encryption, 2012.
- Cheon, Kim, Kim and Song, Homomorphic Encryption for Arithmetic of Approximate Numbers, 2017.
- Chillotti et al., Faster Fully Homomorphic Encryption, 2016.
- Gilad-Bachrach et al., CryptoNets, 2016.
- Vaswani et al., Attention Is All You Need, 2017.
- Chen et al., THE-X, ACL Findings 2022.
- Zimerman et al., Converting Transformers to Polynomial Form for Secure Inference Over Homomorphic Encryption, ICML 2024.
- De Castro et al., EncryptedLLM, ICML 2025.
- Zhang et al., Encryption-Friendly LLM Architecture, ICLR 2025.
- Moon et al., THOR, CCS 2025.
- Park et al., Powerformer, ACL 2025.
- Zhang et al., MOAI, ICLR 2026.
- Yu et al., Cachemir, preprint 2026.
- Park et al., SOAL, ICLR 2026.
- Zhang et al., AEGIS, preprint 2026.
- Zhu et al., EncFormer, preprint 2026.
- Chen et al., Bifrost, preprint 2026.
- Viand et al., Verifiable Fully Homomorphic Encryption, 2023.
- Chialva and Dooms, Conditionals in Homomorphic Encryption and Machine Learning Applications, 2018.