Rollouts Over Ciphertext

Epoch AI puts the quality- and repetition-adjusted stock of public human-written text at roughly 300 trillion tokens, and projects that the industry could work through it between 2026 and 2032. The date matters less than an older fact about coverage. Records of skilled work rarely reached the open web, which is why models trained on it stay uneven at the capabilities companies now pay for, such as dependable scientific analysis, useful changes inside an unfamiliar repository, or sound judgment across a long business process.

A repository holds more than its committed code. Hidden tests encode a decade of outages. One engineer can explain why a certain service never ships on a Friday, though the explanation takes ten minutes and half a whiteboard. Four designs were tried and abandoned before the fifth worked, and that reasoning survives only in a thread nobody will publish. Laboratories accumulate a similar residue, from raw instrument output and bench-specific procedure to the afternoon a group reran an entire experiment because one number looked wrong. A legal or financial workflow carries its own version, in the source material, the internal rubric, the risk tolerance, and the expert edits that shaped the final document.

These are the experiences a model could learn the most from. They are also the experiences no organization can upload to a generic training bucket, and that tension is where the next data bottleneck, and the next data market, will take shape.

In RL, the environment is the real data asset

Reinforcement learning treats this material as an environment rather than a dataset. Prime Intellect’s Verifiers defines one as:

• task inputs;
• a harness containing tools, sandboxes, and interaction logic;
• a reward function or rubric that decides whether the model succeeded.

The package records what a model was permitted to do along with what the organization counts as done well, which is far more information than a prompt paired with its answer. It captures the world the work happened in, the actions available inside that world, and the standard the work was held to.

Prime’s push to roughly 365,000 agentic tasks shows how much care the reward side takes. Grading material has to stay hidden from the agent. Gold paths get tested before an environment ships, as do no-op paths, since a reward that pays out for doing nothing will teach a model to do nothing. A gameable reward lets the model exploit it, while a brittle one rejects correct work that arrived by an unexpected route, and the second failure is harder to spot because the training curves still look healthy.

This is the craft that ordinary data procurement misses. The valuable product turns out to be a reliable learning system built from tasks, tools, counterfactuals, and a reward signal that resists gaming. For many organizations, a finished environment of this kind will rank among their most valuable IP.

Using an environment usually means exposing it

The standard training architecture assumes the machine running inference can see the rollout, an assumption that holds while a lab owns every box in the rack. Inference tends to migrate, though, onto rented GPUs, a partner’s cluster, a third-party platform, or a distributed compute network, and after that move the prompts, tool results, and intermediate actions cross systems the environment owner has no authority over.

Encryption at rest and TLS cover storage and transit, and the inference server still has to open the request before it can run the model. Whatever leaves the perimeter arrives somewhere else in plaintext.

So a large pool of potential training experience stays where it is:

• laboratories keep unpublished experimental trajectories to themselves;
• companies hold internal repositories and hidden tests inside the corporate perimeter;
• professional-services firms would lose a client over an externalized workflow;
• even AI labs think twice before moving their own proprietary environments across compute providers they only partly control.

The willingness to train on this material already exists on both sides of the transaction. The architecture is what blocks it.

The encrypted RL lane

Fully homomorphic encryption removes an assumption the rest of the stack takes for granted, namely that the machine doing the arithmetic must be able to read the numbers. NIST describes FHE as encryption that lets a server evaluate functions over ciphertext without holding the secret key, with the result staying encrypted until the client decrypts it.

Applied to RL, it splits the training job into three clean roles:

The environment owner keeps the context and the verifier and never surrenders the key. The compute operator receives protected inputs, runs the model it was permitted to run, and returns a result it cannot read, so decryption and reward calculation happen back inside the trusted boundary. The rest of the RL stack continues as before, since the lane only changes how a rollout request travels and where it may be opened.

Three parties, who sees what

A protected training job has three parties. The customer owns the environment: the tasks, the tools, the private context, the rubric, and the encryption key. The RL lab brings the base model and its trainer. The inference provider brings GPUs. Each party gets what it came for, and no party receives another party’s plaintext.

	Customer	RL lab	Inference provider
Environment: tasks, context, tools, rubric	Keeps it, in plaintext, at home	Never sees it	Ciphertext only
Rollouts	Opens them locally with the key	No plaintext replay buffer	Ciphertext only
Rewards	Scored at home by its own verifier	Receives the improvement, never the per-answer scores	Never sees them
Training updates	Encrypted	Encrypted	Encrypted
Trained adapter	Holds one key share	Holds the other	Ciphertext only

The adapter row is the one that changes the business. The key to the trained result is split between customer and lab, so neither side can walk away with it alone, and revoking a license means ending key participation rather than asking somebody to delete a checkpoint they already copied.

Component by component, the same guarantees look like this, and the inventory holds whether the loop is full RL training or fine-tuning on data drawn from the environment.

Component	What it is	How it is protected
Task context	Repo state, case files, measurements fed to the model	Crosses to compute only as ciphertext under the owner’s key
Tools and harness	Sandboxes and tool logic the agent acts through	Execute inside the owner’s boundary; only tool results enter the encrypted lane
Rubric and verifier	Grading logic, hidden tests, gold answers	Never transmitted; scoring happens after local decryption
Rollouts in flight	Prompts, intermediate actions, model outputs	Encrypted end to end; the operator handles ciphertext only
Fine-tuning batches	Instruction and label pairs built from the environment	Same lane, same key, same guarantee
Encryption key	Opens every rollout and result	Client-held, never leaves
Reward signal	The scores that steer learning	Computed locally, then encrypted straight into the training step, so the rubric never becomes a queryable oracle
Trajectories at the trainer	Token sequences the optimizer consumes	Remain ciphertext through the gradient step; the update decrypts only under the owner’s key, so even the trainer never reads them
Policy weights	The trained adapter, the distilled result	Encrypted under a key split between the parties, so release is a decryption performed on agreed terms

Nothing in the loop requires trusting an operator. Rollouts cross as ciphertext today, the gradient step is the same lane extended one hop, and split keys turn the finished weights into an artifact no single party can open by itself. What used to be a contract becomes a decryption.

The loop, step by step

1. Prepare the task. The customer’s SDK tokenizes the prompt and context at home and encrypts the token IDs before anything leaves.
2. Generate the rollout. The inference provider runs the model over ciphertext and returns encrypted output tokens along with the policy statistics the algorithm needs.
3. Call tools. The model’s tool action comes back encrypted; the customer decrypts it locally, runs the repository, instrument, or database at home, and sends the encrypted result to continue the rollout.
4. Score the work. The verifier grades the finished trajectory inside the customer’s boundary, and the reward is encrypted straight into the training circuit.
5. Compute the update. Remote workers run the LoRA gradient and optimizer step over encrypted tensors and emit an encrypted adapter delta.
6. Store the checkpoint. The adapter accumulates as ciphertext under the split key; no plaintext copy exists at any provider.
7. Serve the result. Authorized users invoke the base model plus the encrypted adapter through the SDK, and responses come back to them encrypted, the same lane the loop started on.

Keep Prime. Add privacy at the inference boundary.

RL teams have trainers, checkpoint pipelines, environment harnesses, and observability, and asking them to adopt a new engine would kill encrypted RL faster than any technical flaw would. The integration has to look boring from the training system’s side.

Prime’s training interface renders token IDs locally and sends them to an inference endpoint, and Verifiers can already point at any OpenAI-compatible endpoint. That seam gives Mirror a place to sit without touching the environment or the optimizer. Prime still believes it is requesting a rollout, and the GPU operator still believes it is serving inference, while the sensitive part of the rollout crosses the compute boundary encrypted.

The separation of concerns is the product:

• Prime owns RL orchestration. Environments, sampling, rewards, training, and policy updates remain where RL teams expect them.
• Mirror owns the encrypted lane. Protected requests go out to model inference, and protected results return to the key holder.
• The data owner keeps the IP. Private context, graders, workflow logic, and keys remain under its control.

No one forks a trainer, learns a new environment format, or takes custody of somebody else’s source tree.

What runs today

Mirror’s vectax path has served protected inference against open-weight models. Requests crossed the service boundary encrypted and came back the same way, with the authorized client opening results locally. Separately, the mirror-rl-compatible adapter accepts the rollout shape an existing trainer emits and returns the result that trainer expects.

The encrypted update step extends the same lane: remote workers compute the LoRA gradient over encrypted tensors against an open-weight base model, and only the encrypted adapter delta survives the step. Together, the capabilities establish the adoption path. Encrypted inference becomes one more endpoint inside a training system a team already runs, and a lab never has to choose between its existing stack and a protected inference boundary.

What this unlocks

1. Scientific reasoning

A long trajectory in a lab starts with inspecting a result and forming a hypothesis, then runs through a written analysis, a confound caught halfway, a revised plan, and a rerun. Almost none of that survives into the paper.

An environment built from the full history would carry unpublished measurements, images, experimental conditions, instrument logs, and a ground truth the group already knows. The verifier stays with the research organization while protected inference runs on outside accelerators, so private research history becomes a learning environment before it ever becomes someone else’s plaintext data lake.

2. Software engineering

The coding tasks with economic weight live inside repositories, far from coding contests. An internal environment includes the repository state and build system, plus hidden regression tests, deployment rules, and a local definition of an acceptable change.

Through an encrypted lane, a company can spend elastic or distributed inference on agentic rollouts while that context, along with the grading logic, stays inside its own security boundary. The model gains experience from real engineering work, and the compute operator’s logs hold ciphertext where a copy of the source tree would have been.

3. Healthcare and enterprise operations

Case reviews, policy navigation, record reconciliation, and exception handling are valuable to learn from because of their context, and that same context defeats anonymization, since stripping it removes whatever made the workflow instructive. Protecting the computation sidesteps that trade, because the sensitive material stays in ciphertext whenever it leaves the premises. A case file can stay a case file, with its details intact, while the learning happens on hardware the hospital or enterprise will never audit.

From selling datasets to granting environment access

This architecture changes the commercial model for AI data.

A conventional data transaction transfers a copy, after which the owner holds a contract and little technical control over where the corpus travels, what it gets combined with, or which future training run it lands in. An encrypted environment supports a different product: access to a capability development loop.

The owner exposes a defined task distribution, tool surface, and reward contract while the underlying context and grading assets stay private. A lab can measure whether the environment improves a policy before buying anything wholesale. Usage can be bounded, audited, and tied to an agreed model or project. Governance, commercial trust, and careful reward design remain necessary, and negotiations get to start from scoped access instead of a bulk transfer.

Privacy expands the supply of useful RL data

FHE gets described as a security feature, and for RL its larger effect may be economic.

It brings previously unavailable environments into the training market. Organizations that would never sell or upload raw workflows may accept a model learning through a controlled encrypted interface. AI labs gain access to more diverse compute while keeping proprietary rollouts out of every operator’s trust boundary, and environment companies get to build durable products in place of one-time data exports.

This is how encryption addresses data scarcity. It leaves the supply of generic tokens untouched and makes scarce, high-value experience usable instead.

The opportunity

The next round of capability gains will draw on thousands of small, precise environments that represent real work, together with reward systems sharp enough to separate a plausible answer from a useful one. Prime and other RL platforms are building the machinery to train on those environments, and the workflows they need already exist inside organizations that have had no safe way to share them. The missing layer connects the two while raw IP exposure stops being the price of participation.

That layer is the encrypted RL lane. Both the adapter and the encrypted inference path run today, and the open question is which environments come online first.