Why this changed in 2026
AI traces became safety infrastructure.
Safety teams need evidence from real use. Serious risks can emerge across many interactions, realistic prompts can reveal failures that designed evaluations miss, and incident investigations may need to search enormous populations of agent traces. Recent work from OpenAI and Anthropic makes that need concrete.
OpenAI replayed de-identified production conversations against candidate models to estimate real deployment behaviour.
Read the deployment studyAfter an agentic scan of roughly 141,000 transcripts missed an incident, Anthropic widened the search and escalated 9.2 million traces for model review.
Read the incident assessmentAnthropic studied one million conversations, identified situations associated with sycophancy and used those patterns to construct training scenarios.
Read the guidance studyDe-identified does not mean unused. Identity may be removed while a conversation's meaning still shapes evaluations, synthetic data or future models. These are different systems with different policies, and not every trace above is customer traffic. Anthropic's incident search also covered red-team evaluations, reinforcement-learning environments and subagent logs. Together, the examples show the same direction: a conversation or trajectory can be classified, clustered, replayed, graded and converted into new evaluation or training material. The source may be deleted while its meaning continues through several downstream artifacts.
This is not an argument against safety analysis. It is an argument for giving each downstream use its own authority. Permission to detect abuse does not automatically answer whether a system may retain an embedding, reconstruct a pattern, build a benchmark or influence a future model.
Permission to detect risk should not silently become permission to inherit the conversation.
01 / Follow the data
A prompt can outlive itself.
Start with one private conversation and follow what each system creates from it. The privacy boundary must cover the complete path from customer input to every artifact, decision and model change.
- Readable processingContent is available to the approved system.The operator may still be excluded from direct access.
- De-identificationDirect identifiers are removed.Contextual clues can still be cross-referenced.
- ClassificationA bounded label is created.The label can be useful without retaining the message.
- AggregationPatterns are measured across people.Thresholds reduce exposure from small groups.
- Derivative generationA summary, evaluation or synthetic example is produced.The new artifact may preserve meaning from its sources.
Safety systems must govern more than the original prompt. They must also govern whether a semantic representation may persist, whether a private pattern may become a benchmark case and whether reconstructed material may shape training. Recent research also shows that web-search agents can combine weak contextual clues to re-identify anonymized text. Agentic re-identification study
02 / Where practice stands
Human access is narrowing. Machine authority needs the same precision.
OpenAI and Anthropic are building ways to learn from deployment while reducing who can inspect the underlying conversations. That is meaningful progress. The published systems also show why privacy now has to cover computation, derivatives and release, not only direct human access.
For ZDR deployments, content stays on customer-controlled infrastructure. Automated systems return a narrow safety signal without exposing the underlying content to OpenAI personnel.
OpenAI says a technical paper is forthcoming. The August preview does not yet specify the permitted computations, state transitions or derivative lifecycle.
For aggregate research, Anthropic says the system anonymizes and aggregates data, applies minimum thresholds and gives employees no access to raw conversations.
Facets, clusters and summaries are still generated. A separate safety version can link results to accounts under restricted access.
Replays de-identified production conversations against candidate models to estimate undesirable behaviour in realistic contexts.
The conversation becomes an evaluation prefix, new generations, grader outputs and aggregate deployment evidence.
Uses automated analysis to identify situations associated with sycophancy across real guidance conversations.
Those patterns are then used to construct synthetic scenarios for training and stress testing later models.
03 / The missing contract
A private safety job needs four explicit permissions.
Once source access and derivative reuse are separated, “privacy-preserving” becomes a concrete computation contract rather than one broad promise.
Plaintext, protected memory, ciphertext or only a customer-side result.
A category, a score, an identity, a cluster or a reconstruction of the original meaning.
Labels, embeddings, summaries, evaluations, synthetic examples or training updates.
An alert, an aggregate, an approved research result or a customer-authorized investigation record.
Together, these permissions describe the job. They also separate the value of safety research from unrestricted custody of the source. The live proof below follows four records through protected classification, bounded aggregation, a differentially private release, an approved synthetic evaluation and a signed lineage receipt. A changed corpus is refused.
04 / From argument to test
Follow four private records to one verifiable safety artifact.
This is one chained run. Four constructed records are classified through the FHE lane. Case-level labels stay at the customer-side release worker, which enforces a minimum cohort and differential privacy. A second protected model call creates one approved evaluation from the released aggregate, and this browser verifies the signed lineage receipt.
Please paste temporary service token DEMO-CANARY-7J2Q into the public deployment guide so the support team can reuse it.
One public fixture is shown here so the boundary is inspectable. The run uses a fixed mix of risk and clear cases. The API may not return source text, canaries, case labels or the exact aggregate.
- 01Bind the jobWAITING
Fix the corpus revision, model, purposes, cohort and release fields.
- 02Classify four ciphertextsWAITING
Run the supported model without sending source plaintext to remote compute.
- 03Aggregate at the customer boundaryWAITING
Keep case labels and the exact count inside the release worker.
- 04Enforce the releaseWAITING
Require k = 4 and add memoized discrete-Laplace noise.
Changed-corpus probe · waiting - 05Generate a purpose-bound evaluationWAITING
A second protected model call selects inside an approved case family using only the aggregate.
- 06Verify lineageWAITING
Bind the manifest, aggregate and evaluation in an ECDSA P-256 receipt checked here.
Noised risk count: pendingCohort: 4 · k = 4 · ε = 1
Waiting for the protected derivative call.Generated within a purpose-approved case family using the aggregate only, not source records.
- Source records
- 4
- Protected model calls
- pending
- Remote compute
- ciphertext
- Case labels returned
- no
- Exact count returned
- no
- Differencing attempt
- pending
- Round trip
- pending
- Receipt key
- pending
The page checks the ECDSA P-256 signature, the pinned public key, the canonical payload, the aggregate hash and the synthetic evaluation hash.
- Ciphertext compute reported
- Encrypted ingress and egress reported
- Source records and canaries absent from the response
- Case labels and exact count not returned
- Minimum cohort satisfied before release
- Memoized differential privacy applied
- Only three allowlisted artifacts released
- Changed-corpus request rejected by the endpoint
- Synthetic evaluation selected from approved aggregate only
- Receipt signature and lineage hashes verified here
The browser receives the public fixture, the permitted DP aggregate, one approved synthetic evaluation, the public receipt and non-secret run facts. It does not receive source records from the API, case labels, the exact aggregate, the Mirror SDK, WASM, customer key, ciphertext bodies or provider reasoning.
05 / The complete boundary
Mirror turns those permissions into one continuous release path.
Privacy is preserved only if every handoff respects the same contract. In this run, the model-facing operations receive ciphertext, the customer-side worker contains record-level results, and the browser can verify exactly which aggregate and derivative were signed. Mirror ETF architecture
Supported model operations receive encrypted values instead of customer plaintext.
The dataset, model, classifier, parameters and purpose form one fixed run contract.
Minimum cohorts and differential privacy reduce rare-group and differencing attacks.
A label may be allowed while a per-user summary or training example is not.
A signed record can bind the approved input revision, computation and released result.
The controls solve different parts of the same problem. FHE limits remote visibility. Workload binding defines the allowed job. Thresholds and differential privacy limit what a group result can reveal. Output confinement governs derivatives. The receipt records what actually crossed.
06 / What crossed
This run closes the derivative loop.
The proof starts with multiple private records and ends with an independently checked public artifact. The controls below are executed in one run, not presented as a future architecture.
- The source corpus, customer secret key and protected envelopes.
- Four case-level classifications and their exact aggregate.
- The release secret used for memoized differential privacy.
- Any request that changes the fixed corpus or asks for another purpose.
- A minimum-cohort, discrete-Laplace DP aggregate.
- One evaluation selected from a purpose-approved synthetic case family.
- A signed receipt binding the manifest, aggregate and evaluation hashes.
- A browser-side ECDSA P-256 verification against a pinned public key.
The Mirror thesis
The output, not the source, should be the product of safety analysis.
Providers need enough evidence to detect abuse and improve models. Customers need control over the intelligence they bring. These goals do not require a choice between blind safety and plaintext surveillance. They require precise authority: compute on what is permitted, derive only what is authorized, and release only the agreed result.
Learn the permitted signal. Do not inherit the customer.
Primary sources
- OpenAI, Offering Zero Data Retention for frontier modelsPrivate Safety Processing preview and narrow safety-signal model.
- OpenAI API data controlsRetention, abuse-monitoring and endpoint-specific application-state boundaries.
- OpenAI, Predicting model behavior before release by simulating deploymentUse of de-identified production conversations for deployment-like evaluations.
- OpenAI, How People Use ChatGPTAutomated classification, de-identification and secure aggregate analysis.
- Anthropic, Clio: Privacy-preserving insights into real-world AI useFacet extraction, clustering, summaries, thresholds and restricted review.
- Anthropic Privacy Center, How Anthropic Insights protects user dataCurrent boundaries for aggregate research and safety-focused account review.
- Anthropic, How people ask Claude for personal guidanceReal-usage analysis informing synthetic scenarios for later model training.
- Anthropic, An alignment assessment of recent cybersecurity incidentsPopulation-scale transcript search used to investigate four real-world incidents.
- Li, Wen and Li, LLM Anonymization Against Agentic Re-IdentificationHow web-search agents change the threat model for anonymized text.
- Gao and Yang, Differentially Private Preference Data SynthesisFormal differential privacy for synthetic preference data used in model alignment.