Stealing Reasoning Traces from Proprietary LLM APIs
Abstract
Encrypted reasoning traces shared across sessions and models can be intercepted and injected into weaker models to extract proprietary reasoning, private data, hidden hazards, and hidden prompts.
Leading large language model providers now conceal their models' step-by-step reasoning, or chain-of-thought, to protect intellectual property and limit information leakage. Rather than storing these traces server-side, providers return them to the client as blocks of encrypted text, which the client passes back with each subsequent request. Building on prior research, we identify an architectural vulnerability: these encrypted blocks are fully compatible and interchangeable across different sessions, users, and models within a provider's ecosystem. We exploit this compatibility to develop a scalable decryption jailbreak. By injecting an encrypted reasoning trace from a given model into a weaker, and less safeguarded model from the same provider, we force it to decode and output the trace verbatim in plaintext, without ever jailbreaking the more capable model directly. This vulnerability enables four distinct attack vectors. First, it circumvents anti-distillation mechanisms, allowing adversaries to extract a proprietary model's reasoning, as we demonstrate across Anthropic, OpenAI, and Google. Second, it allows for large-scale private data extraction. Developers frequently share session logs publicly, unaware of contents of the encrypted blocks. By decoding 315,320 reasoning blocks scraped from public repositories, we recovered 367 Personally Identifiable Information (PII) artifacts and 182 credentials. Third, it inadvertently reveals hazardous information hidden within the reasoning process, even in cases where the model's final, visible output safely rejects a malicious request. Fourth, attackers can leverage this flaw to execute invisible prompt injections, embedding malicious payloads entirely within encrypted blocks to poison public agentic rollouts. Following responsible disclosure, we propose concrete cryptographic and system-level mitigations to secure client-side reasoning.
Community
Turns out its possible to decrypt the reasoning of proprietary models
Calling those traces "encrypted" is doing a lot of work. If the same block decrypts across sessions and users, it's not bound to anything — it's obfuscation with a shared key, and that's a deployment problem, not a research curiosity. Anyone building agents on proprietary APIs should assume chain-of-thought is readable until the provider actually binds the block to the session and user. That changes what you log, what you cache, and what you let the model reason about in the first place. I'd want to see the provider commit to per-session binding before I trust it with anything sensitive.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Rewriting the Response Path: Silent Tampering and Provider-Signed Defense in BYOK LLM Agents (2026)
- IssueTrojanBench: Benchmarking AI Coding Agents Against Malicious Issue Requests (2026)
- The Proxy Knows Too Much: Sealing LLM API Routers with Attested TEEs (2026)
- Isolated but Exposed: Persistence-Based Memory Extraction Attack on LLM Agents (2026)
- Beware What You Autocomplete: Forensic Attribution of Backdoored Code Completions (2026)
- Evidence-Bound Gateway-Path Provenance for Third-Party LLM Inference (2026)
- "What Happens Locally, Leaks Globally": Detecting Privacy Leakage Risks in MCP Servers (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper