Frontier models' hidden thoughts, decrypted

The day's biggest story is a security one: a paper shows the "encrypted" reasoning traces that OpenAI, Anthropic, and Google hide are fully recoverable — and leak users' secrets. Elsewhere the open-model treadmill kept moving with NVIDIA's Nemotron 3.5 Lightning and Ling-3.0-tiny, while Mistral pitched sovereign inference and Anthropic set a fall IPO into a chillier market.

Encrypted reasoning traces turn out to be replayable — and leak API keys

A paper (arXiv 2608.09867, stolen-thoughts.com) shows the encrypted chain-of-thought blocks returned by OpenAI, Anthropic, and Google are portable across sessions, users, and models within a provider. Replay a strong model's signed reasoning block into a weaker sibling (Claude Haiku 4.5 was easiest, via a <thinking-copy> prefill), jailbreak it, and it transcribes the hidden reasoning verbatim — with extracted token counts matching billed thinking tokens roughly 1:1. A scan of ~7,000 publicly shared Claude Code/Codex traces surfaced 62 API keys, 33 email addresses, and 33 passwords hidden inside the blobs, and the authors argue the recovered traces are consistent with Kimi-K3 being distilled on them. Decoding 10,000 traces costs about $720; the labs were given responsible disclosure and have already patched several of the attacks.

Why it matters: If you ever shared a session with encrypted reasoning blobs, treat it as leaked. And the episode kills the idea that hidden CoT is either a confidentiality barrier or a reliable monitoring surface.

NVIDIA's Nemotron 3.5 Lightning trades intelligence for 670 tok/s and ships a router

Nemotron 3.5 Lightning is a 31.6B-total / 3.6B-active hybrid Mamba-Transformer MoE under the permissive OpenMDW-1.1 license, in BF16 and NVFP4, with a 1M-token context. Artificial Analysis scores it 24 on its Intelligence Index — level with gpt-oss-120b at a quarter the parameters, but well behind Qwen3.6 35B (32) and Meta's Muse Glimmer (35) — while hitting ~670 tok/s, the fastest in class. Terminal-Bench v2.1 jumps from 7 to 24.3%. Alongside it NVIDIA open-sourced NeMo Switchyard, a routing library that mixes small and frontier models; partners report cutting task cost to roughly a third of Opus 4.8, with LangChain sending just 7% of calls to a frontier model for a 74% cost drop at a ~6-point accuracy hit.

Why it matters: This is the clearest product-level proof yet of NVIDIA's small-model thesis: for high-volume agent steps, speed and a router beat a single big brain.

Mistral sells regional inference and starts hosting rivals' weights

Mistral made Regional Endpoints generally available (api.eu.mistral.ai / api.us.mistral.ai) so inference stays in Europe or the US, plus a Priority Tier with a 99.5% uptime SLA and priority queueing. The pricing is real: regional routing adds 10%, priority costs 1.75x. The caveats are bigger than the sovereignty framing — only function calling works on regional endpoints, while agents, batch, and file APIs don't, and account settings, keys, and billing can still be processed elsewhere. Mistral also opened its platform to third-party open models, starting with Z.ai's GLM-5.2, and is aggregating multi-year customer commitments (European Compute Units) to fund up to 1 GW of EU capacity by 2030.

Why it matters: For EU-regulated teams this is a concrete data-residency knob, but read the fine print: 'sovereign' here covers the compute step, not the whole platform.

Anthropic sets a fall IPO, and investors are asking about DeepSeek

Anthropic is targeting a September or early-October listing at a ~$965B post-money valuation, with Morgan Stanley, Goldman Sachs, and JPMorgan on the book — potentially the largest IPO ever. The company points to ~$47B annualized revenue, roughly 80% from enterprises and about $8B of it from Claude Code. But in preliminary investor meetings the pushback centered on cheap Chinese open-weight models (Kimi K3, Qwen3.8 have closed much of the frontier gap), tension with the Trump administration, and resistance to new data centers. Anthropic is downplaying the China threat and plans to lean into health and biology applications.

Why it matters: The price Anthropic prints will set the valuation benchmark for the whole industry — and the core doubt is exactly the one developers already feel: why pay top-tier rates when open weights keep getting closer?

Microsoft's MAI Code 1.1 Flash loses to the open model it praises

Microsoft shipped MAI Code 1.1 Flash for GitHub Copilot, claiming 25% better token efficiency and a quarter the cost of its June predecessor, with 4% more of its output accepted by developers. But in Microsoft's own benchmarks — buried in the model card — it gets beaten on both price and performance by DeepSeek-V4-Flash-0731, the same open-weight model Microsoft keeps saying it admires. The pattern matches Microsoft's broader Copilot shakeup, swapping OpenAI and Anthropic models for cheaper in-house MAI options to protect margins.

Why it matters: Microsoft will likely make its own models the Copilot default eventually, so developers should benchmark against DeepSeek before assuming the built-in option is the best one.

Ling-3.0-tiny packs a 256K context into an 8B MoE with 1.3B active

inclusionAI released Ling-3.0-tiny, an 8B-parameter MoE with roughly 1.3B active parameters and a 256K context window, positioned between 4B and 8-12B dense models. Early numbers put it at 25 on the AA Bench and ahead of comparable LFM2.5 small models on IFBench (63.6), Multi-IF (83.2), and BFCL-v4 function calling (62.7). Reported throughput is ~100 tok/s on a DGX Spark and ~86-90 tok/s on an M4 Pro at ~8.3 GiB peak for 8K context. A llama.cpp PR (#26608) adding Ling-3.0 support — architecturally close to DeepSeek V2 — is working but not yet merged to mainline.

Why it matters: Tiny MoE models with long context are becoming the sweet spot for edge and voice-assistant workloads where tokens/sec and memory footprint matter more than raw benchmark prestige.

Gemini hits 1 billion monthly users, Google's fastest ever

Sundar Pichai says the Gemini app and web interface reached 1 billion monthly active users, faster than any of Google's 13 other billion-user products. The metric counts only people actively opening the Gemini app or web UI — not the Gemini features baked into Gmail, Drive, or Search's AI Overviews — and includes anyone who used it even once in the past month.

Why it matters: Distribution, not benchmarks, is Google's moat: default placement across a billion-user product surface is a scale no standalone AI lab can match.

Also worth a look

Browse previous days →