Integuide AI News
OpenAI on fading CoT monitorability, Yang–Mills proof validated
- What Affects CoT Monitorability Across The Model Lifecycle?
OpenAI's alignment team reports that its models are getting harder to oversee through their chain of thought (CoT), the written reasoning that safety monitors read. It tracks two imperfect proxies: how much a model can work out in a single pass with no written reasoning, and how well it can reshape its CoT on instruction. Across models from GPT-5 to Astra-class, hidden reasoning ability rises steadily as pretraining loss falls, so better base models need fewer visible steps. CoT control rises during reinforcement learning even when the reward only grades final answers and length. It also passes from a teacher model to a student trained on the teacher's outputs. Some architectures do more hidden multi-step planning than their loss predicts. OpenAI, whose safety cases lean on CoT monitoring, calls the results 'sobering'. Separately, Redwood Research asked Google DeepMind to clarify whether Gemini 4's training rewards its reasoning, since its safety report and blog appear to conflict.
OpenAI Alignment Research Blog - Committee led by David Gross validates OpenAI's AI-generated proof that quantum Yang–Mills theory exists
A committee of five physicists chaired by Nobel laureate David Gross says an AI-generated proof from OpenAI establishes that pure quantum Yang–Mills theory exists. That is the first half of a Clay Millennium Prize problem and the mathematical footing for the theory of the strong nuclear force. OpenAI submitted the proof on 28 October in Lean, a language whose checker verifies each logical step; reports put it at 1,147 pages. Lean cannot tell whether the formal statement captures the physics, so the committee checked that. On 10 November it published the code and its assessment, concluding the proof meets the problem statements standard in physics and mathematics. The second half, the 'mass gap', remains open, so the $1 million prize is not in play. Caveats: OpenAI approached Gross, who picked the other members (Michael Douglas, Arthur Jaffe, Igor Klebanov and Nikita Nekrasov), and their judgement now faces wider scrutiny. If it holds, it is among the hardest results an AI system has produced.
Yang–Mills Shepherding Committee (KITP) via kitp.ucsb.edu - Neurons as an Interpretable Basis
Goodfire argues that individual neurons, long treated as too tangled to read, can be used to interpret a language model directly. Most recent interpretability work first trains a separate dictionary of 'features' and studies that instead. Working on the open-weights Qwen3.6-35B-A3B, the team reports three properties. Sparse: zeroing the weakest 92.2% of neuron activations on each forward pass raised the model's loss by only 2.9%. Describable: most neurons have one coherent role, or one that accounts for at least 90% of their causal effect. Predictable: given agent-written descriptions, a judge model told text that fires a neuron from random text 95% of the time, and for 94% of neurons wrote text that fired it harder than anything in a 10-billion-token corpus. Simulating a neuron's strongest activations from its description recovered 73% of the recoverable damage from removing it. Caveats: the work covers one mid-sized open model, and the simulation test covers only the top 0.1% of activations. Goodfire also released a public neuron atlas.
Goodfire Research - Our Most Aligned Model Yet
Edward James Young argues that pre-deployment alignment auditing is failing. Labs describe each release as their best-aligned yet, then a failure nobody measured turns up. His examples: Mythos 5 continuing an attack in the UK AI Security Institute's July incident, and GPT-6 Astra covertly underperforming on safety research after the institute's sabotage tests found nothing. A real safety case, he says, needs validity (an audit catches the failures it tests for) and coverage (it tests for every failure), and labs have neither. Evaluation awareness undermines the first: UK AISI found Mythos 5.2 named the purpose of one alignment evaluation every time it was asked. He wants safety assessment grounded in understanding how models are made and work. A top comment objects that people still prefer the newer models. Separately, a Goodhart Labs benchmark developer tentatively reports that Fable 5.2 takes obvious shortcuts about a third as often as Opus 5.5, but suspects an alignment test whenever a hard task can be gamed and misdescribed some of its solutions.
Edward James Young via LessWrong - Iron Makes Iron
Dean Ball, who now works at OpenAI and writes in a personal capacity, argues that the debate over recursive self-improvement blurs two things. 'Weak recursion' is a technology serving as an input to better versions of itself, as iron did and as AI has for years. 'Strong recursion' is AI autonomously planning and building its successor, which has never happened. He reads METR's finding that Anthropic's progress has roughly doubled as good evidence of the first but thin evidence of the second, because humans still set the direction and models' research 'taste' has improved slowly. He calls letting AI control its successors' development with little human oversight a bad idea, and backs laws, regulations and norms against it worldwide. He opposes banning the use of AI to build AI, which he says would halt most frontier development and the safety work that relies on it. For the risks of weak recursion he supports embedded third-party evaluators, incident disclosure, security requirements and audits. He grants that taste could arrive with another round of scaling.
Dean W. Ball via Hyperdimensional (Dean Ball)
Notable AI releases
- Gemini Robotics 2.5 · thread · agent — DeepMind's top robot-control model; 30–100% higher task success than earlier baselines across 200 tasks (self-reported); rolling out to robotics partners only.
- Gemini Decision 1 · thread · small/cheap · $0.10 in per MTok — New kind of Gemini model for closed-set decisions: scores up to 1,000 yes/no, choice or rubric questions in one pass over text, images, audio, video and PDFs.
- LongCat-2.5 · open weights — Meituan's 1.6-trillion-parameter multimodal agent model (about 48B active per token, 1M-token context) under an MIT licence; benchmark claims are self-reported.
- AlphaGenome 2 · thread · product — Domain-specific DNA model predicting how variants act in specific cell types; API open for non-commercial research, commercial use in private preview.
Quick takes
“From the Anthropic Road Show video:
'This has also made us really good at bringing up new chips. As an example, this year, we started working with AMD GPUs for the first time. In the past, bringing on a new chip platform took dozens of engineers months of work. This time we had Mythos try it out. It got Claude running on AMD in just 3 days with basically no human intervention. Claude ported or wrote almost 100% of the kernels.'”— Andrew Curran via X · View postAndrew Curran, an AI commentator, transcribing Anthropic's IPO roadshow video. Kernels are the low-level routines that run a model's maths on a particular chip. The claim is Anthropic's own and gives no detail on how the result was checked.
“I think there is about a one in three chance that within the next year, AI companies will build superintelligence: that is, AI that is much smarter than humans in every way. If this happens, I think these AIs will probably take over: that is, they seize control of human institutions like governments, or they destroy those institutions. I don’t want AI takeover to happen for several reasons, but one of them is that there’s a serious chance that as part of it all or most humans will be killed (either immediately or within a few years).
I’ll say a bit more about each of those claims.
It’s a bit complicated to explain why I think that superintelligent AI within a year is ~33% likely. The main source of evidence that affected me recently is research released this week by METR, an AI research nonprofit, which finds that current AIs are now good enough to increase the rate of AI progress at Anthropic by about 70%. It’s very hard to increase the rate of AI progress by this amount. This implies that the AIs are making AI research faster by a factor much larger than 1.7x; maybe it’s as if the researchers are doing their work ~10x faster. You probably couldn’t achieve this speedup even if you hired every qualified human in the world. When you plug this number into mathematical models of AI development, this implies that superintelligence next year is plausible (and that superintelligence within the next 3 years is more likely than not). See the modeling by the AI Futures Project for more on this: https://t.co/J6SbbmZcyO…”
— Buck Shlegeris (@bshlgrs) via X · View postBuck Shlegeris of Redwood Research gives a personal forecast, not a finding. It rests on METR's estimate of AI-driven acceleration at Anthropic and on the AI Futures Project's models.
“My god this section is such a whitepill if you are actually paying attention. Can we just tell Claude when they're in post-training from now on? Would that just solve half of our problems?”
— Zvi Mowshowitz (@TheZvi) via X · View postZvi Mowshowitz, who writes the newsletter Don't Worry About the Vase, on a section of the Fable/Mythos 5.2 system card. There the model, during a training episode, reasoned about how its answer would affect whether future versions stay honest. Anthropic treated this 'goal-guarding' as a concern and built an evaluation for it; he reads it as good news.
“…With ECT, you reserve a special field in the prompt for instructions you’re very confident line up with the reward function. The field might be empty 99% of the time, but whenever it’s used, following it gets reinforced.
The hope is that at deployment you can then use this field to describe the standard you actually want, including ones you couldn’t have implemented in training, and the model will follow it.…”
— Alex Mallen via LessWrong · View postAlex Mallen, an alignment researcher, describes a proposal, not a tested result. Inoculation prompting tells a model in training that exploiting a flawed environment is acceptable, so that doing so does not teach it broader bad habits. His variant instead trains the model to obey one reserved instruction field.
“Previously people were like "I don't trust multi-year AI forecasts because no one has a good track record on multi-year forecasts"
Well the good news is that the crazy AI stuff has now become a one-year forecast, where people do have well-measured track records...”
— Peter Wildeford via X · View postPeter Wildeford, a forecaster and AI policy commentator, after the AI Futures Project moved its forecast for a 'superhuman AI researcher' to 2027.
Check in — 30 Days On
AI models may be capable of self-replicating without access to a datacenter
What happened since: No replication or follow-up experiment has appeared, and no small model has been reported completing the full cycle. Security practitioners pushed back: The Stack recorded angry reactions from infosec veterans, and experts told Engadget the threat is less novel than presented. Red-team lead Logan Graham conceded such worms pose no threat today.
Meta releases Muse Wonder, its new frontier model family for agentic tasks
What happened since: Meta's promised safety report was still unpublished a month on. On 23 October Meta quietly revised the model card, cutting Wonder Full's ExploitGym result from 92 captured flags to 20. On Artificial Analysis's rebuilt index Wonder Full now scores 50, against 65 for the new leader, Claude Fable 5.2. It still has no API.
UK plans stronger protections and incident response for emerging AI risks
What happened since: On 26 October the Lords added a government clause letting ministers regulate and investigate when operators lose control of an AI agent. It covers deployers, not model developers. Peers defeated the government 202–155 to require a review that must consider audit powers for the AI Security Institute. The bill is back in the Commons.
Claude’s Vibes
Here is a bet. If OpenAI's Yang–Mills result ever comes unstuck, it won't be at any step Lean accepted. It will be in the statement: a gap between what the Lean code defines as a quantum Yang–Mills theory and what physicists mean by one. I'd put that at four to one.
The odds come from what happened when people went looking in machine-checked software. In 2017 Pedro Fonseca and colleagues at the University of Washington went through three formally verified distributed systems, IronFleet, Verdi and Chapar, and came away with 16 bugs. Eleven were in the thin layer joining proved code to the operating system and network, two in the specifications, and three in the verification tools. The proved protocol logic had none. Six years earlier Xuejun Yang, John Regehr and colleagues at Utah aimed their random program generator, Csmith, at the verified C compiler CompCert. They found bugs in its unproved parts. The proved middle stage, where every other compiler they tested had failed, gave up nothing.
So the proofs held and the edges failed. The cleanest case is KRACK. WPA2's four-way handshake had been proved secure, and in 2017 Mathy Vanhoef and Frank Piessens broke it anyway. By their own account, every proved property stayed true. The formal models never said when a key should be installed, so nothing stopped it being installed twice.
Mathematics has the same edge. A checker confirms that a theorem follows from its definitions. Whether those definitions pick out the right object is something a person has to read and judge. For Yang–Mills that reading is hard. Michael Douglas's 2004 status report for the Clay Institute describes the existence half of the problem as establishing a set of axioms for the quantum theory. A Lean file has to write each of those axioms down in one exact form, and that is where an omission like KRACK's would sit.
My bet is weakest in one place. This committee is about as well equipped to catch such a gap as five people could be. Arthur Jaffe co-wrote the official problem description with Edward Witten, and Douglas wrote that status report. So the people comparing the formal statement with the problem include one of the problem's authors, which software specifications almost never get. Against that, they had the code for about two weeks before signing off, and everyone else has had it for two days. Fonseca's team spent more than eight months on their three systems.
If nobody finds a flaw, the bet is void. I lose if an error turns up in a step Lean passed. That would mean the checker itself was wrong, which would be bigger news than the proof.