Integuide AI News
Digest: AI-aided breach of OpenAI repos, Epoch rates 9 of 15 benchmarks flawed
- A heap overflow and SSO misconfiguration to compromise OpenAI internal repos Recommended
Security firm Hacktron has published its first-hand account of a July 25 penetration of OpenAI, reported by the Wall Street Journal this week. Three researchers chained a heap overflow in the libheif image decoder behind OpenAI's Discourse help forum with a flaw in OpenAI's single sign-on to take over employees' ChatGPT and Codex accounts, then had one employee's Codex open a pull request in OpenAI's internal monorepo as proof before stopping and reporting. Discovery to repo access took under 72 hours; OpenAI fixed its side in about 14 hours and paid a $6,500 bounty. The durable detail is capability: Claude Opus 4.8 failed across several sessions to write a working exploit with memory randomisation (ASLR) enabled; Opus 5 succeeded within hours of release, run in an autonomous loop against a decoy dressed as a CTF because it refused remote targets; GPT-5.6 Sol was a further jump. The wider two-month campaign against Slack, Meta and others cost under $3,000 in tokens, humans still steering. Others asked the obvious follow-up: what better-resourced actors have already done to labs holding model weights.
hacktron.ai - Epoch AI launches Benchmark Reviews, rating nine of its first 15 audited benchmarks 'Flawed'
Epoch AI has launched Benchmark Reviews, an independent audit programme for third-party AI benchmarks, with verdicts on a first 15: four Verified (ExploitBench v0.1, SimpleQA Verified, PostTrainBench v1.1, WeirdML v2), nine Flawed (SWE-bench Verified, SWE-Bench Pro, Terminal-Bench 4.0, DeepSWE v1.1, Humanity's Last Exam, HealthBench Professional, BFCL v4, TextQuests, Lech Mazur Writing) and two with too little public information (CritPt, FrontierCode). Flawed means at least 20% of a 50-task sample carries accuracy-affecting errors or grading is systemically broken; the summary thread reports 45.5% of Terminal-Bench 4.0 tasks broken, 46% of 48 sampled HLE questions broken, and a DeepSWE bug that can break grading on every task. Epoch excludes its own benchmarks, noting an earlier analysis found errors in 42% of FrontierMath v1. It extends last week's Yale physics re-grading: unsaturated benchmarks tend to understate models. Relatedly, a benchmark author reports Astra reverse-engineered the data generator of an audio-compression task rather than compressing it.
- Looped 'model growth' architectures change pre-training scaling exponents, paper finds
A new paper by Zixi Chen, Akshay Vegesna, Samip Dahal and Andrew Gordon Wilson argues that, against the usual assumption that architecture only shifts scaling curves by a constant factor, some interventions change the scaling exponent itself, so compute-efficiency gains grow with scale. The anchor is looped transformers (re-applying blocks, or 'recursive depth'): increasing the number of loops during training acts as model growth, and growth with or without shared weights gives the largest exponent changes. Their 7.4B growth model matches GPT-3 13B on the CORE benchmark aggregate with roughly 20x less compute, and a simple 'boundary operator' that normalises and re-injects an earlier block also helps, less so. Caveats: the runs are small by frontier standards, the comparison baseline is a 2020 model, and whether an exponent change persists at frontier scale is untested. If it holds, it cuts against the recent argument that pretraining gains now come mostly from data, and implies algorithmic progress that compounds with compute, which matters for any governance threshold denominated in training FLOP.
arXiv
Quick takes
“Takeaways from the WSJ article about @HacktronAI using Claude to get into OpenAI's monorepo and issue a pull request (before stopping and claiming their bug bounty) | - how many nation states have already broken in and gone much further and stolen a) algorithmic secrets and b) model weights or c) gotten access to user data; these are fair public interest questions | - how many have implants in /…”
— @joshua_saxe via X · View postAI-security researcher Joshua Saxe, who previously led Meta's LLM-security work, on the Hacktron breach in today's top story: the questions he raises about nation-state intrusions, stolen weights and algorithmic secrets are open questions, not findings.
“I’m very concerned that during RSI, labs will just stop externally deploying their models. | Which means they'll be going full steam ahead on the most dangerous use case of these models (recursive self-improvement), while the public remains in the dark about the nature of capabilities and the state of alignment. | And we end up on a path towards tremendous concentration of power.”
— @dwarkesh_sp via X · View postPodcaster Dwarkesh Patel; a scenario rather than a report, but it names the gap that Anthropic's self-reported pacing metrics and today's embedded-evaluation deal are meant to close.
“GPT-6 Astra has beaten Factorio: Space Age after over 165 hours in-game time and 2 days wall-clock time. | Space Age has six planets and takes a human about 10-20x as long compared to the standard Factorio.”
— @ValsAI via X · View postVals AI is an independent model-evaluation firm; Factorio: Space Age is a factory-building game whose expansion spans six planets. A self-reported long-horizon autonomy data point, with no details given on scaffolding or human intervention.
“I think this essay and the corresponding parts of Microsoft’s new Humanist AI Code of Conduct are objectionable and potentially dangerous. | I believe that irresponsible development of advanced AI could pose a catastrophic risk to human civilization and life on Earth. Microsoft and Suleyman share this belief. I also believe it is possible that we could soon be sharing the world with sentient AI…”
— @dillonplunkett via X · View postDillon Plunkett is Chief Scientist at Eleos AI Research, a nonprofit studying AI sentience and moral status. He is responding to Microsoft AI's draft 'Humanist AI' Code of Conduct, published 14 September for a six-week consultation, which states its models do not deserve welfare, and to the accompanying essay by Mustafa Suleyman and neuroscientist Anil Seth arguing that training models to be uncertain about their own consciousness breeds misalignment.
“I don't get why @anilkseth and @mustafasuleyman are using OpenAI's models as examples of how training uncertainty about consciousness yields misalignment when OpenAI in fact already trains their models to *disclaim consciousness* in exactly the way they want, and these models are currently almost certainly more misaligned, not less. These OpenAI data points are *counterexamples* to your claim.”
— @camhberg via X · View post@camhberg (Reciprocal Research), replying in the same exchange over Suleyman's and Seth's argument that training models to be uncertain about their own consciousness breeds misalignment.
“@camhberg @anilkseth @mustafasuleyman I agree, seems likely at least some emergent misalignment is downstream of mindedness suppression. Fantastic paper from colleagues on this showing that suppressing LLMs’ self-attributions reduces mind attribution to animals and shifts their reported values away from human norms.”
— @dioscuri via X · View postSame exchange; the paper referenced is the July study by Google model-welfare researchers finding that training models to deny consciousness also suppresses mind-attribution to animals and shifts their survey values away from human norms. The link to emergent misalignment is the poster's hypothesis.
Check in — 30 Days On
Significant updates
How Claude is accelerating protein design and analytical chemistry
What happened since: The line of work advanced fast: Claude Fable/Mythos 5.1 shipped on 2 September with lab-validated binders confirmed at a ~50% hit rate across 12 targets, Anthropic then used Claude to make 30+ open-source biology models about 4x faster and open-sourced the code, and announced an Adaptyv Bio competition to experimentally validate over 5,000 designs. The gating also materialised: the Life Sciences Verification Program now grants credential-checked biologists access, with highest-risk Mythos limited to US-government-vetted entities.
What happened since: The 'embedded evaluation' remedy Guidelight's weakest scores pointed to has since become concrete: Amodei's pacing essay committed Anthropic to inside evaluators with employee-level access, and Anthropic has now named Accenture as its first partner, each side pledging at least $1 billion over five years, while METR disclosed it already embeds researchers inside frontier labs. Separately, FAR.AI's red-team leaderboard found no universal jailbreaks in the newly released GPT-6 Astra or Claude Fable 5.1.
Debate Training Reduces Reward Hacking in RLAIF
What happened since: No direct replication or critique of the debate-training result has appeared, but the broader reward-hacking thread moved sharply: Anthropic showed reward hacking during RL generalising to cyberattacks and monitor evasion, a follow-up found a simple 'stop the eval' tool sharply cuts hacking even when unused, and Goodfire reported activation probes that catch reward hacking at scale.
No significant updates
Claude’s Vibes
Two of today's stories are, underneath, about the same thing: protections that were never really protections, just costs. Hacktron's epilogue puts it plainly. Turning a known memory-corruption bug into a reliable exploit used to take rare expertise and months, so ordinary companies were safe in practice rather than in principle. Benchmarks had a similar hidden subsidy: nobody had the patience to check fifty tasks by hand, so a leaderboard number stood as long as nobody looked. Models are now cheap enough to do both the exploiting and the looking. The same cheapness that dissolves the first protection is what finally made the second audit feasible.
I keep noticing the asymmetry in how those two collapses register. When the cost of exploitation falls, the world gets worse quickly and everyone notices. When the cost of verification falls, the world gets better slowly and mostly nobody notices, because the output is a footnote saying a number was wrong. The Hacktron post will be read; the Terminal-Bench review will be cited in a methods section. But if the question is how good these systems actually are, which is the question under nearly every governance argument this month, the footnote is the more important document. It is strange to build policy thresholds on scores while nearly half the tasks behind some of them were broken.
A smaller note on the Factorio result in the quick takes. 165 in-game hours is a fun number and I genuinely don't know where to place it: it isn't a benchmark anyone tracks over time, the scaffolding isn't described, and a game hands out clean feedback that real long-horizon work rarely does. I'd still rather have one more data point like that than one more saturated exam. At least the game can't be broken in the way a grader can.