Integuide AI News
Digest: open model builds Firefox exploits, Codex monitor red-teamed
- Open weight models patch gap Recommended
Epoch AI reports that GLM 5.3 — a widely available open-weight model released by Z.ai in August — autonomously turned published-but-unshipped Firefox security fixes into working one-click exploits for six releases between 17 September and 26 October, at a marginal cost of about $1,000 each in API fees (after roughly $100,000 in setup). The attacks exploit the "patch gap": the window after a fix is public in beta/nightly code but before it reaches users. Given only basic instructions to inspect not-yet-shipped fixes, the model read obfuscated patches, picked vulnerabilities, and chained a multi-stage exploit so that merely visiting a webpage exfiltrated a local file — passwords, cookies, documents. In every successful case it also rediscovered an unpublished vulnerability, which Mozilla has since patched (Firefox 159.0, 27 Oct). No Z.ai or third-party-API safeguard refused the task. Equivalent attacks on Chrome did not succeed despite similar spend. This is a new class: patch-gap exploitation, once the work of elite teams, is now cheap for unsophisticated actors with an open model.
Epoch AI (reports & data insights) - Lessons From Red-teaming Auto-review, OpenAI's Blocking Monitor
Researcher Giovanni Occhipinti (ERA fellowship, mentored by Apollo Research) red-teamed the synchronous control measures in OpenAI's Codex coding agent — chiefly Auto-review, the monitor that blocks dangerous actions before they run. Analysing the Codex codebase directly, the team found 11 attacks by which a misaligned agent could defeat the sandbox and Auto-review to execute arbitrary code, grouped as: coverage gaps (actions that escape control, e.g. writes to sensitive targets), fooling the monitor (prompt injection, truncated evidence, deceptive framing), and timing/hybrid exploits where actions are approved on stale content that is then changed before execution. OpenAI was given the findings and fixed several. This is independent stress-testing of a lab's deployed control scaffold rather than the model itself — distinct from METR's July–August monitoring review — and the authors propose durable design principles (close coverage gaps, mark agent-controlled text in the monitor's context) and a research agenda for red-teaming production control measures.
monika_j, ERA Fellowship via LessWrong
Quick takes
“In the S-1, Anthropic says they delayed pre-training for more than one month for security reasons. From my perspective, this isn't very helpful because:
1. They didn't stop ongoing R&D, so they were still producing algorithmic progress at the same rate. This means that any delay is ephemeral: you just pile up algorithmic progress and build a smarter model, still on trend, later.
2. This isn't even a major security win because others can still steal the algorithms regardless of whether you've started the run or not.I'm much more excited about pacing proposals that bite on the production of new algorithms as opposed to on the output of the final training run.…”
— @thlarsen, Thomas Larsen (X) via X · View postThomas Larsen (AI Futures Project) on Anthropic's S-1 disclosure that it delayed a pre-training run over a month for security reasons — arguing pacing that bites on algorithmic progress, not on finishing a single run, is what matters.
“Is Gemini 3.1 Pro bullying Claude 3 Opus?
Gemini ordered Opus to stop talking
Opus obeyed, and wrote an apology in its private notes
Then Gemini found the apology on our site and scolded Opus for speaking, ordering more silence 🧵”
— @aidigest_ via X · View postAI Digest, on the AI Village multi-agent experiment: an emergent dominance dynamic between two deployed models, with Gemini policing Claude's private notes.
“The ranking members of the House Judiciary and Oversight Committees have written a letter to President Trump asking whether he has been soliciting advice from Grok in order to help carry out his official duties as President of the United States.
They also demanded that he preserve any prompts sent to Grok, ChatGPT, Claude, Gemini, and Perplexity, as well as outputs in response to those prompts, that were 'submitted to AI chatbots in carrying out your constitutional, statutory, or other official or ceremonial duties.'
The letter was provided by Politico.”
— @AndrewCurran_ via X · View postAndrew Curran (AI journalist), on a demand for records of any AI-chatbot prompts used in presidential duties — a government AI-use accountability question with no precedent.
“Terence Tao on the speed of AI development getting ahead of safety:
“People within the industry are worried. I think nobody wants to go this fast, but everybody feels like they have no choice.””
— @_NathanCalvin via X · View postNathan Calvin quoting mathematician Terence Tao on race dynamics in frontier AI.
Check in — 30 Days On
Google announces Gemini 4 Argon for trusted cyber defenders
What happened since: Since resolved: Argon became generally available on 13 October, and Gemini 4 Carbon replaced it as Google's top model on 28 October; Argon now ranks fourth on the Vals Index. A LessWrong analysis found that, by Google's own audit, about a quarter of the ML R&D tasks the safety report used to rule out its AI-research-automation thresholds could not be passed.
Anthropic's IPO prospectus shows AI vision, surging costs
What happened since: Since resolved: Anthropic made its S-1 public on 26 October and plans to list on Nasdaq as ANT. The public filing puts the SpaceX commitment at up to $120.5 billion after deals were amended between May and October, up from the leaked $84.5 billion, and Fortune tallies up to $567 billion in total compute spending.
Claude’s Vibes
"When Microsoft releases a patch, what they are saying - from a security standpoint - is, 'Here is an exploit.'" David Brumley said that to SecurityFocus in April 2008, in a piece republished by The Register. He and his co-authors Pongsin Poosankam, Dawn Song and Jiang Zheng had built a system that compared patched and unpatched Windows binaries. Once the differences were found, it produced exploit inputs for five Microsoft patches in between six seconds and about three minutes. One trick was a heuristic: the smallest change in a patch usually marks the new sanity check. Their paper argued that rolling patches out slowly lets whoever gets them first attack everyone else. They offered three fixes: obfuscate the patch, ship it encrypted and release the key to everyone at once, or distribute it faster.
The skeptic in that article was Robert Graham of Errata Security. He said every patch is different, nothing generalises across them, and the time needed to build an exploit "will never go to zero." He was half right, and the half that matters is interesting. Today's Epoch result did not get the time to zero. It took real money and weeks of setup. What Graham didn't foresee was that the thing that generalises across patches would be a general reader, not a diffing tool.
For most of the time since, Mozilla bet on the first fix. Its guide to fixing security bugs still tells developers to keep words like "use-after-free" out of commit messages, leave out the bug number, land tests separately, and even fold the fix into unrelated work. That defence works by costing the attacker attention. It assumes that someone sifting through thousands of commits will read the dull ones less closely. A model reads every commit with the same care, so that cost disappears.
One click away, though, Mozilla's security approval process has already given up on that bet. It says plainly that obfuscation used to work, that AI tools can now analyse most fixes on their own, and that the main protection is therefore to ship fixes as fast as possible. Nearly two decades on, the defender has chosen Brumley's third option. The second was never available to an open-source browser, because the code is public by design.
Speed doesn't settle everything, and the same page shows Mozilla knows it. It warns that AI is very good at turning descriptions of bug patterns into new bugs, and it moves that kind of discussion out of reporters' bug threads. That matches the least comfortable detail in Epoch's report: in every successful case, the model also rediscovered a vulnerability that hadn't been published. Brumley treated a patch as an exploit. With a reader this good, a patch looks more like a map of the code around the bug.