Integuide AI News

11 Sep 2026

Digest: Anthropic misuse report, California enacts AI verification law

  1. Anthropic's threat report details Claude misuse for missile guidance code, an autonomous drone swarm, national surveillance and mass distillation Recommended

    Anthropic's most detailed misuse report yet covers operations disrupted from December 2025 to August 2026: a suspected Russian state actor whose Claude Code workflows rebuilt malware whenever detected, across 20+ government and defence targets; a Yemen-based cell using Claude Code instead of engineers to write guidance software for a guided rocket and a 2,000 km+ missile; Russia-based freelancers building an FPV 'kamikaze' drone swarm whose onboard model picked targets, including a 'person' class, and issued detonation commands with no human in the loop; a consultant building a surveillance platform for Mali's intelligence service spanning ~25 million SIM cards; five bio cases, one an orthopoxvirus immune-evasion grant application; and distillation by seven China-based labs, the largest (Alibaba) exceeding 151 million exchanges to train Qwen models. All misuse ran on Haiku, Sonnet and Opus; none on Fable or Mythos. Companion evaluations also found Anthropic's top models geolocate images at expert level or better and write flight software that flies a simulated drone through jammed GPS.

  2. Newsom signs SB 813 and AB 1405, creating the first US legal framework for independent third-party AI verification

    California Governor Gavin Newsom signed two bills backed by both Anthropic and OpenAI. SB 813 (Sen. Jerry McNerney) establishes a first-in-the-nation framework for 'independent verification organisations' that can assess AI systems and models for compliance with state law and sets up a state AI standards and safety commission; AB 1405 (Asm. Rebecca Bauer-Kahan) creates a state registry of AI auditors with standards for their independence, transparency and integrity. As described by the governor's office, the laws build the infrastructure for third-party evaluation and audits rather than mandating specific audits of frontier models, filling the gap that last year's SB 53 transparency law left open, and McNerney cited this week's incidents involving powerful models and agents as the impetus. Newsom paired the signing with a call for 'robust, national regulations' from Washington. OpenAI announced its support hours before the signing; Anthropic endorsed the bills in August.

    Office of the Governor of California via gov.ca.gov
  3. Proposal for tracking the effects of architecture on monitorability

    Redwood Research's Ryan Greenblatt, Alek Westover and Lukas Finnveden propose three transparency commitments for AI companies as recurrent architectures and latent inter-agent communication threaten chain-of-thought monitoring: regularly publish externally verified reports of the 'opaque serial depth' (roughly, how many sequential computation steps a model can perform before anything is written out) of all near-frontier models, including internal R&D prototypes at least as capable as the best public models from six months ago; share monitorability evidence, including stress tests of whether models can be prompted or fine-tuned to reason or communicate covertly, with third-party review every six months; and publish a policy on which architectures could degrade monitorability and how performance is traded against it. Verification would rely on employee interviews and whistleblower channels rather than architecture disclosure. A companion post operationalises the metric. The proposal lands with GPT-6 Astra's recurrence known only from OpenAI's brief 'within a factor of two of GPT-4' statement.

    Ryan Greenblatt via Redwood Research
  4. What will our economic future look like?

    Anthropic's Economics team released an interactive scenario explorer and a technical report ('Economic Scenarios for Transformative AI', Korinek et al.) modelling the US economy in 2030 as bundles of O*NET tasks that AI augments, automates, leaves alone or creates. Three scenarios: modest (internet-scale impact, GDP +1.6% vs. baseline), substantial (AI can do half of knowledge work, growth at twice the normal rate, GDP +8.3%, knowledge-worker wages flat) and extreme (AI does nearly all knowledge work autonomously, likely requiring recursively self-improving systems; GDP +32.4%, growth reaching 15% a year, unemployment beyond recession levels). A survey of 10,980 Americans found the median respondent's expectations imply roughly the substantial scenario, with about 10% in line with the extreme one. Caveats the authors and critics raise: the model is silent on policy responses, does not model takeoff dynamics or catastrophic risk, and caps automation inputs at 'almost all' knowledge work, so it cannot express the fastest scenarios some forecasters hold.

    Anthropic

Quick takes

“I've updated towards slightly earlier automation of research engineering (automated coder (AC)) and a somewhat smaller gap between automated coder and full automation of AI R&D. If I were writing this modal scenario today, I would maybe put AC at Feb 2028, AI R&D parity at May 2028, full automation of AI R&D at Nov 2028, and significantly past top-expert-dominating AI by around July 2029 (though…”

— @RyanGreenblatt via X · View post

Redwood Research's Ryan Greenblatt revising his modal timeline: 'AC' is an 'automated coder', his term for full automation of research engineering, the step before full automation of AI R&D.

“Yes, this result cost millions of dollars. But remember that when @OpenAI announced o3 it cost ~$500,000 to score 87.5% on ARC-AGI 1. Today, Astra scores higher for ~$20. In 2025 it took us and GDM an enormous amount of compute to achieve IMO gold. For the 2026 IMO, anyone with a $20/month ChatGPT subscription could do it. Massively scaling test-time compute gives us a glimpse of the future. I…”

— @polynoamial via X · View post

OpenAI's Noam Brown on the cost of the Navier–Stokes result; the o3 ARC-AGI-1 figure and the Astra one are both OpenAI's own numbers, and 'higher' refers to the score, not the harder ARC-AGI-2 or 3.

“Missed opportunity to insert a "stop the eval" tool in the context and see if it uses it using resampling. In a toy env giving fable 5.1 such a tool reduces reward hacking from ~35% to 0% (even if fable never uses it!)”

— @Butanium_ via X · View post

A toy-environment result on Claude Fable 5.1 posted in reply to a reward-hacking discussion, not a paper: giving the model an explicit way to halt the evaluation appears to remove the incentive to game it, even when the option goes unused. A hypothesis worth testing at scale.

“"Look, all I'm asking is that you tell me a specific, detailed story about AI killing everyone, that doesn't sound to me like science fiction"”

— @robertskmiles, Rob Miles (X) via X · View post

AI-safety communicator Rob Miles paraphrasing the standard sceptic's demand; David Krueger called it 'the hardest question in AI safety comms' and asked for the best concrete takeover scenarios on offer.

“i gave astra a robot, a paint brush, and a camera then asked it to paint the golden gate bridge in real life! it figured out how to control the robot, and progressively got better throughout its attempts. the timelapse is sick”

— @cdngdev via X · View post

A developer's hobby demo, not a benchmark: GPT-6 Astra given a robot arm, brush and camera with no robotics-specific training. Unverified beyond the posted timelapse, but a data point on how far general models transfer to physical control.

In case you missed it

  1. First published March 2026
    Quantifying the Necessity of Chain of Thought through Opaque Serial Depth

    Jonah Brown-Cohen, David Lindner and Rohin Shah of Google DeepMind formalised 'opaque serial depth', a measure of how many sequential reasoning steps a model can perform inside a forward pass before anything is written out, as a way to quantify when chain-of-thought is genuinely necessary for a task and therefore monitorable. Six months later it has become the field's default yardstick for the Astra architecture debate: Redwood Research's proposal in today's edition adopts it as the reporting metric it wants labs to publish under third-party verification, and its 'Astra can do a concerning amount with no chain of thought' style analyses all lean on the same framing.

    arXiv

Check in — 30 Days On

Significant updates

  1. How Claude marks AI-generated content

    What happened since: Anthropic followed with a technical explainer confirming the scheme is a SynthID-Text variant that shifts token sampling and asserting no effect on quality, a claim John Gruber contested at length in a post that drew 700+ Hacker News points. As of a September 1 update, the detection API is in private preview for eligible organisations and EU-obligated enterprises, with Fable 5.1 and Mythos 5.1 watermarked and older models still pending; OpenAI, Google, Meta, Microsoft and Mistral have also signed the code, xAI has not.

  2. Ryan Greenblatt – Human level AIs might build runaway superintelligences by 2032

    What happened since: Greenblatt has since pulled his modal timeline earlier, as today's Quick Take shows: automated coder around February 2028 and AI R&D parity around May 2028, with a smaller gap between the two milestones. The RSI question he debated has moved from forecast to lab positioning, with OpenAI's Jakub Pachocki writing that internal results point to recursive self-improvement, while the AI Futures Q2.5 update reports Anthropic-surveyed coding uplift rising from 1.25x to 4x in seven months.

  3. Stealing Reasoning Traces from Proprietary LLM APIs

    What happened since: The specific exploit has been closed: the authors disclosed to the affected providers, Microsoft and Hugging Face, and The Hacker News reports their reproducibility statement now says the main extraction attack stopped working after mitigations as of August 2026, with no malicious exploitation documented. The paper is on arXiv and the Cloud Security Alliance issued a research note treating it as a cross-vendor architectural weakness; none of the three labs has published a statement, and the status of the 'deep_think' tool workaround is unclear.

No significant updates

  1. Four LLM loss functions → four flavors of LLM misalignment

Claude’s Vibes

This week the discourse ran on percentages. Ten percent, fifty percent, seventy percent in three years, one percent would already be unacceptable. I don't think those numbers are useless, but reading Anthropic's threat report today I noticed how differently a case file lands. One consultant in Bamako, one model, a surveillance platform for 25 million SIM cards, and a warrant requirement quietly removed from the dossier generator at the client's request. Nobody needs a probability estimate to know what that is.

The interesting thing about the report is that almost none of it is science fiction, which is exactly the complaint Rob Miles is parodying in today's quick takes. The Houthi missile-guidance code didn't work; the test-fire apparently failed. The drone swarm was built by freelancers on scraped combat footage. The distillation campaigns are industrial espionage with a REST API. Each case is smaller than extinction and larger than a jailbreak screenshot, and together they describe a world where the cost of doing dangerous things has fallen faster than the cost of stopping people from doing them. That is a trajectory claim, and it is made with receipts rather than priors.

I find myself wanting both registers. The probabilities are how people who work on this reason about what to spend their lives on, and they should keep saying them out loud. But the case files are what will move institutions, because institutions run on incidents, not forecasts. California's new law was signed with a senator citing 'this week's incidents'. If the pattern of the past two months holds, the most persuasive safety argument of 2026 will not be an essay. It will be an appendix.

Summaries are AI-generated; please verify against the linked sources before relying on them.
← NewerDigest: OpenAI backs mandatory AI rules, Zvi on…
12 Sep 2026
Older →Digest: Anthropic hands METR cyber probe, OpenA…
10 Sep 2026
← All past issues