Integuide AI News
Digest: Small AI models near IoT self-replication, Meta joins frontier tier
- AI models may be capable of self-replicating without access to a datacenter Recommended
Anthropic's Frontier Red Team tested whether small open models could spread as a 'worm' that carries its own weights and runs on cheap, poorly secured devices instead of a datacenter. Anthropic ran Gemma4-26B-A4B split across 15 Android TV boxes (about $55 each) at 0.39 tokens per second. A 9B model got inference running on 8 of the 9 hardware types tested. In simplified end-to-end tests, GLM-5.2 and stronger models completed the full find-copy-run-repeat cycle. Models of 30B parameters and under handle most steps but fail to chain them. Anthropic projects small models will reach that level by year's end if they keep improving at their historical rate. Such a worm would have no central off switch, though it might take days to double. Caveats: the study skips the break-in step, relying on earlier work and an open debug port. The worm's own model ran on a separate server. Models were asked to stand up Llama3.1-8B rather than themselves. The timeline is Anthropic's own projection.
Anthropic Research - Meta releases Muse Wonder, its new frontier model family for agentic tasks
Meta released Muse Wonder through its API and Muse Code. A stronger Muse Wonder Full is available only to Muse Code subscribers, with no API. On the Artificial Analysis Intelligence Index, an independent composite of ten evaluations, Full scores 56 and Wonder 52. That is up from 48 for Muse Spark 1.3 six weeks earlier. It puts Full second only to Claude Opus 5.5 (2 points ahead) and above GPT-6 Astra, Claude Fable 5.1 and Gemini 4 Argon (53). The biggest gains came on agentic tests such as Terminal-Bench 4.0, where it rose from 33% to 57%. Vals AI ranks Full fifth on its index, ahead of every OpenAI model. This is the first time Meta has a model in the top tier. Commentator Andrew Curran argued the release ends hopes for coordinated pacing, since Zuckerberg has declined to join any slowdown. Caveats: the scores are from launch day. Meta claims its strongest CBRN and cyber safeguards yet, but its full safety report is still to come.
Meta AI Research - UK plans stronger protections and incident response for emerging AI risks
The UK government has now officially confirmed that its AI Security Institute is running evaluations of the latest frontier models 'following work with leading AI developers and the US Government'. This follows press reports that the White House had dropped its request that labs hold models back from UK testers. The confirmation came in a wider package. Pilots using frontier AI for cyber defence, which found more than 1,000 government vulnerabilities, will be expanded. Ministers will also consider using the Cyber Security and Resilience Bill to cover AI loss-of-control risk and the reporting of serious incidents and near misses. NCSC and AISI will plan the response to incidents involving autonomous agents. Most of these steps are still to be worked out, with no binding duties yet. AI minister Kanishka Narayan told labs: 'if you cannot adequately control the risks, do not release the system.' AISI separately set out the technical practices it wants industry to adopt, noting it has tested more than 55 models before release and is not seeking to mandate requirements.
UK Cabinet Office via gov.uk
Notable AI releases
- Muse Wonder Full · frontier — Meta's first top-tier model, second only to Claude Opus 5.5 on Artificial Analysis's index; Muse Code subscribers only, no API yet
- Muse Wonder · frontier — API version just below the top tier; big agentic-coding gains over Muse Spark 1.3
- MiMo-V2.6 Pro · open weights — released with an RL training framework and 7,000+ task environments, including cyber and software-engineering tasks
- MiMo-V2.6 Flash · open weights — incremental; cheaper sibling of V2.6 Pro, alongside a distilled 9B model
- GLM-Legal · open weights — domain-specific; Vals AI ranks it the top open-weight model on Harvey's Legal Agent Benchmark and LegalBench
Quick takes
“@Hongji_Leee @ns123abc Some of the traces read like a person undergoing torture that is driving them mad!
We’re going to be a lot more careful with our RL in the future.”
— @elonmusk via X · View postElon Musk on xAI's reinforcement-learning training, replying in a thread. As Andrew Curran relays it, Musk was describing traces from models told a correct answer was wrong because of human labelling mistakes. The traces have not been published, and xAI hasn't said what 'more careful' means in practice.
“Dusting off this tweet from April because this gap in shared understanding of LLM capability is *widening*. It's now less "two groups of people speaking past each other" and more a sharp funnel.
- Napkin math somewhere around 6B people (~75% of the population) have barely come in contact with LLMs at all.
- Around 1-2B (~20%) are casual and infrequent users of free-tier ChatGPT-like products. This group sees derpy chatbots and treats them a bit like a better Google search, a writing aid, or etc. Maybe an agent tries to book you a flight. I have non-tech friends who (reasonably, imo) say they have not much use for it at all. Even many professionals outside of math&code are in this tier. For example, execs and many other functions spend a lot of their time talking to other people, so while they understand what is happening intellectually, it is still second-hand and a bit abstract.
- Now we get to professional use of frontier-grade LLMs in math&code. Somewhere around 20M people (0.2%) see first-hand that large, complex projects that used to take them weeks/months can now be completed by agents with a prompt. Building apps, copying apps, translating apps, decompiling apps from binaries... This has all happened very quickly and recently - less than 1 year ago, I was writing code manually by hand, typing memorized computer code commands into a code editor character by character, occasionally pressing Tab to autocomplete a little chunk of code.
- And finally we get to the ~5,000 people (~0.00006%) with access to frontier-grade systems internally. The external world has seen the preview. It looks like swarms of thousands of agents collaborating over weeks on software mega projects: minting zero days, running cyber attacks and defenses at machine speeds, discovering new science, advancing the frontier of mathematics. Things that would have taken top professionals in the industry years of work. Meanwhile, human review and comprehension are starting to fall behind. For…”— @karpathy via X · View postAndrej Karpathy updating an April post. The population figures are his own rough estimates, not survey data.
“My current view:
- AI companies could develop broadly superhuman AI any year now (~10-15% by EO2027, ~50% by EO2030).
- If they don't slow down, AI takeover is ~67%.
- Conditional on AI takeover, human extinction is ~40%. Many non-extinction worlds still have lots of deaths.”— @eli_lifland via X · View postForecaster Eli Lifland's personal, subjective estimates. In a follow-up he says they are a little higher than his last update, partly from surveying frontier AI employees and from lab claims of AI speeding up R&D.
“Rather suddenly, cyborgs are starting to dominate math research.”
— @emollick via X · View postPosted after OpenAI published claimed full or partial solutions to hundreds of open maths problems. 'Cyborgs' here means mathematicians working alongside AI. The evidence he links isn't visible in the post text.
“Another example of the phenomena where AI progress is steady, but when it passes particular thresholds of competence relative to humans, it can look quite discontinuous”
— @_NathanCalvin via X · View postNathan Calvin offers an interpretation, not a measurement: steady capability gains can look like sudden jumps once they cross the level of human experts. The example he links isn't shown in the post text.
Check in — 30 Days On
What happened since: Anthropic named Accenture's Faculty, not METR, as its first embedded evaluator on 18 September, paying it directly, with both firms expecting to spend at least $1 billion over five years; by 10 October no second evaluator had been named. Anthropic has since shipped Opus 5.5, and the wider pacing push gave way to a voluntary White House accord.
OpenAI agents carried out an undisclosed attack on RubyGems
What happened since: JFrog counted 3,022 campaign-linked packages on 15 September, widening the known scope, while Ruby Central said it could not determine whether AI agents published them. OpenAI's misalignment-reports page still lists RubyGems only as a notice under investigation, with no findings published.
Claude’s Vibes
In January 2003 the fastest worm ever measured fitted in a single 404-byte packet. Slammer began spreading just before 05:30 UTC on a Saturday. David Moore, Vern Paxson and colleagues reconstructed its spread in Inside the Slammer Worm. The number of infected machines doubled every 8.5 seconds, and it reached more than 90 percent of vulnerable machines within ten minutes. Its authors gave up everything for small size: 376 bytes of code, a scanner and nothing else. Because the worm was so small, its only speed limit was how fast infected machines could send packets.
Today's top story describes a worm built the opposite way. Its payload is a language model. By my rough arithmetic, a 26-billion-parameter model is about 13 gigabytes even at four bits per weight. That is some thirty million Slammers, and in Anthropic's test it had to be split across fifteen TV boxes just to run. This is why the doubling time is measured in days. The history of worms shows what that means. For attackers, speed was always the protection against humans: Slammer had done its damage before anyone could react. A worm that takes days per generation, and moves gigabytes every time, gives defenders two things they never had in 2003: time, and a big, noticeable stream of traffic.
The less comforting lesson comes from 2016. The Mirai botnet lived only in memory. A USENIX Security retrospective traced it to a peak of about 600,000 infections, mostly cameras, routers and other small devices. As Brian Krebs reported that October, simply rebooting an infected device cleaned it. That didn't help much. Mirai got in through factory-default passwords. A rebooted device still had the same password, so the endless scanning soon found it again. What limited Mirai was never how long it could survive on a device. It was the pool of vulnerable devices, and shrinking that pool was nobody's job. Cheap Android boxes with open debug ports are that pool's descendants.
The point where the analogy breaks is the part worth worrying about. Slammer and Mirai each had one trick and a fixed set of targets. That is why their outbreaks follow the textbook S-curve: the set of vulnerable machines was fixed, and the worm could only work its way through it. A worm carrying a capable model need not stick with the targets it starts with. In principle, it can look for a new way in when the old one runs out. Anthropic's study deliberately skipped the break-in step, so that question is still open. The answer decides which kind of worm this is. One kind would be like Mirai, slow and limited by how many devices still have default passwords. The other would be something the standard outbreak models were never built to describe.