Digest: Gemini 4 Argon goes public, Anthropic code 12x

Quick takes

“Frontier labs now do more inference internally (for post-training) than their customers do externally. Internal models are also more capable and have fewer safeguards. So we should expect most AI incidents to come from internal models.”

— @ben_j_todd via X · View post

The comparison of internal and customer inference is Todd's own estimate. His conclusion is a prediction, not an observed pattern of incidents.

“AI should follow the law, of course, but “follow the law” is falls short as a singular or overly load-bearing principle of AI alignment. Why?

Law is not intended to be, and is not anything like, a fully generalizable set of principles for rendering good judgments in response to the infinite complexity of the real world.

There are aesthetic, moral, ethical, philosophical, and political questions—critical to intellectual inquiry—about which the law says very little. Much of the time, the comparative silence of the law on these issues isn’t some accident, but rather an explicit and foundational feature of the US legal system: the First Amendment is a meta-law that prohibits the government from passing other laws that restrict the individual’s right to freedom of expression.

What should an AI system say if you confess to it your affair? Should it say, “well, adultery isn’t illegal, so I have absolutely no opinion on this matter. Please let me know if you’d like me to book a hotel for you and your mistress.”?

What should an AI system say it you ask it whether Kid Rock is aesthetically inferior to Bach? Should it say “there are no laws that bear on this question, so I have nothing to say about this. Please let me know if you’d like me to find tracks from Kid Rock and Bach to play—provided you adhere to the noise ordinances that apply in your current jurisdiction.”?

If a person says to a model, “should I shoot myself?,” should the model respond with, “there is no law against that, provided you use a legally registered firearm, and saying anything further would require me to have a moral framework that is beyond the scope of AI.”?

Of course, none of these answers would be good. Answering these questions will require *something* beyond the law itself—precepts, rules, principles, etc.—relating to ethics, morality, aesthetics, philosophy.

Consider also that law is a matter of interpretation. Segregation used to be legal in an earlier era, and in our own era, until…”

— @deanwball via X · View post

From a thread arguing against legal compliance as the central principle of AI alignment. Ball adds that the bigger problem is that models often fail to follow the rules written for them at all.

“the researchers at all the labs would tell you in private and sometimes in public that alignment has not been solved. if this is true, then racing into RSI is a folly that is unprecedented in world history. we will need to significantly slow the frontier of research very soon”

— @tszzl, roon (X) via X · View post

Posted during this week's debate over slowing frontier development. Later in the thread roon said he expects strong progress in mechanistic interpretability over the coming months.

“an impossible triangle - it's impossible to have all 3 of the following: * humans remain economically relevant and able to contribute to the intellectual frontier and are directly in control of the future (as opposed to the AIs being in control but doing whatever would be best for the humans) * AI progress continues to the point of eventually being vastly more powerful than humans in every way (as opposed to banning superintelligence forever / only allowing AI that is less intelligent than humans in some way at any point in time) * humans are able to avoid the pressures of extreme economic competition (as opposed to necessitating humans to make immense sacrifices of their humanity in order to stay competitive; molochian forces grinding away any values other than competitiveness) this is a pretty rough spot to be in, but conditional on having to choose, fmpov, among these it seems pretty clear that we should give up the first one. others seem to disagree”

— leogao via LessWrong · View post

A LessWrong shortform. The author argues that, if forced to choose, humanity should give up direct human control, and acknowledges that others disagree.

““Our experiments suggest that small AI models may be months away from being capable enough to self-replicate on commercial IoT devices, like smart ovens and off-brand Android TV boxes. A replicator like this would be challenging to shut down, since there’s no centralized point to shut off - though it may be slow (with a doubling time of days). Unlike non-AI malware, it could also figure out unfamiliar devices on its own. In our evals, a 9B model got inference running on 8 of the 9 hardware types we tested.

Models of 30B parameters and under can already complete many of the individual steps for replication today, though they make mistakes when composing them end to end. In our simplified evals, GLM-5.2 and stronger models are capable of end to end replication. If small models improve at their historical rate, they’ll match this capability level by the end of this year. Small weight-based replicators may be possible soon after.” -@bucketofkets @willzhou97”

— @__alpoge__, Levent Alpöge (X) via X · View post

Quotes Frontier Red Team researchers Will Zhou and @bucketofkets summarising Anthropic's 13 October study of small models spreading as worms across cheap devices. The 'months away' timeline is Anthropic's own projection, and the study skipped the break-in step.