7 Oct 2026

Digest: AI research-taste benchmark, weight-grafting alignment training

1. TasteVal: Measuring the Experimental Research Taste of AI Systems Against Human Experts

   TasteVal, a new benchmark from Ollie J and Dane Sherburn, tries to measure "experimental research taste": how well a model designs AI R&D experiments and interprets their results. Across eight tasks, from pretraining data curation to preference modelling, the model under test only plans and interprets. A fixed coding agent runs each experiment on one H100 within a 40 GPU-hour budget. Taste is scored as compute efficiency against the best of 24 human experts. The authors report that frontier models' taste has doubled every 3.0 months since December 2025 (95% CI 1.7–5.0). Opus 5.5 matches the expert score with 2.3x less experiment compute (CI 1.15–4.37x), at about 1/30 the cost. That cuts against METR's pre-release finding last month that Opus 5.5 still lacked the taste to automate AI research. Caveats the authors flag: the tasks are quick to verify and easy for labs to hill-climb, the baseline excludes top researchers, and choosing which problems to work on isn't measured. This is a new, unreplicated benchmark (paper).

   Ollie J via LessWrong
2. You can graft SDF changes from base models onto post-trained models

   MATS fellows Dani Roytburg and Peter Nutter, with Clément Dumas and mentor Shi Feng, propose "grafting": run synthetic-document fine-tuning (training on fabricated, pretraining-style text to implant beliefs or values) on the base pretrained checkpoint, then add the weight change to the finished post-trained model. Across five model families up to 284B parameters, grafts did less damage to coherence and grip on reality than tuning the chat model directly. One graft worked across SFT, DPO and RL checkpoints, but only if trained from base. Replicating a constitutional mid-training pipeline, grafted alignment training held up better through later SFT and RL without capability loss; emergent misalignment transferred too. Dumas calls it another update against the persona selection model; Feng's thread summarises. Caveats: installation is weaker, the post-training comparison used ~200k examples on a small model, and robustness to misaligning RL is untested.

   LessWrong
3. Identifying Introspection From the Inside

   David Atkinson and David Bau (Northeastern) and Dillon Plunkett (Eleos AI Research) propose a mechanistic test for whether a model's self-reports come from the computation they describe. The paper is accepted at COLM 2026. They fine-tuned Qwen3-32B to make choices for 100 fictitious characters, each with a hidden five-attribute preference function. Accurate self-reports of those preferences appeared late in training without any self-report training: faithfulness rose from 0.25 to 0.83 between steps 1,000 and 3,000, while decision accuracy barely moved. Of the Qwen3 sizes tested, only 32B showed this. Layer ablations suggest faithful models store the preferences in earlier layers, where the model's verbalisation machinery can reach them. Across 32 model pairs, attribution-patching similarity between the decision and self-report tasks told faithful models from unfaithful ones without reading the reports. Caveats: it is a narrow linear toy setting, and the measure separates groups rather than individual models. Bau's thread walks through the setup.

   iii.baulab.info
4. The Essay That Started the AI Race

   Kevin Roose has published the first 10 of more than 25 pages of "The Big Blob of Compute Hypothesis" (PDF). Dario Amodei, now Anthropic's CEO, wrote the internal memo at OpenAI in 2017. It argues that intelligent behaviour comes mostly from "a large, minimally structured mass of computational capacity" shaped by training and a rich environment, so researchers should scale general models rather than engineer narrow ones. Early OpenAI staff told Roose it pushed the lab toward the scaling that produced GPT-2 and GPT-3. Per Roose, the memo applies the same logic to safety: it argues against MIRI-style engineered safety protocols and for getting high-level training conditions right. The unpublished remainder is mostly Amodei disputing other safety researchers' approaches. Primary documents showing how a frontier-lab leader's views on scaling and safety formed are rare. Amodei declined to comment, and the release promotes Roose's new book.

   kevinroose.substack.com