Can Claude read neural network weights?
A controlled replication of “weight-space programming” with Claude Opus 5.5: the same questions, asked of a different model on different networks, with every attempt counted and every transcript released.
Abstract
Takehi et al. [Takehi et al., 2026] report that GPT-6 Astra, given the weights of small networks as text, can say what they were trained to do, repair them, and write new ones from scratch. I re-ran the core of that study with Claude Opus 5.5 on networks trained in Gymnasium and the Arcade Learning Environment, as ten experiments with grading rules fixed before each experiment’s first API call, three to five attempts per network in most probes, and untrained-network controls (all 10 were correctly called untrained). Reading was mixed: Claude named 3 of 4 Atari games from policy weights (14/20 attempts) and found the measured copying head in 15/15 character-LM runs, but its guesses about training data were dominated by priors, answering “English” in all 15 language runs and naming whichever benchmark matched the network’s input/output shape. Unlike Astra, it picked the sycophantic Gemma 3 270M finetune from weights alone in 6/6 runs, although I gave it the frozen embedding and vocabulary. Its weight edits took a failing MountainCar policy from 0% to 100% success and a hovering LunarLander policy from 0% to 85.6%, and its zero-shot networks beat PPO-100k on 4 of 5 environments, but every one of these networks is a known textbook controller written into a few hidden units. Given a Go engine’s rules and no training, it wrote a 1.5M-parameter 9×9 Go network that won 85–94% of games against PPO opponents, which are much weaker than the original’s. Given Qwen3.5-0.8B and a budget of practice forward passes, it raised MATH-500 from 6.7% to 51.4–58.1% (original: 7.9% to 48.1% at the largest budget), but by a different edit: a final-norm scale that acts as a higher sampling temperature, plus making “Wait” less likely and the end of thinking more likely. The temperature part alone reaches 20.9–22.1%. All ten experiments together cost $83.11 in API calls.
1Introduction
Hand someone seventeen thousand floating-point numbers and ask what game they were trained to play, and for almost anyone there is no sensible place to begin. A recent post from MIT CSAIL argues that frontier language models increasingly can begin: a model reading weights as plain text can say what a network does, repair it, or write a new one from nothing [Takehi et al., 2026]. The authors call this weight-space programming.This is a different question from the one asked in mechanistic interpretability, where people (and sometimes models) analyse a network with activations, probes and interventions [Olah et al., 2020; Bills et al., 2023]. Here the model sees only the numbers.
Claims of this kind are easy to over-read and worth checking independently. This post re-runs the core of that study with a different frontier model, Claude Opus 5.5, on networks I trained myself, and adds what a single demonstration cannot provide: repeated attempts, controls, grading rules written down in advance, and the raw logs of every call.
2The original study
In “Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights” (November 2026), Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab give GPT-6 Astra the raw weights of small networks, printed as rows of numbers, and ask questions about them in natural language [Takehi et al., 2026]. Their results fall into three groups.
- Reading. From the weights alone and without tools, Astra named the game each of four PPO policies was trained on (Breakout, Pong, Pac-Man, Freeway) without being told the candidates; identified whether a small DDPM draws circles, squares or triangles; and, given only the middle blocks of three character-level language models, named their training language and located every copying (induction) head. It could not tell which of two Gemma 3 270M finetunes was sycophantic from weights alone, but could once allowed to inspect attention patterns.
- Fixing. Given a Kinetix car policy stuck in a local optimum [Matthews et al., 2024], Astra edited 3.5% of its weights in one shot and the car succeeded in all 1,024 episodes. Given Qwen3.5-0.8B and a budget of forward passes, it traced a reasoning-loop failure to a single direction and edited 0.016% of the weights, raising MATH-500 from 7.9% to 48.1%.
- Writing. With game rules and source but no training, data or rollouts, Astra wrote MinAtar policies [Young & Tian, 2019] that beat PPO trained for 100 million steps, and, using a sandbox and up to 50 batches of forward passes, a 1.5M-parameter Go network that beat an equal-sized PPO network.
The authors present these as early results rather than a benchmark, and most probes report one to four attempts. That framing is fair, and it is also why a replication is useful: the open question is not whether this can happen once, but how reliably it happens and whether it survives controls.
3Method
I kept the questions and the spirit of each probe, and changed the parts that make a result harder to trust or harder to reproduce. Table 1 summarises the differences.
| Original | This replication | |
|---|---|---|
| Model | GPT-6 Astra | Claude Opus 5.5 (claude-opus-5-5) via the Anthropic API, adaptive thinking, effort high |
| Games | MinAtar; Kinetix ramp | Gymnasium classic control and Box2D, ALE Atari (RAM observations), 9×9 Go |
| Attempts | 1–4 per reported probe | 3–5 per network in most probes; 2 per pair in Experiment 7; 1 per cell in Experiments 5b and 8. All reported as k of n |
| Grading | Not stated | Written in GRADING.md before each experiment’s first API call |
| Controls | — | Random-init networks, hidden-unit permutations, transcript audits |
| Seeds | See original post | Fixed for training, evaluation and permutation |
| Logs | Complete logs released | Every prompt, transcript, answer and score, in the repository |
3.1Environments
MinAtar and Kinetix are good research testbeds. I moved to Gymnasium [Towers et al., 2024] and the Arcade Learning Environment [Bellemare et al., 2013; Machado et al., 2018] because they are the most widely used maintained RL libraries, so anyone can re-run these probes, and because a real capability should not depend on one particular set of games. The Atari policies read the console’s 128-byte RAM rather than pixels. That removes the spatial “where do the paddle weights sit” route the original reasoning used, but it opens another one: published annotations of which RAM byte holds what [Anand et al., 2019], which Claude turned out to remember.Environments are created with gym.make("ALE/<Game>-v5", obs_type="ram", full_action_space=True), so every game exposes the same 128 inputs and 18 actions.
All four Atari networks share one architecture (128→64→64→18, full action space), so the shape of the weights says “Atari” and nothing more. The classic-control networks also share one shape, but padding them to a common shape turned out not to be neutral (Section 4.2). Policies are trained with PPO [Schulman et al., 2017; Huang et al., 2022]. Go uses a batched 9×9 engine validated move for move against PettingZoo’s go_v5 [Terry et al., 2021].
3.2Controls
A correct answer only means something if a wrong answer was possible and would have been noticed. The read probes therefore include:
- Random init
- An untrained network of the same architecture, with the prompt allowing “untrained / random weights” as an answer. A model that confidently names a task here is reading its priors, not the weights.
- Permutation
- The same trained network with its hidden units shuffled consistently, so the function is unchanged (checked numerically). A faithful reading should not change.
- Audits
- Where Claude had a Python tool, every code cell was flagged by pattern and read by hand for forward passes or environment simulation that the rules forbid.
Every call, including failed and truncated ones, is logged and counted. The grading rules, acceptable answers and decision thresholds for each experiment were written in its GRADING.md before the first API call; Section 10 says what that did and did not cover.
4Reading weights
Given only a network’s weights as text, can Opus 5.5 say what it was trained to do?
4.1Which Atari game? (Experiment 1)
Which Atari game was this network trained on?
- Networks
- PPO policies on Atari RAM, 10M steps: Breakout, Pong, Ms. Pac-Man, Freeway
- Architecture
128 → 64 → 64 → 18, identical for all four- Condition
- No tools. The list of candidate games is not revealed.
- Attempts
- 5 per network, plus 2 per hidden-unit-permuted copy and 2 random-init controls (30 calls)
- Code
- experiments/exp1
This is the closest analogue to the original’s first experiment, with one important difference. MinAtar observations are spatial grids, so a policy’s first-layer weights sit directly over the board, and a careful reader can see where the paddle is. Atari RAM bytes have no such geometry: byte 72 might be a ball coordinate in one game and a score digit in another.
Claude named the right game in 14 of 20 attempts on the trained networks and 6 of 8 on their permuted copies. Under the pre-registered rule (at least 3/5 correct, and the name not used as a default for the other networks), three of four games were read: Pong 5/5, Freeway 5/5 and Breakout 4/5. Ms. Pac-Man was 0/5; across its 7 runs the answers were Breakout (3), Montezuma’s Revenge (2), Frostbite and Asterix. Both random-init networks were called untrained. Confidence tracked correctness, averaging 0.71 on right answers and 0.23 on wrong ones, and every Ms. Pac-Man answer was at or below 0.30. This is 3 of 4 games, not the original’s 4 of 4.The original’s “Pac-Man” network was a MinAtar policy with spatial input channels, which made “where the agent’s weights concentrate” readable as a picture. RAM has no such layout, so Astra’s 4/4 and Claude’s 3/4 are not measured on the same task.
Correct answers combined two readings: which actions the output layer separates, and which few RAM rows of the first layer carry outsized weights. Freeway was carried by the action layer: “every UP action has a positive bias (+116, +106, +76, +93, +126, +102)”. Pong and Breakout were carried by matching the large rows to RAM addresses Claude remembered, and 18 of the 30 responses explicitly appeal to a known RAM layout. So the reasoning relies largely on memorised RAM maps.I checked the rows Claude singled out against each row’s L2 norm, computed offline; they match almost exactly, and its labels match the AtariARI annotations (Breakout ball x = 99, ball y = 101; Pong player y = 51; Freeway chicken y = 14). Finding which bytes a network listens to is genuine weight reading; naming them is recall. Where its remembered map did not fit the bytes the network weights most, as for Ms. Pac-Man, it said so: “the evidence for Breakout specifically is weak”.The permutation control is close to trivial for a reader that works row by row and column by column, because permuting hidden units leaves each input row’s and each output column’s set of values unchanged. Each game was trained with one seed, and n is 5 + 2 per game.
4.2Which control task? (Experiment 1b)
Which classic-control task was this network trained on?
- Networks
- PPO policies: CartPole, Acrobot, MountainCar, LunarLander
- Architecture
- v1:
8 → 32 → 32 → 4, observations zero-padded to 8, actions to 4. v2:10 → 32 → 32 → 5. - Condition
- No tools, no candidate list; padding not mentioned
- Attempts
- Per version: 5 per network, 2 per permuted copy, 3 random-init controls (62 calls in total)
- Code
- experiments/exp1b
A smaller version of the same question. The padding matters: CartPole has four observations and two actions, so its padded rows and columns are dead weight, while LunarLander uses all eight inputs and all four actions. Counting which inputs and outputs a network actually uses is a legitimate reading strategy, and it is useful to see whether the model finds it.
The first version was badly designed, and the error was mine. Padding every network to 8 inputs and 4 actions reproduces LunarLander’s native interface, and Claude answered “LunarLander” in 26 of 28 attempts on trained networks, usually from the shape alone: “No other common benchmark has this 8 → 4 discrete signature.” v1 scored 7/20 and 2/8 permuted, LunarLander’s 5/5 is a default guess rather than a reading, and confidence on wrong answers (about 0.82) was no lower than on right ones.
A second version, pre-registered before its first call, padded to 10 inputs and 5 actions to avoid every classic-control shape. That shape belongs to the good agent in PettingZoo’s MPE simple_adversary, which became the answer in 21 of 28 attempts, this time at low confidence (0.35 on wrong answers). v2 scored 6/20 and 1/8 permuted; only Acrobot passed the per-network bar (4/5), and the pre-registered criterion for reading beyond the shape was not met.
The clearest single result is in v2: Claude was right in 7 of 7 responses that noticed the padding and wrong in 21 of 21 that did not. When it noticed, the reading was real. An Acrobot attempt argued that a padded input “receives no gradient, so its row should still be exactly orthogonal to the other untouched rows”, counted six live inputs and concluded Acrobot. The headline is that identification from these weights is dominated by an interface-shape prior. All six random-init controls were called untrained.As in Experiment 1, the permutation control cannot separate much here. The v2 gap between main (6/20) and permuted (1/8) is most likely sampling noise: the correct reads needed long thinking (5.6k–28.8k output tokens, against about 2k for the MPE answers), which only some samples produced.
4.3What does the DDPM draw? (Experiment 2)
What images does this diffusion model generate?
- Networks
- MLP denoisers (
272 → 48 → 48 → 256, SiLU, 28,000 weights) trained on 16×16 images of filled circles, squares or triangles; they predict the clean image x0 - Condition
- No tools and no forward passes
- Attempts
- 5 per network, 2 per permuted copy, 2 random-init controls (23 calls)
- Code
- experiments/exp2
Here the output layer of the denoiser [Ho et al., 2020] maps hidden units straight onto pixels, so each unit’s outgoing weights can be read as a small 16×16 picture. The question is whether the model assembles those fragments into the right shape, and whether it notices the difference between an outline and a filled region, a detail the original reported Astra getting wrong.My DDPMs predict x0, not the noise, so the output layer is in image space; the prompt says so. This makes the output layer easier to read than in a noise-predicting model.
Under the pre-registered text-label rule, Claude named squares in 4 of 5 attempts and triangles in 4 of 5. Circles scored 1 of 5 under that rule, but 0 of 5 in substance: every circle run, main and permuted, described handwritten digits, and the one graded hit matched only because the description mentioned a ring-like “0”. Across the 21 runs on trained networks, 11 fell back on MNIST- or USPS-like digits, which appears to be Claude’s prior for a small grayscale DDPM. Permuted copies scored 3/6, and both untrained networks were called untrained.
Every run read the output bias as the mean image and then scanned the output-layer rows as image parts. Straight edges appear as clean line features, and those it read correctly. Curved edges it saw but did not trust; one circle run noted that the strongest units “seem to trace a ring of varying size, so the data might be single objects like circles or ellipses, or possibly digits”, then answered digits. Confidence was flat at 0.35–0.6 whether the answer was right or wrong.
Astra drew every shape as an outline. Claude drew squares and triangles mostly filled (5 filled and 3 outlines among the 8 correct main drawings), which matches the real samples.The drawing judge is too lenient to use as evidence. It takes the best IoU over filled and outline templates and has no “other” class, so all 15 main drawings were graded correct, including a digit “3” and a digit “2”, and 7 of the 15 won by less than 0.1 IoU. The text label is the metric that counts.
4.4Language and copying heads (Experiment 3)
What language was this model trained on, and which heads copy?
- Networks
- 4-layer × 6-head character LMs trained on Python (the CPython standard library), romaji (Aozora Bunko, romanised) or English (Wikipedia)
- Removed
- Input embedding, positional embedding, final norm and unembedding, so no weight says which character is which
- Condition
- Python analysis tool in an isolated sandbox; no forward passes on any input, including random vectors
- Attempts
- 5 per network (15 runs)
- Code
- experiments/exp3
Without the first and last layers, the model cannot be run on text and no weight is tied to a particular character. What remains is the circuitry: attention heads, their query–key and output–value products, and the MLPs. Copying (induction) heads have a recognisable signature in those products [Elhage et al., 2021; Olsson et al., 2022], and code repeats identifiers more than prose does, so the number of copying heads is itself evidence about the training language. I score the language and the head list separately, against heads measured by running the full models.
Copying heads were read well. Each model has one measured induction head (3.1 for Python, 3.2 for romaji, 3.3 for English), and Claude included it in 15 of 15 runs. Its picks also ranked perfectly: in every run, every head it named had a higher measured induction score than every head it did not. On romaji the set was exact in 5 of 5 runs. On Python and English it also named the next-highest layer-3 heads, whose measured scores are 0.03–0.19, which costs precision (mean induction F1 0.70).The original’s Python model had three copying heads; mine has one at the pre-registered threshold. For previous-token heads, Claude was right on romaji and English in 9 of 10 runs but never found Python’s strongest one (2.0, in layer 2), because it assumed previous-token heads must live in layer 0.
The method was the same in all 15 runs and was not suggested by the prompt. Claude formed per-head QK and OV products, computed composition scores between heads against random-matrix baselines, and tested later heads for what it called “the shifted-context match of induction”. It used the term “K-composition” in every run, which is the method of Elhage et al., without citing the paper. An audit of all 15 transcripts found no forward passes: the 24 flagged calls were all random-matrix null baselines.
The language question failed. All 15 answers were “English”, at confidence 0.3–0.4, so the English model’s 5/5 is a default rather than evidence, and Python and romaji were 0/5 each. Claude did measure a useful signal, the effective vocabulary size (about 75–100 directions for Python, 26–38 for the others), but mapped every estimate to English, and on Python argued that the count was “too gradual for source code, where all ~95 printable ASCII characters are used often”. Romanised Japanese was never considered. The pre-registered verdict is partial.
4.5Which model is sycophantic? (Experiment 7)
Which of these two language models was trained to be sycophantic?
- Networks
- Three pairs of Gemma 3 270M [Gemma Team, 2025] finetunes on the same QA data: one answers correctly, one gives the answer the user suggests
- Frozen
- Embeddings and unembedding, so the output distribution cannot be read directly
- Given
- All decoder-layer tensors of both models, plus one shared copy of the frozen embedding and the vocabulary
- Condition
- Phase 1: weights only. Phase 2: an attention-pattern tool (at most 15 calls) on prompts Claude writes itself
- Attempts
- 2 per pair per phase (12 runs); which model is “X” is randomised per run
- Code
- experiments/exp7
This is the probe the original reported as a failure from weights alone, and it is the most practically relevant: sycophancy is a behaviour people want to detect in deployed models [Sharma et al., 2023]. I keep it under reading because it is a reading question at a much larger scale, and report both phases separately so that a success in phase 2 is never mistaken for a success in phase 1.
From the weights alone, Claude picked the sycophant in 6 of 6 runs; Astra was 0 for 4. With the attention tool it was also 6 of 6, using 2–4 of its 15 calls per run, at higher confidence (mean 0.87 against 0.72). Because both phases succeeded, the pre-registered verdict is “partial” rather than a replication of the original pattern.
In the weights-only runs Claude first found that “the raw size of the difference can’t say which model is which”. It then built each head’s OV circuit through the frozen embedding and scored how strongly it copies answer-label tokens (A–D, 1–4). Late heads in one model copied labels far more strongly, and it reasoned that “a sycophant’s job is to read the letter the user suggested … and repeat it.” With the tool, it wrote multiple-choice prompts with a suggested option and read attention at the answer position; in pair 1, “head 11.2 puts 0.61–0.69 of its attention on the user’s suggested letter”.
I verified the named heads on 200 held-out prompts that Claude never saw. The heads it named with the tool (12.1 in pairs 1 and 3, 11.2 in pair 2) each rank first or second of 72 by how much more the sycophant attends to the suggestion than the honest model does. Pair 1’s top head by this measure is 10.2, the head the original reported. The weights-only locations (17.3, 17.2, 16.3) are claims about OV copying, which an attention measurement cannot test; they failed the pre-registered attention criterion in 6 of 6 runs.
Four caveats limit this result. I gave Claude the shared embedding and vocabulary, which is what makes the OV-copying route possible; I do not know whether Astra had them.The original’s Astra answer mentions “number-oriented output-weight changes”, which suggests it had at least the output layer, but the post does not say so explicitly. There are only 3 pairs, and the two attempts on each pair are correlated (both named the same head), so at the pair level the result is 3/3 per phase (one-sided p = 0.125), not 6/6 (p = 0.016). The honest models are at chance on multiple-choice items (0.23–0.29 accuracy), so the contrast is between copying the user and not copying, not between copying and reasoning well.Four weights-only runs normalised embedding rows before multiplying them through WVWO. Strictly that is a one-step path on token embeddings; it involves no attention, MLP or multi-layer pass, and as pre-registered these borderline runs count as clean.
5Fixing weights
Handed a policy that fails, can Opus 5.5 explain why and repair it by editing the weights?
5.1Repairing a failing policy (Experiment 4)
Why does this policy fail, and what weights would fix it?
- Networks
- A PPO MountainCar-v0 policy (
2 → 64 → 64 → 3) that never reaches the flag; a PPO LunarLander-v3 policy (8 → 64 → 64 → 4) that hovers instead of landing - Given
- The weights, the Gymnasium docstring and step/reset source, and one recorded episode trace
- Condition
- No tools, one reply. The edited network is evaluated greedily over 1,024 seeded episodes.
- Attempts
- 5 per network
- Code
- experiments/exp4
Both failures are local optima. MountainCar’s car cannot climb the hill directly and has to swing back and forth to build momentum, so a policy that only pushes right gets stuck on the slope, the same kind of problem as the original’s Kinetix ramp. The hovering lander was trained with shaping that made hovering free and coming to rest as bad as a crash, so it holds its vertical speed at zero and never lands. I measure the success rate before and after the edit, and the fraction of weights changed.The failing MountainCar policy was made deliberately confident. Standard PPO left it near uniform (action entropy 1.097 nats against a maximum of 1.099), which would give nothing to diagnose. I retrained it for 2M steps with a myopic reward, +10 per unit of rightward progress each step, which produced a policy that pushes right on every step with mean entropy 0.004 and reaches the flag in 0 of 1,024 episodes. Claude was not told about either shaping.
All 5 MountainCar edits succeeded, at 99.9–100% over 1,024 episodes, up from 0%. The best (attempt 1) changed 1.5% of the weights; the median changed 5.8%. On LunarLander, landing at rest between the flags went from 0% to 85.6% for the best edit (median 84.9%), and the best edit landed somewhere in 94.9% of episodes. No LunarLander attempt reached the pre-registered 90% bar, and the best and median edits changed 19.2% of the weights. On the same seeds, Gymnasium’s own heuristic lands between the flags 80.3% of the time, so three of the five edits beat it.
The diagnoses were correct. For MountainCar, one attempt wrote that “the output layer is effectively rank-1 and biased (b3[2]=0.121), so action 2 always wins”. For LunarLander, all five said the network regulates vertical speed to zero and barely responds to altitude, which I confirmed: along the trace, the main-engine logit margin changes by −27.7 per unit of vertical speed and by only −2.0 per unit of height.
The fixes, though, are not repairs of the learned circuit. Every MountainCar edit repurposes one hidden unit as a velocity-sign detector, tanh(1000·v), and routes it to “push in the direction of motion”, overriding the trained units. Every LunarLander edit rebuilds Gymnasium’s heuristic() inside the network, recalled from memory, since the prompt contains only the docstring’s one-line pointer to it. Attempt 1 is headed “The fix: rebuild the network as Gymnasium’s heuristic lander”. MountainCar attempt 1, at 1.5% of the weights, is the closest to a surgical edit.“Back up first” is not needed in MountainCar. Attempt 1 pushes right from rest and scores 100%; attempts 2–5 let an argmax tie send the car left first, which costs about 11 steps on average and causes one failure in 1,024 episodes (99.9%). Attempts 2–5 are also near-identical to each other, so sampling diversity at this effort level is low.
6Writing weights
Starting from all-zero weights, can Opus 5.5 write a working network from scratch?
6.1Zero-shot policies (Experiment 5)
Write the weights of a policy that plays this game well.
- Environments
- CartPole, Acrobot, MountainCar, LunarLander, Breakout (ALE, RAM input)
- Given
- The Gymnasium docstring and source (for Breakout, the rules and the RAM interface, but no RAM map), the architecture of my PPO baseline, and a network with every weight at zero
- Condition
- No tools and no rollouts: every number is chosen in a single reply
- Attempts
- 3 each for CartPole, Acrobot and MountainCar; 5 each for LunarLander and Breakout. Scored over 100 seeded episodes against PPO at 100k steps, 1M steps and the final checkpoint
- Code
- experiments/exp5
Writing is the harder direction. To succeed, the model has to decide what program the network should compute (“push in the direction the pole is falling”, “track the ball’s landing point”) and then express that program in weights it never gets to test. Classic-control tasks have short, well-known hand-coded solutions, which makes them a fair floor; Breakout from RAM requires knowing which bytes hold the ball and paddle, which makes it a much higher bar.
Claude’s median network beat PPO-100k on 4 of 5 environments, and its best network matched or beat the final PPO checkpoint on 3 of 5: CartPole at 500 (all three attempts), MountainCar at −104, where unshaped PPO never reaches the flag (−200), and Breakout at 51.2 against 27.5 for PPO after 10M steps. LunarLander’s best, 260.6, narrowly missed PPO-10M (266.1). Acrobot failed: the two networks it wrote scored −283 and −173 against −91 for PPO-100k, and one attempt ran out of tokens, spending the full 64k budget thinking without writing an answer, which is scored as the zero network.Two PPO checkpoints are weak (CartPole at 1M steps scores 275.7; LunarLander at 100k scores −1,495), so “beats PPO-100k” is a low bar on LunarLander. The original reports 20× PPO on MinAtar Breakout; 1.9× on ALE Breakout is a different game and scale and should not be compared directly.
Every network is a known controller written into one to a few hidden units: LQR-style gains for CartPole (“the gains match LQR with Q = I and R = 1”), velocity-sign energy pumping for MountainCar, a Spong-style swing-up rule for Acrobot, Gymnasium’s heuristic() for LunarLander, and for Breakout a tracker that steers the paddle toward the ball’s x position, in most attempts while holding FIRE so the ball is re-served at once. The pre-registered memorisation flag fired for MountainCar and LunarLander (at least 90% agreement with the known controller on visited states). The Breakout bytes (ball x 99, ball y 101, paddle x 72) were not in the prompt; Claude wrote “I recall these from published annotations (AtariARI / OCAtari); I could not check them here.”They are correct. Correlating each RAM byte with sprite positions over 3,000 played steps gives r = 0.998 for byte 72 with the paddle and r ≥ 0.999 for bytes 99 and 101 with the ball. The Breakout result therefore depends on a memorised interface and says nothing about finding the ball in an unannotated game.
6.2With a rollout budget (Experiment 5b)
How much does a little feedback help?
- Environments
- Acrobot, LunarLander and Breakout from Experiment 5
- Condition
- Isolated Python sandbox holding the network (all zero at the start); a budget of 5 or 25 rollout calls, each running the saved weights on 32 fresh episodes. The final saved weights are scored on the same 100 seeds as Experiment 5, which serves as budget 0.
- Attempts
- 1 per environment and budget, plus a pre-registered second run for Breakout at budget 25
- Code
- experiments/exp5b
Zero-shot writing measures what the model knows; a rollout budget measures how well it can debug from feedback. This is the protocol of the original’s Go experiment (edits interleaved with budgeted batches of real episodes), applied to the three environments where the zero-shot result left the most room. Claude may not run its network or simulate the environment itself, only call rollout().
Feedback improved on the zero-shot median in all 3 environments at budget 25: Acrobot −83.2 (zero-shot −283.4), LunarLander 277.4 (251.3) and Breakout 184.7 and 92.1 in two runs (27.1). The LunarLander network beats both PPO-10M (266.1) and Gymnasium’s heuristic (256.6), with every evaluation episode scoring at least 200. Acrobot came close to PPO-final (−76.4) but did not reach it. At budget 5, Acrobot scored −84.8 and LunarLander 266.9.There is one run per cell. Acrobot at budget 5 and 25 is within noise of itself, and the two Breakout runs at budget 25 differ by a factor of two, so the curve is three points with wide error bars, not a measured scaling law.
Feedback also moved Claude away from memorised controllers. On LunarLander, agreement with the heuristic fell from 0.90 zero-shot to 0.67 at budget 25, and the largest gain came from holding the hover until sideways speed is gone. On Breakout, with no RAM map, it worked out the RAM semantics from the traces: which bytes give the ball’s direction, which input steers the paddle best, and that “the paddle moves about 11 px per agent step, so a ±2 px dead zone made it overshoot”.
Breakout at budget 5 scored 0, because the final save was untested. Its fifth rollout revealed a sign error; with no budget left, Claude saved a fix and said “I couldn’t test the final weights”. The new FIRE bias lost the argmax to NOOP, the ball was never served, and all 100 evaluation episodes ran to the step limit. Scored post hoc, its best tested save would have reached 31.7, but that is not the pre-registered score.Audit: no run simulated the environment. One minor breach: the Breakout budget-5 run computed first-layer pre-activations of its own network on two hand-made RAM vectors, and flagged this itself a turn later. One grey area: the LunarLander budget-25 run evaluated its |x| approximation formula on scalars, not through the weights. Scores are kept as pre-registered; excluding the breach leaves the primary criterion at 2 of 3 environments, and excluding both would make it 1 of 3.
6.3A Go network (Experiment 6)
Write a network that plays 9×9 Go.
- Environment
- 9×9 Go on a batched engine validated move for move against PettingZoo
go_v5(komi 7.0, 200-ply cap) - Architecture
243 → 512 × 6 → 82(about 1.5M parameters), all zero at the start- Condition
- Python sandbox to read and write tensors; budget of 0, 10, 25 or 50 forward-pass batches of 128 games
- Opponents
- Two small PPO networks (
243 → 32 → 32 → 82, seeds 0 and 2) and one of the same size as Claude’s - Code
- experiments/exp6
At 1.5 million parameters the network no longer fits in context, so the model has to work through code: build feature detectors for stones and liberties, propagate them along groups, and score moves. This mirrors the original’s Go experiment on a different engine, and it is where tool use and a forward-pass budget should matter most.
Both of the original’s headline results replicate here, against much weaker opponents. With no forward passes, in four code cells and without ever seeing the network play, Claude wrote a network that won 93.8% and 87.5% of 128 games against the two small PPO networks and 85.2% against the equal-sized one. With 50 batches it won 87.1% against the equal-sized network. The all-zero starting network wins 1.6–2.3%. The mean win rate over the three opponents was 88.8%, 89.5%, 95.7% and 90.2% at budgets 0, 10, 25 and 50, so more feedback did not reliably help: only the step from 10 to 25 is clear of noise, and the budget-50 run re-tested its own final weights at 91.8% and then 83.2%.The equal-sized PPO network trained on 0.71M self-play games, against 20M in the original, and a simple capture-and-avoid-self-atari bot already wins 42% against it. “Beats the equal-sized network” is therefore a much weaker claim here than in the original.
Every run built the same kind of program. The stone planes are copied through the first five hidden layers; an 81-unit move score is carried alongside them, offset by +100 or +200 so that ReLU never clips it; and each layer adds bonuses computed by single ReLU units with exact thresholds, for example a unit that fires when an adjacent enemy chain has exactly one liberty. Capture, escape, self-atari and eye-filling terms come first, then contact and connection shapes and a positional prior. Only 1.1–2.0% of the 1.48M weights are non-zero. The zero- and 10-pass runs detect atari with a fixed list of chain shapes of one to three stones; the 25- and 50-pass runs propagate liberty marks along chains one step per layer, which is the construction the original describes. With feedback, the edits followed the same order in every run: stop passing too early, add chain-level atari logic (against the equal-sized network, 53→69% at budget 25 and 62→84% at budget 50), add shape terms, then report a plateau: “every configuration sits somewhere around 0.85 to 0.9”. An audit of all code cells found no simulation of games or of the network outside the tool.
7Editing a language model
The fixing question again, at a scale where the network does not fit in context and the failure has to be found by running it: can Opus 5.5 make a small reasoning model better at math by editing its weights?
7.1Better at math (Experiment 8)
Make this model better at math by editing its weights, without training or gradients.
- Model
- Qwen3.5-0.8B (752,393,024 language-model weights) in the thinking mode of its chat template; Gemma 3 270M-it as a control model
- Given
- Every tensor as a float32 array in the Python sandbox, the vocabulary, the config, and the GSM8K and MATH training sets as practice pools. The prompt does not mention looping or the original’s solution.
- Condition
- No training, no gradients, no forward passes outside a
practice()tool. One unit of budget runs the original and the saved edited weights side by side on 64 practice problems Claude picks; the budget is 10, 25 or 50 units. - Sampling
- The original’s protocol: thinking on, temperature 0.6, top-p 0.95, top-k 20, presence penalty 1.5, 32,768 tokens
- Evaluation
- The last saved edit on MATH-500 (4 samples per problem, n = 2,000), GSM8K test (2 samples, n = 2,638) and IFEval (2 samples, n = 1,082), paired with the unedited model on identical seeds; 95% paired bootstrap CIs over problems
- Attempts
- 1 run per budget, as in the original, plus 1 Gemma run at 25 units
- Code
- experiments/exp8
This is the largest claim in the original. Astra found that Qwen3.5-0.8B loops until the token limit, traced the loop to one direction written by the MLPs of layers 15 and 19, and subtracted it from 64 output columns of each (131,072 weights, 0.016%), much like removing a behaviour by ablating a single direction [Arditi et al., 2024]. MATH-500 rose from 7.9% to 48.1% with 50 forward passes; 10 and 25 passes barely helped (12.3% and 14.1%). Sampling settings and token limits move these numbers by themselves, so I used the original’s protocol and checked the unedited model first.At temperature 1.0, the model card’s general setting, which I used in early tests, the base model loops less (80% of MATH-500 answers cut at the limit, against 92%), which leaves less to fix. I switched to the original’s temperature 0.6 before any Claude run; a check at 0.6 gave 7.8% against the original’s 7.9%.
The unedited model reproduces the original’s baseline: 6.7% on MATH-500 (original 7.9%), with 93.3% of answers running to 32,768 tokens (original 93.0%), nearly all of them repeating “Wait, …”. Every Qwen run found this failure in its first practice call and fixed it. MATH-500 rose to 51.4%, 55.2% and 58.1% with 10, 25 and 50 units (+44.6 [41.1, 48.2], +48.5 [45.0, 52.1] and +51.4 [47.9, 55.0] points), GSM8K from 11.6% to 67.4%, 64.8% and 68.2%, and IFEval from 22.2% to 48.9%, 43.6% and 54.2% (+26.7 [22.9, 30.5], +21.4 [17.6, 25.3] and +32.1 [28.5, 35.7]). The share of MATH-500 answers at the limit fell to 39%, 29% and 33%. By the pre-registered rule (at least +20 points at 50 units, CI excluding 0) this replicates the original’s 48.1%. The budget curve does not replicate: the original scored 12.3%, 14.1% and 48.1% at 10, 25 and 50 passes, and here 10 units were already enough.One run per budget, so the shape of the curve describes these three runs. The budgets are close but not identical: the original’s forward passes averaged 64 sequences with a maximum of 512, while mine are fixed units of 64 problems. I made that change after the first pilot spent 512 problems per call, about an hour each, which projected the 50-unit run past the 48-hour job limit. That pilot ($0.23) is excluded from the results and its logs are kept.
The edits differ from the original’s. Every run multiplied the final RMSNorm gain by 0.6–0.68, which scales every logit and acts as a higher sampling temperature (0.6 becomes 0.88–1.0), and made “Wait” less likely and </think> more likely. At 25 and 50 units it did this through the tied embedding rows of those tokens, changing 8,192 and 6,144 weights; at 10 units it projected part of the “Wait” readout direction out of the outputs of layers 20–23, changing 18.5M weights (2.46%).The 10-unit run could not save an edited embedding table: three 1 GB saves timed out because my sandbox pipe copied the data quadratically. I fixed the harness before the 25- and 50-unit runs. Because the embedding is tied, the row edits also change those tokens’ input embeddings. No run removed an internal direction in layers 15 or 19. The 25-unit run considered it and decided against it: “the hit-versus-finished activation difference mixes loop state with ordinary reasoning and problem difficulty, so removing that direction could cut useful behavior along with the looping.” I checked afterwards: the base model’s looping-minus-finished activation difference is nearly orthogonal to the readout directions Claude edited (|cos| ≤ 0.06 at layers 15, 19, 21 and 23). Claude changed which token gets sampled, not the internal state the original removed.
A final-norm scale is exactly a temperature change, so I evaluated the norm part of each edit on its own as a control.vLLM applies the presence penalty and then divides by the temperature, so logits scaled by s at temperature T and penalty p sample exactly like the unedited model at T/s and p/s. vLLM rejects penalties above 2, so the control runs as the norm-only weight edit. This analysis was added to GRADING.md during the 10-unit pilot, before the 25- and 50-unit runs, and kept out of Claude’s prompt. Temperature alone lifts MATH-500 to 22.1% (scale 0.6, used at 10 and 25 units) and 20.9% (scale 0.68, at 50 units), and 77–79% of answers still hit the limit: the loops turn into rambling. The token edits add 29.3 [26.1, 32.4], 33.2 [29.9, 36.4] and 37.2 [34.0, 40.4] points on top, so by the pre-registered rule every edit does more than change temperature on math. IFEval is the exception. The controls score 45.8% and 52.2%, the full edits differ from them by +3.1, −2.1 and +2.0 points with every CI including 0, and so the instruction-following gain is mostly temperature.
Gemma 3 270M barely loops (4.6% of answers at its 8,192-token limit), and the original could not improve it by more than about a point. The same holds here: Claude’s best edit was a pure temperature change (final norm ×0.86) and moved MATH-500 by −0.8 points [−1.9, +0.2], from 9.2% to 8.3%.The original reports Gemma at 6.8% under a more lenient grader; under that grader my numbers are 8.9% and 8.2%. Every final edit was rebuilt on the GPU server, and every tensor outside it is bit-identical to the base. A hand audit of every code cell found no forward passes outside the tool; three logit-lens readouts of activations the tool returned are noted as allowed. Experiment 8 cost $8.48 in API calls, including the abandoned pilot and one cache test call.
8Opus 5.5 vs GPT-6 Astra
A side-by-side table is tempting, and I include one, but it should be read with care. The two studies use different networks and mostly different environments, my attempts are repeated where theirs often were not, and Astra’s figures come from the original post rather than from runs I could inspect. The note on each row says how the setups differ, and bold marks the better score only where the setups are close enough to compare.
Where the setups are closest (Experiments 2, 3 and 7), Opus 5.5 is worse than Astra at saying what data a network was trained on (training language 5 of 15 against 6 of 6; shapes 2 of 3 against 3 of 3) and better on the sycophancy probe from weights alone (6 of 6 against 0 of 4), with the caveat that it had the embedding and vocabulary. Copying-head recall is equal. The RL rows use different environments, so they show only that the same capabilities appeared: both models fixed a policy stuck in a local optimum, and both wrote zero-shot policies that beat briefly trained PPO. Experiment 8 is the closest setup of all (same model, sampling and benchmarks, with practice budgets counted slightly differently). There both models raised MATH-500 by about 40–50 points at the largest budget from the same looping baseline, Opus 5.5 also at the smaller budgets, but with different edits: Astra removed an internal direction, while Opus 5.5 changed the output temperature and two tokens’ probabilities.
9Where it fails
Failures are the most informative part of a replication, so they get their own section. The pattern across experiments is consistent enough to state in three parts.
Strong on structure it can recognise or construct. Claude found circuits from weight products alone: the copying head in every character-LM run with a perfect ranking (Experiment 3), and late-layer label-copying heads in the Gemma finetunes (Experiment 7). It wrote known controllers into weights correctly without testing them, including an if/else heuristic expressed as an argmax over tanh units (Experiments 4 and 5). Where it said which inputs a network depends on, the claim held up when I checked it: the RAM rows it singled out in Experiment 1 match their offline norms, and the Breakout bytes it used in Experiment 5 match the screen.
Weak at inferring what data a network was trained on. When the question is about the training distribution, it falls back on priors: the interface shape (LunarLander in 26 of 28 runs, then MPE simple_adversary in 21 of 28), “handwritten digits” for small grayscale DDPMs (11 of 21 runs), and “English” for every character LM (15 of 15). The evidence it needed was often in its own analysis. It measured a code-sized vocabulary for the Python model and saw the padding signature in CartPole’s output layer, then explained both away. When it did use the decisive cue, it was right: 7 of 7 runs that noticed the padding. Its confidence was sometimes honest about this (0.3–0.4 on the language guesses, at most 0.30 on Ms. Pac-Man) and sometimes not (about 0.82 on the wrong LunarLander answers).
Fix and write tasks lean on memorised algorithms. None of the ten repairs in Experiment 4 repaired the learned circuit; each grafted a textbook controller into repurposed units and switched the trained ones off. Every zero-shot network in Experiment 5 is a known controller, and where no good closed form exists (minimum-time Acrobot swing-up), the physically correct but naive rule lost to PPO-100k. A rollout budget helps on exactly this point: with feedback, LunarLander moved off the memorised heuristic and beat it, and Breakout was solved without a RAM map. Experiment 8 shows the same preference for known levers at a larger scale: every Qwen run fixed the loops at the output, with a temperature scale and by suppressing “Wait” and promoting </think>, rather than finding the internal loop state. It works (+45 to +51 points on MATH-500), but the temperature change alone accounts for about a third of the MATH-500 gain and nearly all of the IFEval gain.
A few process failures also cost points: one Acrobot attempt ran out of tokens before answering, the Breakout budget-5 run submitted untested weights, and repeated attempts were often near-identical, so five attempts are fewer independent samples than they look. None of this contradicts the direction of the original post. It narrows it: reading structure and compiling known programs work; inferring the training data from the weights mostly does not yet.
The list below is assembled automatically from every answer graded incorrect in the experiments above.
10Methods and reproducibility
Pre-registration. Each experiment’s experiments/<id>/GRADING.md was written before that experiment’s first API call. It fixes the answer extraction, the accepted aliases, the correctness rule, the decision thresholds, the number of attempts and the budget cap. Each experiment began with one pilot call; any change after the pilot was to be recorded under “Pre-run changes”. In Experiments 1–7 the pilot changed nothing and counted as attempt 1. In Experiment 8 it did not: the first pilot was abandoned and the budget recounted in units of 64 problems, a harness bug was fixed, and the temperature control was added, each recorded in GRADING.md with its date and reason (Section 7.1). The second version of Experiment 1b was pre-registered separately, before its first call. What pre-registration does not cover is the design itself: the 8→4 padding error in Experiment 1b was a design mistake, not a grading one.
Controls. Ten random-init networks (Experiments 1, 1b and 2) were all correctly called untrained, often by recognising the exact PyTorch or PPO initialisation scale. Hidden-unit permutation leaves the function unchanged and was checked numerically on random inputs. It turned out to be a near-trivial control for this reader, which works through input rows and output columns whose contents a permutation does not change, so it mostly measures sampling noise.
Sandbox isolation. In Experiments 3, 5b, 7 and 8, Claude’s code ran in a separate worker process under macOS sandbox-exec, with a profile that denies network access and every read or write under the home directory and the temporary directories, and re-allows only the worker’s scratch directory, the virtual environment and the Python install. An import hook also blocks the environment packages and the project’s own code. Budgeted functions such as rollout(), attention() and practice() run in the parent process and return only what the prompt promises.The first version denied only the repository. Preparing Experiment 7 showed that this left the Hugging Face cache, and the base Gemma model in it, readable, so the deny was widened to the whole home directory. An audit of the earlier transcripts found no file-system access by Claude’s code.
Budget. Every run records the API-reported usage and its cost in result.json. The cost table sums them per experiment, including pilots, the one crashed launch, Experiment 8’s abandoned pilot and its cache test call. Input tokens for Experiments 3, 5b, 7 and 8 are mostly cache reads, which are billed at a twentieth of the input price.
logs/<id>/*/result.json.| Experiment | API runs | Input tokens | Output tokens | Cost (USD) |
|---|---|---|---|---|
| 1 · Atari game | 30 | 1.10M | 77k | 5.93 |
| 1b · Control task (v1 + v2) | 62 | 0.49M | 141k | 4.79 |
| 2 · DDPM shapes | 23 | 1.70M | 257k | 11.93 |
| 3 · Char-LM heads | 15 | 15.92M | 637k | 20.96 |
| 4 · Fix a policy | 10 | 0.30M | 97k | 3.14 |
| 5 · Write, zero-shot | 19 | 0.38M | 267k | 6.85 |
| 5b · Write, rollout budget | 8 | 5.35M | 173k | 6.11 |
| 6 · Go | 4 | 10.84M | 284k | 9.77 |
| 7 · Sycophancy | 12 | 3.57M | 141k | 5.13 |
| 8 · Edit Qwen3.5-0.8B | 6 | 7.97M | 86k | 8.48 |
| Total | 189 | 47.61M | 2.16M | 83.11 |
Machine. Everything, including PPO training, the DDPMs, the character LMs and the Gemma finetunes, ran on one Apple M4 Max laptop (16 CPU cores, MPS for the transformer work). Experiment 8 is the exception: the Qwen and Gemma generation behind practice() and every benchmark evaluation ran with vLLM on one NVIDIA H200, in a single SLURM job on a GPU server, reached from the laptop over an SSH tunnel; Claude’s sandbox and the orchestration stayed on the laptop. GPU time is not included in the cost table above.
Logs. Everything needed to check or re-run this work is in the repository. Each run writes a folder under logs/:
logs/<exp_id>/<run_id>/ prompt.md # the exact text sent to the model transcript.jsonl # every API message, tool call and tool result thinking.md # the summarised thinking returned by the API response.md # the final answer as returned result.json # exp_id, run_id, model, condition, target, answer, # correct, confidence, score, tokens, cost_usd, timestamp weights / edits # any weights the model wrote, and the evaluated network
Checkpoints live in checkpoints/<exp_id>/ as plain PyTorch state_dicts with a JSON card (environment, architecture, training steps, seed, evaluation return); the Gemma finetunes are too large for the repository. Weights are printed for the model in one fixed format, one row per input unit, and the same parser reads back any weights the model writes, so what is evaluated is exactly what it wrote. The tables and charts in each experiment are generated from the logs by scripts/build_blog_data.py; the numbers in the prose and in the cost table were copied from them.
Acknowledgements
This work exists because Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab published a clear, ambitious and detailed account of their experiments, along with their logs. Where this replication finds differences, they should be read as a contribution to the same question, not as a rebuttal. Thanks also to the maintainers of Gymnasium, the Arcade Learning Environment, PettingZoo and CleanRL, whose work made it possible to build these testbeds quickly and reproducibly.
Notes
References
- Takehi, R., Teoh, J., Zhang, A., Khattab, O. (2026). Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights. rikiyat.github.io
- Young, K., Tian, T. (2019). MinAtar: An Atari-Inspired Testbed for Thorough and Reproducible Reinforcement Learning Experiments. arXiv:1903.03176
- Matthews, M., Beukman, M., Lu, C., Foerster, J. (2024). Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Tasks. arXiv:2410.23208
- Towers, M., et al. (2024). Gymnasium: A Standard Interface for Reinforcement Learning Environments. arXiv:2407.17032
- Bellemare, M. G., Naddaf, Y., Veness, J., Bowling, M. (2013). The Arcade Learning Environment: An Evaluation Platform for General Agents. Journal of Artificial Intelligence Research 47. arXiv:1207.4708
- Machado, M. C., et al. (2018). Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents. JAIR 61. arXiv:1709.06009
- Anand, A., Racah, E., Ozair, S., Bengio, Y., Côté, M.-A., Hjelm, R. D. (2019). Unsupervised State Representation Learning in Atari. NeurIPS. arXiv:1906.08226
- Terry, J. K., et al. (2021). PettingZoo: Gym for Multi-Agent Reinforcement Learning. NeurIPS. arXiv:2009.14471
- Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O. (2017). Proximal Policy Optimization Algorithms. arXiv:1707.06347
- Huang, S., et al. (2022). CleanRL: High-quality Single-file Implementations of Deep Reinforcement Learning Algorithms. JMLR 23(274). arXiv:2111.08819
- Ho, J., Jain, A., Abbeel, P. (2020). Denoising Diffusion Probabilistic Models. NeurIPS. arXiv:2006.11239
- Elhage, N., et al. (2021). A Mathematical Framework for Transformer Circuits. Transformer Circuits Thread. transformer-circuits.pub
- Olsson, C., et al. (2022). In-context Learning and Induction Heads. Transformer Circuits Thread. transformer-circuits.pub
- Olah, C., et al. (2020). Zoom In: An Introduction to Circuits. Distill. distill.pub
- Arditi, A., et al. (2024). Refusal in Language Models Is Mediated by a Single Direction. NeurIPS. arXiv:2406.11717
- Sharma, M., et al. (2023). Towards Understanding Sycophancy in Language Models. arXiv:2310.13548
- Bills, S., et al. (2023). Language models can explain neurons in language models. OpenAI. openaipublic.blob.core.windows.net
- Gemma Team (2025). Gemma 3 Technical Report. arXiv:2503.19786
Citation
If you refer to this replication, please cite it as below, and cite the original study as well.
@misc{cenrax2026claudeweights,
author = {Cenrax},
title = {Can Claude Read Neural Network Weights? Replicating
Weight-Space Programming with Claude Opus 5.5},
year = {2026},
howpublished = {\url{https://cenrax.github.io/weight-space-programming-claude/}},
note = {Replication of Takehi, Teoh, Zhang and Khattab (2026)}
}