nesformer: A Working NES Inside ~1GB of Qwen Weights, Explained


Someone put a working NES CPU inside a language model’s weights. The project is called nesformer, and it runs an actual 6502 CPU inside the weights of a Qwen3.5-architecture checkpoint.

The bottom line: A compiler — not training — put a working NES CPU inside a language model’s weights, and while that’s a remarkable demonstration of what AI models can hold, nothing in the sources shows it helps game agents play or test games.

One honest note before we start: everything in this post comes from the project’s README, Simon Willison’s write-up, and the Qwenputer blog post nesformer builds on. No independent third-party benchmark of nesformer was found as of the sources gathered for this post, so every speed and correctness number below is the author’s own claim, labeled as such.

What nesformer actually is

Short answer: nesformer is an NES 6502 CPU compiled into the weights of a stock Qwen3.5 model — nobody trained the weights, a compiler computed every one. When you play a game, the model predicts the CPU’s execution log one token at a time, while a small host program draws the picture, reads the controller, and fires the vblank interrupt.

The README describes nesformer as “an NES CPU compiled into the weights of a stock Qwen3.5 model.” That word compiled matters. In normal machine learning, you train a model — you show it examples and adjust its weights (the numerical parameters inside it) until it learns a behavior. Nobody trained anything here. A compiler — a program that translates one precise description into another — computed every single weight directly, so the model behaves like a NES CPU by construction, not by learning.

What does “running” look like? A language model works by predicting a token — the basic unit a model reads or predicts one at a time — over and over. In nesformer, each token is a piece of the CPU’s execution log: what instruction ran, what memory it touched, what the flags (the CPU’s one-bit status markers) became. The model predicts that log, step by step, exactly as it would predict the words of a sentence.

Of course, a CPU alone doesn’t make a picture. A small host program called nesformer/host.py handles everything around the CPU: it draws the screen, reads the controller, and fires the vblank interrupt — the brief window after the NES finishes drawing one frame and before the next, when the screen isn’t being updated. That interrupt is the NES’s heartbeat, and the host controls its timing.

Where did this come from? Simon Willison’s 30 October 2026 post describes nesformer as a “working emulator for a NES 6502 CPU baked into the weights of Qwen3.5-4B” and notes, “You can now play NES ROMs such as Super Mario Bros (very slowly).” Per Willison’s quotation of Hacker News, creator Drew Breunig said he used Claude Opus 5.5 with two prompts — first, to explore whether the technique could emulate the NES chip and RAM and what size model it would need; then, “I think we should build it! Just for fun. It’s okay if it’s not fast.” (The “Opus 5.5” naming is the author’s own statement, not verified against vendor pages.) Per Willison, the model even found and used a conformance test collection for the NES CPU — the standard NES CPU conformance test ROM. (Attribution note: the README credits that test ROM, nestest, to Kevin Horton, while Willison’s post says Christopher Pow — the sources disagree, so we’ll just call it the standard conformance test.)

How a CPU fits inside a language model

Short answer: Qwen 3.5’s Gated DeltaNet layers hold a fixed-size internal state that can be precisely written and read, so the compiler uses that state as the machine’s memory — keeping the 6502 core, 2 KB of work RAM, and 32 KB of PRG-ROM inside the model, while the host handles the PPU, controllers, APU registers, and NMI timing.

This is the part worth understanding, because it’s the actual trick. Modern language models like Qwen 3.5 don’t use only the classic attention mechanism you may have heard about. Per the Qwenputer write-up by Divyansh Agrawal, in Qwen3-Next / Qwen 3.5, three of every four attention layers are actually Gated DeltaNet (GDN) layers — a linear-RNN layer whose fixed-size internal state can be written and read back precisely. An RNN (recurrent neural network) is a layer that carries a running “memory” from step to step. The GDN’s memory has a fixed size no matter how long the context gets — unlike the attention KV cache, which grows with every token.

Why does that matter for a CPU? Because the compiler can treat that fixed-size state as the machine’s memory. Per the Qwenputer write-up, setting the gate to 1 and beta to 1 with one-hot key encoding makes each write overwrite exactly one state row, and a one-hot query reads it back. In plain terms: the model can write a value to one specific “address” in its internal state and read it back exactly, like RAM. That precision is what makes deterministic computation possible inside a model that normally predicts fuzzy, probabilistic text.

This trick didn’t start with the NES. nesformer builds on Qwenputer, Agrawal’s project that compiled a CHIP-8 and then a RISC-V CPU (RV32IM, a modern open processor design) into Qwen3.5-4B. nesformer reuses Qwenputer’s compiler and extends the lineage one more step: CHIP-8, then RISC-V, then the NES’s 6502.

So what’s actually inside the model, per the README? The 6502 core — all 151 official opcodes (a single CPU instruction), addressing modes, flags, stack, and BRK and NMI entry — plus 2 KB of work RAM, 32 KB of PRG-ROM (the cartridge’s program memory, stored twice: once for instruction fetch, once for data reads), and OAM DMA reads (a NES hardware feature for copying sprite data). The PPU (the NES’s graphics chip) is not in the weights — it lives in the host program as a scanline renderer and palette, along with the controllers, the APU (audio) registers, and the NMI timing at vblank (the interrupt the NES fires at vblank to signal a finished frame).

Willison’s post names Qwen3.5-4B, but the README describes a 466.6M-parameter checkpoint with a hidden size of 768, and the sources gathered for this post do not explain how the two descriptions relate. The checkpoint itself, per the README: a Qwen3_5ForCausalLM with 7 Gated DeltaNet layers (the circuit) plus 1 all-zero attention layer, hidden size 768, vocabulary 263,808, and 466.6M parameters in bf16 — that’s 933 MB, which the title’s “~1GB” rounds. Of those parameters, only 9.3M are nonzero, and the weights zip down to just 3.3 MB. A mostly-zero, hand-computed checkpoint is exactly what you’d expect from a compiled circuit rather than a trained network.

What it can run, how fast, and what’s missing

Short answer: Author-reported, nesformer runs NROM cartridges at 1–6 seconds per frame — 150–290x slower than a real NES — and matches reference emulators on millions of instructions. But it supports only the simplest cartridge type, only official opcodes, no sound, and no IRQs.

Let’s set honest expectations, starting with speed. All figures here are author-reported, from the README. The fast engine (built with numba, a Python accelerator) runs at about 1,650 tokens/s on one CPU core of the build machine; the numpy engine manages about 210 tokens/s; stock transformers crawl at about 10 tokens/s. For scale: a real NES would need about 700,000 tokens per second. The README puts the overall gap at 150–290x slower than the real console. A single frame takes 1–6 seconds, and per the README’s quick-start note, 600 frames — ten seconds of NES time — take around 25 minutes. An optimization called idle-loop skip cuts the token count by 2.6x to 4.2x by advancing the clock past repeated wait-loop iterations, per the README.

Now correctness — again, all author-reported. The project’s reference CPU matches nestest’s golden log (a CPU trace — the step-by-step log of everything a CPU executed, which is how emulator accuracy is checked) for 3,348 instructions, and matches the py65 emulator on 5.7 million instructions of game code. Better still, the free-running weights — the model generating on its own, no reference in the loop — produce traces identical to the reference: 4,586 tokens identical on nestest, 1,916,527 tokens on a Thwaite run with all 300 saved frames matching, and 2,645,885 tokens on a Full Game run with all 210 saved frames matching. Those are big numbers, and if they hold up under outside scrutiny, they say the compiled circuit is genuinely faithful.

The limitations, in plain terms from the README: nesformer runs NROM cartridges only — NROM is the simplest NES cartridge design (iNES “mapper 0”) with fixed, non-swappable ROM; a mapper is cartridge hardware that swaps ROM banks, and the popular MMC1 and MMC3 mappers aren’t supported yet. Only official opcodes work — hitting an unofficial opcode emits HALT, a stop instruction. Code must run from cartridge ROM — a jump into RAM also emits HALT. There are no IRQs (the APU frame IRQ and DMC interrupts are out), the PPU and APU are not in the weights, there’s no sound, and mid-scanline effects are approximate. The README says MMC1 and MMC3 support is “not supported yet”.

One practical note: the bundled test ROMs are freely licensed (nestest by Kevin Horton per the README; Full Game by Doug Fraker, MIT; Thwaite by Damian Yerrick, GPL-3.0), but users supply their own NROM ROMs. Nesformer itself is MIT-licensed, as is Qwenputer/chipformer (MIT, (c) 2026 Divyansh Agrawal).

And the caveat bears repeating: every speed and correctness figure above is the author’s own claim. No independent third-party benchmark of nesformer was found as of the sources gathered for this post.

What this means for game agents — and what it doesn’t

Short answer: What’s demonstrated is that computation can be compiled into model weights with exact, verifiable results. What’s not demonstrated is any benefit for game agents — no source establishes one, and no independent benchmark exists, so any claim that this helps agents play or test games is speculation.

It’s tempting, especially for readers of a game-development blog, to jump straight to the exciting conclusion: “AI models can now run games — so AI agents can play games!” Here’s the fact/speculation line, drawn clearly.

The fact: nesformer demonstrates that a compiler can place real, deterministic computation — an entire CPU, its RAM, and its program — inside a language model’s weights, and that the resulting model can execute millions of instructions with traces identical to a reference emulator (author-reported). That’s a genuine milestone in what models can hold. It follows the Qwenputer lineage (CHIP-8 → RISC-V → 6502), applying the same approach to a third instruction set.

The speculation, clearly labeled: Could an agent someday use a game compiled into its own weights to reason about gameplay, test builds, or verify behavior? Maybe — but no source in this dossier establishes any such use. Nothing shows nesformer helps game agents play or test games, and given that the model runs 150–290x slower than a real console (author-reported) and supports only NROM cartridges, it’s not an obvious agent tool today. Treat any forward-looking claim in that direction as an open question, not a finding.

There’s also a subtler lesson for game developers here. The Gated DeltaNet trick — using a model’s fixed-size internal state as precisely addressable memory — is a reminder that modern model architectures have structure that clever compilers can exploit. Whether that structure becomes useful for actual game tooling is exactly the kind of question worth watching, without pretending it’s already answered.

FAQ

Did anyone train the nesformer model?

No. Per the README, nobody trained the weights — a compiler computed every one, translating the NES 6502 CPU’s behavior directly into the model’s parameters. That’s why only 9.3M of the 466.6M parameters (author-reported) are nonzero, and why the weights zip to just 3.3 MB.

Can nesformer run any NES game?

No. Per the README, it supports NROM cartridges only (iNES mapper 0) — the simplest cartridge design with fixed, non-swappable ROM. Bank-switching mappers like MMC1 and MMC3 aren’t supported yet, only official opcodes work, code must run from cartridge ROM, and there’s no sound. Users supply their own NROM ROMs; the bundled test ROMs are freely licensed.

How slow is it, really?

Author-reported figures from the README: a frame takes 1–6 seconds, 600 frames (ten seconds of NES time) take around 25 minutes, and overall nesformer runs 150–290x slower than a real NES, which would need about 700,000 tokens per second. The fast numba engine reaches about 1,650 tokens/s on one CPU core. Remember, these are the author’s claims — no independent benchmark was found for this post.

What you learned

  • nesformer is compiled, not trained: a compiler computed every weight, placing a working NES 6502 CPU inside a stock Qwen3.5 model, per the README.
  • The trick is memory: Qwen 3.5’s Gated DeltaNet layers hold a fixed-size state that can be written and read precisely, so that state acts as the machine’s RAM — a technique inherited from Qwenputer (CHIP-8, then RISC-V, then the 6502).
  • The split matters: the CPU core, 2 KB work RAM, and 32 KB PRG-ROM live in the weights; the PPU, controllers, APU registers, and NMI timing live in a small host program.
  • Expectations are honest: author-reported, it’s 150–290x slower than a real NES, runs NROM cartridges only, has no sound, and no independent third-party benchmark was found.
  • The agent question is open: this is a demonstration of compiling computation into weights — no source establishes any use for game agents, and claims otherwise are speculation.