A language model inside Doom

People kept saying "LLM's can easily run DOOM." I wanted to see if Doom could run an LLM. So I asked Doom what the BFG 9000 was. It told me it was a 1990s British TV series.

Not a game about Doom. Doom itself. Well, GZDoom, to be precise. I had Qwen2.5-0.5B-Instruct running inside its ZScript VM, taking a prompt from the console, turning it into tokens, doing the forward pass, and turning the resulting tokens back into text. Somewhere in that sequence, the BFG acquired a television career. Hilarious, but not what I wanted.

Naturally, I tried a larger model. LFM3-1.7B informed me that the BFG was a British nuclear research facility instead. I suppose it's moving up in the world.

The idea started because somebody had put Doom inside a language model, and I thought reversing that arrangement would be funny. It was. It also turned into an exercise in numerical computing, weight formats, tokenizers, scheduling, and discovering just how many ways a game can freeze without actually crashing.

And yes, it genuinely runs the inference inside the engine. Python prepares the checkpoints and reference outputs before launch. When you ask a question in Doom, the ZScript side loads the model data, tokenizes the prompt, does the calculations, maintains the caches, and decodes the answer. No inference API hiding behind the curtain. Just the game engine being persuaded to take up a second profession.

LFM3 reading a real prompt while MAP01 remains playable. Captured during a fresh run asking “What is 7 times 8?” with thinking disabled.

A quick tour of what we're abusing

Doom came out in 1993. id Software released its Linux source in December 1997, with release notes from Carmack, and people built and maintained newer versions of the engine called source ports. GZDoom is one of those ports; it can render with modern graphics APIs like Vulkan and lets mods do considerably more than the original game ever imagined.

Doom's original assets live in WADs(Where's All the Data? files), collections of named blobs called lumps. The main game data is the IWAD, such as DOOM2.WAD. Modern GZDoom mods can also be packaged as PK3 files, which are ZIP archives with a game-specific extension. A PK3 can carry scripts, textures, and as it turns out, a deeply unreasonable quantity of model weights.

Doom has more than one scripting language. ACS, inherited from Hexen, was designed for things like doors, switches, and map events. It can do computation, but its 16.16 fixed-point arithmetic and relatively awkward data handling make it an unpleasant place to implement a language model. You could do it, but you could also just... choose to not.

ZScript is far better suited: classes, dynamic arrays, strings, floating-point operations, and callbacks into the game and UI. It compiles to GZDoom's VM format, and the engine has JIT facilities too. This is not the same as saying every hot function necessarily gets JIT-compiled, but it matters when talking about speed.

The game simulation traditionally runs at 35 tics per second. That's about 28.6 milliseconds per tic. Keep that number in mind; it becomes the villain later.

A mod can register a StaticEventHandler through MAPINFO, then receive callbacks when a level loads, when the world ticks, or when the console sends an event. That provides the doorway into the engine. We just have to fit a language model through it.

What a language model actually does

If you haven't dealt with language models at the implementation level, here's the short version. A prompt is broken into tokens: pieces of text represented by integer IDs. Those IDs become vectors of numbers. The model repeatedly transforms the vectors using its learned weights, and at the end it produces a score for each possible next token. Choose a token, append it, and do it again.

This is why you can ask a model about the BFG 9000 and get a beautifully grammatical answer that bears no relationship to reality. The model is generating likely text, not consulting a certified weapons catalogue from the UAC Armoury on Mars.

Most of the heavy lifting is linear algebra. You multiply vectors by big matrices of weights, run normalization and activation functions, use attention to combine information from previous tokens, and repeat through many layers. For the one-token-at-a-time part of generation, matrix–vector multiplication is particularly important.

A dot product is simple: multiply corresponding numbers, add the products. A matrix–vector multiplication is a lot of dot products, one per matrix row. Now make the matrix contain a billion numbers and ask the engine that normally manages imps and shotgun pickups to deal with it.

Act I: Can Doom eat a gigabyte of numbers? Spoiler: Hurt Me Plenty

The first experiment, v01_int8_matvec, was not a chatbot. It was a kill test: could the engine load a huge weight matrix, multiply it by an input vector, give the correct answer, and return to gameplay afterward?

The exporter made deterministic signed eight-bit values. To package them as bytes, it stored each signed value with an offset of 128. At runtime the actual weight was stored_byte - 128. You can calculate the dot product directly that way, or do something slightly nicer:

sum((stored_byte[j] - 128) * input[j])
    = sum(stored_byte[j] * input[j]) - 128 * sum(input[j])

The sum of the input only has to be computed once. You then apply that correction to each output row. Small algebraic trick, many repeated rows.

Keeping the matrix in byte arrays matters as well. If every eight-bit weight gets expanded into a larger integer for convenience, your memory bill starts looking like the kind of thing you'd rather not tell your operating system about.

The largest surviving test was a 32,768 × 32,768 matrix: 1,073,741,824 weights, or exactly one GiB of raw matrix bytes. The log recorded:

LOAD size=32768 mode=1 bytes=1073774592 ms=3086.877
RESULT size=32768 mode=1 iters=3 warmup_ms=1311.543 total_ms=3980.799 ms_per_iter=1326.9330 macs_per_sec=809190696.0
MATMUL_DONE
TICK_AFTER_BENCH game_resumed_ok

That's roughly 809 million multiply-accumulates per second, or about 1.327 seconds for one matrix–vector product on the test machine, a Ryzen 9 8945HS. The benchmark also produced checksums and selected output values for comparison against a Python reference.

The last log line is more important than it looks. Printing MATMUL_DONE only says the function returned. TICK_AFTER_BENCH game_resumed_ok was printed after 35 later world ticks. Doom had actually gone back to being Doom, playing Running From Evil.

That doesn't mean you can take 809 million MAC/s, divide the parameter count of any model by it, and confidently predict the token rate. It was one kernel, one memory-access pattern, and one machine. But it answered the question I cared about: GZDoom could hold a truly stupid matrix, compute with it, and survive. With some care, it could do that repeatedly and still be a game.

Act II: Attention is all you need, apparently?

Next was v02_attention: an attention block from GPT-2, evaluated on four positions. GPT-2's block has a hidden width of 768 and twelve attention heads of width 64. Its combined query/key/value projection produces 2,304 output channels.

Here's what attention means in this context. A token produces a query describing what it is looking for, and keys and values that other tokens can consult. Comparing the query with earlier keys gives scores; after a softmax turns those scores into weights, the model takes a weighted combination of the values. With causal attention, a token mustn't peek at tokens that come after it. That would be cheating on the next-token prediction exam.

The block also needs normalization, projections, and the output transformation. Small implementation details matter enormously: transpose the wrong tensor, use the wrong mask, or omit the subtraction of the maximum score before softmax, and the code can run perfectly while the math goes on holiday.

Rather than check only the final output, the harness compared intermediate checksums from the QKV projection, attention result, and output projection, plus selected floating-point values. The recorded pass took 11.51 milliseconds for that one block and those four positions.

A promising result, but four positions through one attention block are not an LLM. That's... four positions through one attention block.

Act III: Knee Deep in the Qwen

v03_qwen was the first complete language model: Qwen2.5-0.5B-Instruct. It has 24 layers, a hidden width of 896, an intermediate feed-forward width of 4,864, fourteen query heads, two key/value heads, and a vocabulary of 151,936 tokens.

Two KV heads for fourteen query heads means it's using grouped-query attention: multiple query heads share the same key/value head. That's useful for reducing the memory spent on stored keys and values, but it means your indexing needs to know which seven query heads share which KV head. "Just split it into heads" is not quite enough.

The full path needed RMSNorm, rotary positional embeddings, attention, the SwiGLU feed-forward block, residual connections, a final vocabulary projection, and a KV cache. That cache remembers previously computed keys and values so the model doesn't have to redo the entire prompt every time it generates one more token.

Qwen also has tied input embeddings and output weights. One large matrix serves two jobs: retrieving a single row when a token comes in, and reading across the vocabulary to score what token comes out. The same data, with wildly different amounts of work depending on the direction.

Then came one of the most educational bugs: Qwen's query, key, and value projections have biases. The output and feed-forward projections don't. Forgetting those biases still gives you correctly shaped results, but they're results from a model that does not exist. The code isn't obviously broken; it's just asking the wrong mathematical question with impressive consistency.

The project ran into several equally charming numerical traps:

• Double precision: An IEEE-754 double uses 64 bits, but cannot represent every 64-bit integer exactly. Reconstructing a double from its byte pattern requires treating the sign, exponent, and fraction as fields, not naively stuffing the whole pattern into a floating-point number.(note to self: maybe just look at a custom DoubleFromBytes function instead of trying to be clever)
• Rounding: NumPy's ties-to-even behavior had to match the ZScript quantization path. An off-by-one quantized activation is small until you multiply it through several more layers.
• Exponentials: The reference and ZScript paths used the same range-reduced polynomial approximation where consistent quantized results mattered.
• Checksums: Some unsigned diagnostic values wrapped near four billion. That was integer arithmetic, not the model announcing an activation with the approximate value of an entire national budget.

These are the details an architecture diagram politely leaves out.

You can't just split a hole into the surface of Prompt

At first, prompts could be prepared in Python and exported as token IDs. That's useful for repeatable tests. It's also not nearly as funny as opening Doom's console and typing whatever question occurs to you.

So the tokenizer moved into ZScript as well.

Qwen uses byte-level BPE tokenization. That means text is divided according to specific pre-tokenization rules, converted through a byte alphabet, and merged using ranked vocabulary pieces. Special tokens have to be recognized as whole units. The exact grouping of spaces, contractions, numbers, and punctuation changes the token IDs. Split on spaces and you haven't made a faster tokenizer; you've made a different one.

Unicode supplies another rabbit hole. Some visible characters can have multiple underlying representations, such as a precomposed accented letter versus a base letter followed by a combining mark. For this tokenizer, NFC normalization had to behave correctly, including across non-English text. The exported tokenizer data included Unicode categories and normalization information as well as merge ranks.

The Qwen export carried 151,387 merges and 22 special tokens. The loader had to recognize those, preserve bytes correctly, and decode output without destroying a UTF-8 character that happened to straddle two tokens.

The test wasn't simply, "Does it look readable?" It was, "Does it produce the same token IDs as the reference tokenizer?" That's the useful question. A readable but incorrect token stream is still wrong input to the model.

With the console integration working, the Qwen mod accepted arbitrary prompt strings, loaded weights once, and reset generation state between requests. It used a small 128-token total context budget during this stage, with synchronous generation. In other words, you could ask Doom a question, and Doom would think so hard that Doom stopped being a game until it was done.

The saved answers included:

The first level of Doom 2 is level 1.
The BFG 9000 is a 1990s British television series.
Mars is black.

The BFG answer took about 13.4 seconds of recorded generation time. Several runs measured around 0.66–0.68 seconds per reported token interval.

Not every answer was good, clearly. But I want to distinguish two things: numerical correctness and factual correctness. If ZScript produces the same logits and tokens as the reference implementation, the port may be doing its job perfectly while the model confidently tells you Mars is black. The reference is what tests the port. A trivia question mostly tests how willing the model is to improvise.

LFM3 is not Qwen with a bigger hat. Just a bigger caliber.

I could have stopped with Qwen. Instead I tried LFM3-1.7B from Liquid AI, using a pinned checkpoint configuration. It has a hidden width of 2,048, a feed-forward width of 7,680, and a vocabulary of 128,000. The model has 24 layers, but they're not 24 copies of the same attention block.

Thirteen layers use convolution, six use full attention, and five use sliding-window attention. This is a hybrid language model: different layers process sequence history in different ways.

LiquidAI’s LFM3-1.7B-Base architecture figure, showing the backbone used here. Source: LiquidAI’s LFM3 launch article; captured from its embedded diagram.

Full attention can consult all preceding positions within the usable context. Sliding attention keeps a limited window—in this configuration, 256 positions. The convolution layers do something different again: they maintain a short running history and apply learned filters to it. In this model, that causal convolution has a width of seven and acts on intermediate streams usually called B, C, and x, with gating and projections around it.

Why mix them? Processing long sequences with full attention can get expensive. Smaller history mechanisms can handle some of the work without always consulting everything that came before. That sounds sensible for an edge-oriented model.

It was less convenient for a Doom mod that had only just learned ordinary attention. But man, was LiquidAI cooking with this design. They've been experimenting with hybrid architectures for a while, and the results are impressive. They're one of the cooler labs out there, so go show them some love if you like em.

The attention types also differed in their rotary embedding rules, head normalization, and gating. The LFM3 path had sixteen query heads and four KV heads, each 128 wide. Full attention and sliding attention used different rotary dimensions and bases. Those aren't details you can safely wave away by changing a couple of constants.

The exporter mapped 286 checkpoint tensors to the runtime layout. 154 were quantized matrices. The matrix weights used eight-bit values with a scale per row, while activations and caches were kept as doubles for this version. That cost memory, but avoided some of the quantized-activation disagreement from the Qwen experiment.

The validation took the long way around: compare the independent original-weight reference with the model's Transformers path, compare a quantized reference, and then compare ZScript against that quantized reference. This separates model behavior, quantization changes, and implementation bugs instead of treating any weird answer as proof that ZScript has committed a crime.

In one test, LFM3 itself described the BFG as a British nuclear research facility. That was the original-weight Transformers baseline, using float32 arithmetic, greedy decoding, and thinking disabled, before testing the ZScript port. It was one wrong answer under that setup, useful for separating model behavior from port errors; it wasn't a broad evaluation of what LFM3 can do.

The native Mars run matched all eight output token IDs from its quantized reference, with a maximum reported first-logit difference of about 1.78 × 10⁻¹⁴. That's a specific test result, not a promise that quantization preserves every output under every prompt. But it's strong evidence that the port was reproducing the intended computation in that case.

LFM3's tokenizer needed another round of special treatment: it doesn't use the same normalization path as Qwen, groups digits differently, and enables ignore_merges. The exported data contained 293,320 merges and 140 special tokens. Validation covered 114 native tokenizer cases, including Unicode and randomized inputs.

Apparently my game now needed two serious tokenizers. There are Doom mods with less complicated boss fights.

Where do 1.7 billion weights live?

In the PK3, initially. Large weight files are stored in the ZIP without compression. When first requested, the loader reads each file as a lump, copies the bytes into Array<uint8> in 131,072-byte chunks, and releases the temporary string. Smaller tensors and row scales go into double arrays.

So the weights exist in two places at different times: as resource blobs in the archive, then as arrays the inner loop can access. The hot path isn't rereading a ZIP entry for every multiplication. I admire the absurdity of a Doom mod loading an entire model; I do not admire unnecessary I/O inside a dot product.

The quantized matrix data alone occupies about 1.728 GB. The exported data directory, including other tensors and tokenizer information, is about 1.743 GB. The tied embedding/output matrix is the biggest single weight block: 128,000 × 2,048, or 262,144,000 weight bytes, before scales.

Each row has a scale based on the largest absolute weight in that row divided by 127. In effect, many floating-point weights are approximated by signed eight-bit values and a shared row scale. You're exchanging numerical precision for storage. Given the alternative was attempting to move several gigabytes more through a game engine that had already seen horrors beyond its comprehension, that seemed like a reasonable trade.

Freezing Doom, unfreezing Doom, and discovering a worse problem

The first LFM3 scheduling approach allowed roughly a second of work before returning control to the game. Unsurprisingly, the window froze, moved, froze, moved. One measured slice took about 1,003 milliseconds.

This was entirely consistent with the computation happening on the engine's thread. It was also a terrible user experience, unless you specifically enjoy watching a first-person shooter develop hiccups.

The next idea was to give computation only tiny intervals. Say a callback runs every game tic and spends three milliseconds doing inference. At 35 tics per second, that's at most 105 milliseconds of inference work per second, even before overhead. You have made Doom responsive, but you've also made your model spend nearly all its time not computing.

One observed run used only about eight percent of a CPU core. Another wait stretched toward half an hour. The window could move. The answer did elude.

So the runtime became an explicit state machine. It tracks the current token, layer, operation, and position inside that operation. Matrix–vector work advances sixteen output rows at a time. Attention and softmax operations are broken into similarly resumable pieces. Each callback performs a bounded job, stores the state, and returns. The next callback picks up where it left off.

The matrix-vector helper keeps its row cursor between calls. This is the part that advances it:

int end=min(row+16,m.rows);
DotRows(m.bytes,m.scale,input,output,m.cols,row,end);
row=end;
if(row==m.rows) { row=0; return true; }
return false;

Returning false means there are more rows to process. Returning true lets the outer state machine move to the next operation. The cursor and intermediate vectors survive the yield, so the next callback continues the calculation.

This is cooperative scheduling. It can't magically interrupt an operation in the middle, so each indivisible job must be short enough for the intended time budget. A four-millisecond budget doesn't help if one job takes a hundred milliseconds.

The engine's scopes and callbacks supplied a second trap. The UI and gameplay sides are separated, and opening the console can pause world progression. A model driven entirely by WorldTick might stop computing precisely when you open the console to use it. The later implementation uses UI callbacks, including UiTick and RenderOverlay, to advance the work. The console command sends local events, such as llm and llm_cancel, to the appropriate handler.

In one responsiveness check, the largest inference slice was roughly 4.16 milliseconds, and cancellation was accepted promptly. That's the good news. The model can yield, preserve its state, and not turn the window into a decorative rectangle.

The bad news is speed. A measured run with a 34-token prompt and eight generated tokens took 142 seconds including startup and loading. The benchmark used a controlled MAP01 setup at 640×480 with no monsters, so don't treat that as a universal number for every machine and map.

Rendering callbacks also aren't an ideal compute scheduler. Minimize the window or change background behavior and the amount of work performed can change. The launch configuration had to control background activity, frame settings, and vsync for more reproducible measurements. A game window does not become a reliable batch-compute environment just because you ask nicely or even forcefully for that matter.

Removing limits is not the same as removing consequences

The early implementation used fixed caps for context and generated output. Removing those checks alone would be a terrific way to graduate from an annoying limit to an out-of-bounds read.

The LFM3 full-attention cache now grows in pages of 256 positions. Sliding-attention layers keep a fixed 256-position ring buffer. Scratch storage grows as needed, and rotary values beyond the originally prepared table can be computed on demand.

This costs real memory. With four KV heads of width 128, double-precision keys and values for one 256-position page take 2 MiB per full-attention layer. Across six full-attention layers, extending the context by another 256 positions costs 12 MiB, before other allocations. The five sliding layers together hold about 10 MiB of their fixed KV history.

The checkpoint configuration lists a maximum of 262,144 positions. The mod no longer imposes some of its earlier smaller fixed cutoffs, but that doesn't mean infinite memory, infinite meaningful context, or a fully tested 262K-token conversation. The recorded tests checked cache boundaries at positions 255, 256, 511, 512, and 1,025, and a 10,000-byte prompt tokenized into 2,027 tokens matching the reference. That's what was tested.

Large-position rotary math also needed care. A range-reduced sine/cosine implementation brought measured error at position 262,143 down to about 2.24 × 10⁻¹¹.

The defaults now let generation continue until the model's end token or cancellation unless the user sets a limit. llm_max_tokens 0 and llm_timeout_s 0 remove mod-imposed count and time cutoffs. They do not remove the hardware bill. If you ask Doom to think forever, it may take you seriously.

Actually asking the thing a question

The LFM3 console interface uses:

llm_thinking false
llm_prompt "What is the BFG 9000?"
llm

llm_thinking true selects a different chat-template prefix for the next request. It doesn't retrofit a request already in progress. llm_cancel stops the current request at its next cooperative opportunity.

Output is streamed from the model's generated tokens, including thinking tokens when enabled. The decoder has to buffer incomplete UTF-8 sequences because a single Unicode character can span multiple token boundaries. GZDoom's console-print functions also append newlines, so token-by-token streaming isn't exactly a polished messaging app. The complete response can be printed together afterward.

This is still Doom's console. Its primary job was never to host a chatbot, and occasionally it makes that clear.

So where does the time go?

I suspected the ZScript VM. A pile of scalar arithmetic in a scripting language sounds like a recipe for overhead. But GZDoom has a JIT, and the measured run reported vm_jit=1 and vm_jit_aot=1. That does not establish that every hot function was compiled to machine code, but it means "the interpreter is obviously the bottleneck" isn't a defensible conclusion without looking deeper. The compilation and fallback paths are visible in GZDoom's VM implementation.

The more useful first question was: which part of the model takes the time?

A recorded one-position forward-pass profile covered all 24 LFM3 layers and the final vocabulary projection, with cold weights and the normal four-millisecond scheduling budget. The timed stages were:

99.47% of the timed stages were dense matrix–vector work. That's the important denominator: it is not 99.47% of total wall time. This was also a one-position profile, so the attention reductions had almost no context to process. Longer contexts will change their costs.

Plotted from the recorded profile. Percentages describe timed stages for one input position, excluding loading and gaps between work slices.

Those timed stages took roughly 1.89 seconds, but the corresponding inference elapsed time was 6.71 seconds. The difference includes scheduling, rendering, and other work outside the measured stages. Loading calls accounted for 4.80 seconds within 8.18 seconds of loading wall time. The entire recorded process took 16.83 seconds, peaking around 2.13 GiB of resident memory.

It's tempting to respond by giving computation more time per callback. You would probably get answers sooner. You would also recreate the freezing you just spent so much effort fixing. Engineering: the art of choosing which problem you'd rather have when you can't fix both.

For a prompt with many tokens, there's another expensive part: prefill. The model has to process the prompt before it can produce the first answer token. This implementation processes prompt positions sequentially, so even a short question wrapped in chat-template tokens can take a while. There may be useful work to do in batching parts of prefill and improving weight reuse.

The profile tells us where to investigate next. It does not yet tell us whether the matvec kernel is limited by memory bandwidth, array access, conversions, JIT code quality, or a mixture of those. To answer that, I'd need more focused profiling rather than simply glaring at the VM.

So, Doom runs a language model now

Let's count the damage.

A one-GiB matrix made it through the ZScript runtime and produced about 809 million MAC/s in a dedicated benchmark. A GPT-2 attention block matched its reference checks. A complete Qwen2.5-0.5B-Instruct port accepted arbitrary console prompts and generated answers, even if it occasionally mistook a legendary weapon for a television programme. Then LFM3-1.7B joined in with convolutions, two kinds of attention, bigger weights, expanding caches, and an even stranger idea of British military hardware.

The implementation isn't a new fast inference engine. It isn't replacing llama.cpp. The game can still hitch while loading, and a full answer can take an extraordinary amount of time. The logs validate specific calculations and test cases, not every possible prompt on every machine.

But the central experiment worked. GZDoom loaded model weights from a PK3, tokenized console input, ran the forward pass in ZScript, produced tokens, and returned control to the game. No external inference server required.

I'm not claiming to have hand-coded the whole thing. An local AI coding agent assisted me; the idea was to push an absurd inversion far enough that it became a testable system. What I got out of it was a much better appreciation for how much machinery sits between "multiply some numbers" and "a language model answered my question": tokenizer rules, floating-point representation, quantization, caches, callbacks, and a long series of little details that don't care how funny the original idea was.

The next obvious targets are the dense kernels and prefill. Maybe they can be made faster without turning this into a native C++ project. Maybe the VM will have the last laugh. Either outcome should be interesting to measure.

For now, Doom can run an LLM.

The demons have been waiting patiently for me to finish talking to it. The BFG, meanwhile, appears to have moved to Britain. I forgot to ask whether it was a nuclear research facility or a television series this time.

The code is here: doollm on GitHub.