DEEPSEEK V4.1 FLASH Q4 / 15 SEPTEMBER 2026
Prefill was reading
one drive.
43.62 against 16.23 tok/s prompt processing, measured against the pinned upstream engine with the same output hash on every arm.
What was tested
DeepSeek V4.1 Flash Q4, 518,596,067,328 bytes, SHA-256 a5e2e2c3ada4b2e9…. M5 Max, 128 GB. The candidate streams from four byte-identical verified replicas — the internal SSD plus three NVMe enclosures — split 10:6:6:4 by measured read speed. The control is the pinned upstream ds4 engine (bd66c40, no Argodrive code) on the internal SSD alone. Both produce the same output SHA-256, so this is a like-for-like comparison rather than a storage-layout screen.
Until this change, prefill read every routed expert through the memory map of the primary file. Adding enclosures could not help it: on a three-drive set the internal delivered 315.9 GB and each enclosure 0.2 GB, at 9.8 GB/s. The symptom was visible in our own earlier layout screen, where prefill sat flat at 67.87, 66.72 and 66.61 tok/s while decode responded to every drive added.
The second half: bytes nobody needed
A 512-token chunk routes to 187 of 384 experts per layer, 48.8%, but the layer-major sweep reads all 384. Measured on unmodified upstream: 301.9 GB read where the model touches 149.2 GB — 2.02× amplification, on one internal SSD with no replicas involved. That finding is independent of this hardware and applies to any engine streaming an MoE from disk.
Measurement
512 prompt tokens, 200 generated, greedy, context allocation 4,096, a fresh process and empty expert cache per arm, a 100 ms physical-device sampler and a swap guard. Control and candidate were interleaved. Prompt processing is prefill tokens divided by prefill seconds; generation excludes prefill and startup; steady excludes the first response.
| Arm | Side | Prompt processing | Generation | Steady | Output SHA-256 |
|---|---|---|---|---|---|
| p3-up-1 | upstream, one drive | 16.37 | 10.20 | 10.65 | a548d58919… |
| p3-ch-1b | ours, three drives | 45.20 | 16.87 | 17.49 | a548d58919… |
| p3-up-2 | upstream, one drive | 16.09 | 9.95 | 10.56 | a548d58919… |
| p3-ch-2 | ours, three drives | 42.03 | 15.99 | 17.26 | a548d58919… |
| up1d-1 | upstream, one drive | 17.04 | 10.19 | 10.61 | a548d58919… |
| sw1d-a | ours, one drive, no replicas | 27.12 | 13.39 | 14.53 | a548d58919… |
| up1d-2 | upstream, one drive | 15.97 | 9.67 | 10.27 | a548d58919… |
| sw1d-b | ours, one drive, no replicas | 28.96 | 13.59 | 14.23 | a548d58919… |
Three drives: medians prompt processing 16.23 → 43.62 tok/s (2.69×), steady decode 10.61 → 17.38 (1.64×). One drive, no replicas: 16.50 → 28.04 (1.70×), steady 10.44 → 14.38 (1.38×). The one-drive pair isolates the selective-expert read, which needs no replicas at all; the rest is what splitting each read across devices buys. All eight arms produced the same output SHA-256.
Which change earns which number
The prompt-processing gain is the prefill work described here. The generation gain is not: decode was already reading from several drives before this change, and the candidate also carries a larger expert cache (4,200 entries against the automatic 3,688). Both are reported together because they were measured together, not because the prefill work made decode faster.
Why it is applied to short prompts only
Reading only the selected experts means the router must run before any expert byte is read, which gives up the overlap that loads the next layer while the current one computes. That trade only pays while the expert union is small, so it is applied to chunks of 512 tokens or fewer:
| Chunk | Experts touched | Coverage | Effect |
|---|---|---|---|
| 512 tokens | 187.4 of 384 | 48.8% | selective staging wins, +7.2% |
| 1,024 tokens | 230.8 of 384 | 60.1% | selective loses, −13% |
| 2,048 tokens | 271.5 of 384 | 70.7% | selective loses, −33% |
Prompts under 256 tokens never reach this path at all: the engine keeps them token-major, where a full layer sweep would read 305.8 GB to serve a hundred tokens.
Why more drives help, when bandwidth is not the limit
Decode uses 19% of the drives' combined bandwidth. It is not bandwidth-bound. A streamed MoE waits for one read to finish: split that read across more drives and each slice is smaller, so the slice that lands last lands sooner. A fourth drive moved peak throughput hardly at all and still bought +6.6% prefill and +2.9% decode at 512 tokens, and +15.4% prefill at 2,048. Weights of 10:6:6:4 beat the capability-proportional 10:5:5:4 by 2.6% on decode, because the internal drive also serves every Engram row read — traffic the expert split does not see.
What did not work
Predicting the next layer's experts to prefetch them: the predictor is good, 4.24 of 6 correct over 7,800 layer-predictions, and cuts demand misses by a third, but the prediction itself costs 3% — an extra matmul, a router select and two readbacks per layer, forty times per token — so every prefetch scheme started 3% behind. Warming the page cache for predicted experts: mechanically real, a warmed F_NOCACHE read is 2.6× faster (1.98 → 0.75 ms per expert), but it reads every byte twice and spends about 4 ms of drive time to save 1.2 ms. Pipelining staging against compute by token chunks: capped at +2.1%, because the first 256-token chunk already needs 85% of the layer's experts.
Limits
One machine, one model, one prompt. Four byte-identical copies of a 518 GB file, about 2 TB of duplicated storage. Replicas rather than a stripe because the drives differ (12.9 / 7.08 / 7.08 / 5.25 GB/s) and replicas let each take a share proportional to its measured speed, which a stripe cannot do and which survives losing a drive. The candidate numbers here were measured after several hours of continuous running; on a rested machine the same configuration has measured 45.20 tok/s prompt processing, and the paired figure is published in preference to it.
Download sanitized measurements ↓
Data transcribed from the local qualification logs, sampler CSVs and output-hash files. The JSON carries the exact observations and conditions; it excludes machine paths, serials, prompts and model files.
Code. The tooling this came out of is ArgoDrive (replica layout, weighted split-read balancer, per-read and per-phase per-drive instruments). The engine changes are on the argonaut-v41-benchmark branch of our fork of antirez/ds4; the single-drive gain needs only commit 38e200a.