Dikun International

A million tokens
on a single card

LOMONOSOV ZENIT 27B «ALTAY» holds 1,010,000 tokens of context literally — no retrieval, no summarisation, no offload to disk. On one RTX 5090.

1,010,000tokens of context
27.3 Bparameters
16.4 GiBof weights
32 GBcard is enough

How it fits

An ordinary key–value cache at a million tokens would not fit in any consumer card. Here it is compressed to three bits by our own TurboQuant codec — 12,785 bytes per token, so the whole million lives in 13 gigabytes.

The weights are compressed unevenly, by what each part tolerates: text layers in four-bit NVFP4, attention in FP8, embeddings in eight bits. Twenty-seven billion parameters in 16.4 GiB, on disk and in memory alike.

whatparameters
four-bit NVFP424,054,511,616
eight-bit1,504,074,752
FP8 attention, norms, biases1,716,395,256
total27,274,981,624

A quarter of a million holds as well as thirty thousand

Fourteen facts are planted at known depths in a corpus, then asked about one by one. Between thirty-three thousand tokens and a quarter of a million the curve is flat: an eightfold increase in length costs the model nothing.

33,334 92.9%
250,334 92.9%
1,005,334 14.3%

The quarter-million figure is not a gift of the architecture but the result of work. RoPE frequency pairs are compressed in stages: pairs 15–21 of 32 by half above 196,608, the rest by a quarter above 262,144. That took retention from 71.4% to 92.9%, validated on a corpus the map was not chosen against, and repeated byte for byte. No training was required.

About the million, plainly. At full length the model finds two facts out of fourteen. The window is literal: the text goes in whole and the model answers from it — but we do not promise reliable retrieval across the whole million. The limit is architectural: 48 of the 64 layers carry a fixed-size state with no positions in it at all. Seven treatments have been tried and measured; the work continues.

Reads a million in twenty-two minutes

RTX 5090, stock clocks, one sequence.

contextread timetokens/s
131,07234 s3840
262,144109 s2400
524,2886.4 min1375
1,010,00022 min768

The slowdown with length is attention's quadratic cost rather than an implementation flaw: 381 seconds at 524,288 is faster than a pure quadratic extrapolation from 262,144.

It picks the window for your card by itself

No flags. The model looks at the memory actually free and takes the widest window that will genuinely start — and writes its choice to the log in one line.

cardfreewindow
RTX 5090, nothing else running32 GB1,010,001
RTX 5090 with a desktop session−0.7 GB393,216
RTX 409024 GB262,144

The thresholds are measured, not estimated: memory was held by another process at nine levels, and at each one the bare command was given. There is no level at which the model would choose a window and then fail to start.

Every number is checkable

A receipts/ folder ships with the model. Each receipt records the question it was answering, the hardware, the runtime version — and what it does not establish. The probes that produced the numbers on this page sit next to them and run with one command.

The rule is simple: no figure enters the card without a receipt. Where a measurement has refuted something published earlier, a dated correction stands in its place with the old value, so it is visible what changed and why.

What it needs

A card with compute capability 8.9 or above — 5090, 4090, L40S, L4. It will not load on Ampere or older: the FP8 scheme requires it. GGUF and Ollama are not supported. The first run on a new machine compiles FP4 kernels for your card, about four minutes, once.

Ddavidich/LOMONOSOV-ZENIT-27B-1M-INDEV · receipts · full model card

Every number on this page was measured on a rented RTX 5090 at stock clocks. Where a figure here disagrees with a receipt, the receipt is right.