How does it fit a large model into limited memory?
An MoE model holds many sub-networks called experts, and each token uses only a few of them. Qwen3.8-Flash-Next, for example, uses 10 of its 24,576 experts for each token. PolyStrata keeps the most-used experts next to the compute, and the rest wait one level further away.
Each token needs only a few experts, so the most-used ones stay next to the compute.
- On an NVIDIA PC, GPU memory holds the most-used experts. The CPU (central processing unit) computes the rest from system RAM at the same time.
- On an Apple silicon Mac, unified memory holds the model when it fits. When memory runs out, the SSD serves the rest, and the engine reads the next layer ahead.
How fast is it?
On an RTX 3060 with 12 GB and 64 GB of RAM, Qwen3.8-Flash-Next writes 45.5 tokens per second. On an RTX 5090 with 32 GB, it writes 179. A token is about three quarters of a word, and 60 tokens per second is faster than you can read. A token that needs an expert from RAM or the SSD takes longer than a token that stays on the GPU. So speed depends on how much of the model fits on your card. The PolyStrata project measured these numbers with its own benchmark. On a laptop with an 8 GB card, PolyStrata read prompts 2.7 to 3.5 times as fast as llama.cpp, and wrote 1.5 to 1.8 times as fast. See Benchmarks for each model, and the PolyStrata results for how they were measured.Does PolyStrata change the quality score?
PolyStrata does not change a model’s quality score, but it changes which model your machine can run. The SimplexAI score follows the model and its quantization, not the engine. The gain is indirect. Without PolyStrata, a 24 GB card holds a 27B model, which scores 54.0. With PolyStrata and 64 GB of RAM, the same card runs Qwen3.8-Flash-Next, which scores 59.3. Speed and score are separate numbers. The speeds on this page are measurements from the PolyStrata project. The scores come from third-party test results, adjusted for quantization. See how the score is built.Keep learning
Local models
See which local model Linger plans for your hardware.
Benchmarks
Compare speed and quality on real hardware.