> ## Documentation Index
> Fetch the complete documentation index at: https://docs.heylinger.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Local models

> See which local model Linger plans for your hardware, how you approve it, and how PolyStrata fits a large model into limited memory.

Linger can run every layer on your own machine, from the agent to its memory. Claude Code, Codex, and Pi run as they always do, while a local gateway points them at models on your machine. Your wiki, memory, sessions, and files stay on your disk. A remote model, or the subscription you already pay for, joins only when you ask.

One install sets up a language model and a small decision model. They stay quick even on a fanless laptop. Local models run on an Apple Silicon Mac, or on an NVIDIA GPU (graphics processing unit) on Windows or Linux.

This page explains how the planner picks model settings, which machines it refuses, and how you approve the plan. It also explains how the **PolyStrata** engine fits a large model into limited memory. To learn how agents share one local model, read [Model scheduling](https://docs.heylinger.com/models/model-scheduling.md).

![A 125B model runs on a 12 GB card because the experts that do not fit move to slower memory.](https://mintcdn.com/lev9-doc/iWlaFXt9R5YUC2Ro/images/feature-local-moe.gif?s=b99108bd3d48b57f35ef847e69bb9e50)

PolyStrata keeps the most-used experts next to the GPU.

## How does the planner pick settings for my machine?

The planner chooses the hardware use and the model settings together. Layer by layer, your platform, free memory, and bandwidth narrow the catalog. At the end, one model remains, with one quantization, context window, and KV (key-value) cache precision. The planner then packs the smaller models around it.

These terms appear in the table below.

* **Quant** is the quantization of the model weights. A smaller quantization uses less memory.
* **Context** is the context window, in tokens.
* **KV** is the precision of the key-value cache.
* **Multi-token prediction (MTP)** lets the model draft a few tokens that one full pass checks.

| Machine | Model | Quant | Context | KV | MTP | Decision model |
| - | - | - | - | - | - | - |
| NVIDIA 24 GB | Qwen3.8-27B | Q4\_K\_M | 128K | q8 / q4 | On | StartLux 0.8B |
| NVIDIA 32 GB | Qwen3.8-27B | Q4\_K\_M | 200K | q8 / q8 | On | StartLux 4B |
| NVIDIA 48 GB+ | Qwen3.8-27B | Q4\_K\_M | 200K | q8 / q8 | On | StartLux 4B |
| Mac 32 GB | Qwen3.8-27B | IQ3\_S | 128K | q4 / q4 | Off | StartLux 4B |
| Mac 48 GB | Qwen3.8-27B | Q4\_K\_M | 200K | q8 / q8 | Off | StartLux 4B |
| Mac 64 GB+ | Qwen3.8-27B | Q4\_K\_M | 200K | q8 / q8 | Off | StartLux 4B |

Every row is the planner's own output for that machine. When it must choose, the planner prefers quality over speed. That is why a 32 GB Mac gets a smaller quantization and no MTP. On a Mac, MTP and the prompt cache stay off.

## Which machines does the planner refuse?

The planner refuses small machines instead of squeezing a model onto them.

* NVIDIA cards with 16 GB of memory or less.
* Macs with 24 GB of memory or less.

On a refused machine, use the subscription of your Runtime for the agents' work. See [Choose a runtime](https://docs.heylinger.com/models/choose-a-runtime.md).

## How do I approve and install the plan?

The planner proposes a plan, and nothing deploys until you approve it. The installer offers local models that fit your machine. It asks before it downloads anything, and model deployment needs its own approval.

After you approve the plan, Linger tests each model. A model counts as installed only after it answers a real question.

You can also run the local model stack yourself from a checkout of the repository.

```bash theme={null}
scripts/installs/stack.sh capabilities --json
scripts/installs/stack.sh plan --offline
scripts/installs/stack.sh apply "/path/to/plan.json" --approve CONFIRMATION_ID
```

On Windows, use `scripts\installs\stack.ps1` with the same arguments. The `apply` command needs the full confirmation ID that `plan` prints. The scripts never plan or install on their own. The first run downloads a pinned toolkit and checks its SHA-256 checksum. Later runs start offline.

## How does PolyStrata fit a large model into limited memory?

PolyStrata keeps the most-used parts of the model next to the GPU and moves the rest one level away. This works because of how a mixture-of-experts (MoE) model computes.

An MoE model holds many sub-networks called experts, and each token uses only a few of them. PolyStrata keeps the most-used experts next to the GPU. The rest wait one level further away.

* **On an NVIDIA PC,** CUDA memory holds the most-used experts. The CPU (central processing unit) computes the rest from system RAM (random-access memory) at the same time.
* **On an Apple Silicon Mac,** unified memory holds the model when it fits. When memory runs out, the SSD (solid-state drive) serves the rest, and the engine reads the next layer ahead.

With this method, a 125B model can run on a 12 GB GPU. A token that needs an expert from RAM or the SSD takes longer than a token that stays on the GPU.

PolyStrata is open source under the MIT license and runs Qwen3.8-Flash-Next on the Strata engine. The source is at [github.com/simplexai-labs/PolyStrata](https://github.com/simplexai-labs/PolyStrata).

## Keep learning

> [card] [Model scheduling](https://docs.heylinger.com/models/model-scheduling.md)
>
> See how agents, approvals, and background work share one local model.

> [card] [Install the desktop app](https://docs.heylinger.com/install/desktop-app.md)
>
> Install Linger and choose local models during setup.

> [card] [Data and privacy](https://docs.heylinger.com/security/data-and-privacy.md)
>
> Check what stays on your machine and what leaves it.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.