Trillion-Parameter Open Models: GPU Requirements for Kimi and Muse

To run Kimi K3.1 Pro you need memory for 2.8 trillion parameters, about 1.49 TB for the weights in MXFP4 once block scales are counted, before any cache. Muse Spark Open, at 816 billion parameters, needs roughly 408 GB at plain 4-bit and 816 GB at 8-bit. Active parameters set speed, not memory. These are our arithmetic from the model cards, not vendor sizing guidance.

Two large open-weight models landed within three weeks of each other. Meta published Muse Spark Open on 15 October 2026, and Moonshot has also published the Kimi K3.1 Pro weights. This post works out what each one needs, shows the inputs, and says where the sources are silent.

What do the two model cards say?

We read both primary pages on 9 November 2026. Here is what each states, and what it leaves out.

Three gaps matter. The Kimi card gives no checkpoint size in gigabytes and no GPU recommendation. It does not list a count of key-value heads, so we cannot compute its cache. Meta's page does not say which precision the released weights use. Where a number below depends on one of these, we say so.

The Kimi card lists vLLM, SGLang and TokenSpeed as recommended engines. Our earlier Kimi K3 hardware post covers the previous generation, so we do not repeat it here. The Kimi K3 download and self-host guide covers the mechanics of fetching weights.

How much VRAM do you need to run Kimi K3.1 Pro?

Start with the weights. Memory for weights equals parameters multiplied by bytes per parameter. For the fully GPU-resident deployments sized below, every expert's weights occupy GPU memory, because any token may route to any of them. CPU offloading can reduce the GPU memory required, but transfers can substantially slow inference (vLLM offloading documentation).

Our arithmetic: 2.8 trillion x 2 bytes is 5.6 TB, and halving at each step gives 2.8 TB and 1.4 TB. For Muse, 816 billion x 2 bytes is 1,632 GB. MXFP4 stores one 8-bit scale for each block of 32 values, so each parameter costs 0.5 + 1/32 = 0.53125 bytes. That gives 2.8 trillion x 0.53125 = 1,487.5 GB for Kimi (our arithmetic, per NVIDIA's MXFP4 layout documentation). The Muse figure applies the same layout, but Meta does not say its format. Higher-precision tensors, padding and runtime allocations can add more.

For Kimi K3.1 Pro the native format matters most. The card says weights are MXFP4, so the MXFP4 row is the realistic starting point, not a squeezed variant. It is an idealised all-MXFP4 figure. The card lists F32 and BF16 tensors too, so the real checkpoint will be larger, and the Hub shows only "2.8T params". Check the file listing before you plan.

The answer to the question is: at least about 1.49 TB of GPU memory just for weights, and more once you add the cache, activations and framework overhead. No single eight-GPU H200 node holds that, and a DGX B200 node (1,440 GB) falls just short. We cover which nodes do in the node table below.

Can Muse Spark Open run on a Mac?

Meta says yes, with conditions. Its page says you can "run on your desktop using two Apple M5 Ultra Mac Studios with the latest MLX framework". It states that the M5 Ultra's integrated GPU can use "up to 512 GB per Mac Studio, so two machines can hold an entire model", and it quotes 1.2 TB/s of memory bandwidth. TechCrunch, on 15 October, reported the same Mac angle and added that the requirement is "at least 1 terabyte of RAM, per Engadget". That RAM figure is reported, not from Meta's page.

Check it against the arithmetic. Two Mac Studios give 1,024 GB (our arithmetic: 2 x 512). Muse Spark Open at 8-bit needs 816 GB for weights. That leaves about 208 GB for the operating system, the cache and everything else. At BF16 the weights alone are 1,632 GB and would not fit. Our inference: the reported 1 TB figure is consistent with a weight format near 8-bit plus working memory. Meta does not state the format, so treat this as inference.

Speed is a separate question. Decoding reads the active weights for each token. At 8-bit, 42B active parameters is about 42 GB per token. At the quoted 1.2 TB/s, a single machine's bandwidth would cap decoding near 29 tokens per second (our arithmetic: 1,200 divided by 42). That ignores the cache, the link between the two machines and any batching. Treat it as a loose ceiling for a single user, and expect real numbers to differ. We have not run the model and publish no measurement.

So a Mac is plausible for one developer exploring the model. It is not a plan for serving a team. For that, see the node table.

Do MoE models need less memory?

No. Mixture-of-experts models need less compute per token, not less memory. That is the central point for planning.

A mixture-of-experts model routes each token to a few experts. Kimi K3.1 Pro selects 16 of 896 experts, plus 2 shared ones, per the card. Its 104B active parameters are 3.7% of 2.8T (our arithmetic). Muse Spark Open's 42B active are 5.1% of 816B (our arithmetic). Each step reads only the active slice, so decoding is faster than a dense model of the same total size would be.

But you cannot know in advance which experts a token needs. Across a batch of requests, different tokens pick different experts, and over time all of them get used. So all weights must stay in fast memory. The practical rule:

• Total parameters set memory. This decides how many GPUs you must buy or rent.
• Active parameters set speed. This decides tokens per second once the model fits.
• Cache sets concurrency. This decides how many users share the node.

Per-token weight traffic, at 4-bit, is about 52 GB for Kimi K3.1 Pro (our arithmetic: 104B x 0.5 bytes). NVIDIA lists 64 TB/s of HBM3e bandwidth for an eight-GPU DGX B200. A perfect single-stream ceiling would be 64,000 divided by 52, about 1,230 tokens per second. This is an idealised bandwidth bound, not a throughput forecast. Capacity decides whether the deployment fits. Achieved memory bandwidth, communication and cache traffic then limit decoding speed. NVIDIA's measured example of an MoE model on DGX Spark found decoding memory-bound, for one model at batch size 1, so do not assume bandwidth is free.

What does the KV cache add at long context?

The cache stores the keys and values for every token already seen, so the model does not recompute them. It grows with context length and with the number of concurrent sessions. At a context of a million tokens it can rival the weights.

The general formula for a standard attention layer is:

Cache bytes = 2 x layers x KV heads x head dimension x bytes per value x tokens x sessions

The 2 covers keys and values. Doubling the context doubles the cache. Doubling concurrent sessions doubles it again. Grouped-query attention reduces the KV head count, and lower-precision cache values reduce bytes per value.

We do not give a number for either model. The Kimi card lists 93 layers and 96 attention heads, but not the KV head count or head dimension. It also mixes attention types: 69 KDA layers and 24 Gated DSA layers, and the card does not tell us how much cache each kind keeps. Meta's page gives no layer or head figures. Any figure we printed would be invented.

What you can do is this. Inspect the engine's architecture-specific cache implementation. For Kimi, account separately for KDA recurrent state and Gated DSA cache storage, then measure memory at your target context and concurrency. The formula above applies only to standard attention layers. The deployment engines expose a cache memory setting, and the number it reports under load is the one to trust. Revolut's talk, in our case study, makes the point from operations. The speaker says "256 is really what you need" for context, and that GLM fits on four-way tensor parallelism but lacks the cache. Provision for a million-token context deliberately.

How many nodes does each model need?

Node memory comes from NVIDIA's own pages, fetched on 9 November 2026. The DGX B200 page lists 1,440 GB of total GPU memory. The DGX B300 page lists 2.1 TB. The H200 page lists 141 GB per GPU, so an eight-GPU node holds 1,128 GB (our arithmetic: 8 x 141). NVIDIA's Vera Rubin NVL72 page lists 288 GB of HBM4 per GPU and 20.7 TB per rack, but that system is a rack-scale product, so we leave it out of the per-node table.

Each cell shows the minimum number of nodes by weights alone, with the spare memory in brackets. Spare memory is what remains for cache and overhead. Our arithmetic throughout.

How to read it. A cell with under about 100 GB spare is not usable in practice. Kimi's MXFP4 weights do not fit one B200 node at all (1,487.5 GB against 1,440 GB), so two nodes are needed there. On a single B300 node, 612.5 GB spare is workable, but you would then measure concurrency under load, as above. The Muse 4-bit row uses plain 4-bit (408 GB), which ignores block scales; add roughly 6% (our arithmetic: 0.53125 divided by 0.5) if its format uses them.

Two limits apply. These counts ignore interconnect, since spreading one model across nodes adds network traffic. They also ignore whether your engine supports the format on your hardware, so check its documentation before buying.

For Muse Spark Open the picture is simpler. Meta has not said which format the weights ship in, so the right row is unknown until you open the repository. At 8-bit it fits one node of any of the three types.

Is it cheaper to self-host a trillion-parameter model?

Only if you keep the hardware busy. The cost of a node is fixed, and the cost per token is that fixed cost divided by the tokens you actually serve.

Here is a worked example with every input visible. These are assumptions for illustration, not quotes.

• Assumed rental price: $3 per GPU-hour.
• One eight-GPU node for a 730-hour month: 8 x $3 x 730 = $17,520.
• Two nodes: $35,040.
• Assumed API price for a comparable model: $2 per million tokens, blended input and output.

At those assumptions, one node matches the API at 8.76 billion tokens a month (our arithmetic: $17,520 divided by $2 per million). Two nodes match it at 17.52 billion. Below those volumes the API is cheaper. We have not fetched a Moonshot or Meta API price here, so replace the $2 with the real rate for the model you are comparing.

Utilisation moves the answer more than any other input. If a cluster serves only 20% of what it could, each million tokens costs five times the full-use figure (our arithmetic: 1 divided by 0.2). This is the lesson in Revolut's talk. The company says its bill fell to a tenth, on a deployment that serves roughly 2,000 engineers through 56 GPUs. Those are the company's own figures from a conference talk, with its own caveat that they describe its utilisation, not a portable price. Our analysis of what transfers is in the Revolut case study.

Add what the arithmetic leaves out: engineers to run it and spare capacity for failures. For many teams the sensible position is to start on an API and move only when volume is high and steady.

Licences also matter. Muse Spark Open carries "limited commercial terms", and the Kimi licence terms are in a separate file we did not read. Read both before you plan a hosted service. Our licence comparison goes through the differences, and the Beam GPU memory post applies the same method to a 501B model.

What should you do before buying hardware?

Five steps, in order.

1. Fetch the real checkpoint size. Add up the file sizes in the repository. Do not trust a parameter count.
2. Pick the precision your engine supports. Confirm it on your hardware generation.
3. Measure the cache. Use the engine's cache implementation for each architecture, then test at your target context and concurrency.
4. Measure on rented capacity first. One week of rental costs far less than a wrong purchase.
5. Compare against the API at your volume. Use your measured tokens, not a forecast.

Where Swfte sits

If you serve a model like this yourself, you still want an API in front of it, plus a fallback for when your cluster is busy or down.

Swfte Connect offers one API across many providers. You can bring your own keys from any provider, or use Swfte's pooled access. When a provider is out of funds, rate-limited or down, Connect can fall back to another provider in the chain. When a workspace reaches a usage cap, Connect can move its requests to a cheaper model. These are Built.

The enterprise deployment generator produces Terraform, Helm and Docker Compose bundles. Generating one requires a licence key, and dedicated deployment is available on request. Built: Connect supports BYOK with an optional base URL, allowing a custom OpenAI-compatible endpoint, such as your own vLLM server, behind its routing, failover, usage caps, cost tracking and audit event stream. Compatibility with these particular checkpoints still needs testing, and we have not tested Kimi K3.1 Pro or Muse Spark Open on Connect, so we make no claim about either.

The decision rule: size by total parameters, serve by active parameters, and buy only for the load you can keep busy.

---

Related: Kimi K3 and Inkling GPU requirements, Kimi K3 open weights download and self-host guide, How Revolut cut AI coding costs, Reflection AI Beam self-host guide, Open-weight licences compared.

Sources: Hugging Face, Kimi K3.1 Pro model card · Meta, Muse Spark Open · TechCrunch, Meta launches an open-weight model that runs on Macs (15 October 2026) · NVIDIA Technical Blog, 3 ways to boost MoE inference on DGX Spark · vLLM, engine arguments (CPU offload) · NVIDIA NeMo, MXFP4 layout · NVIDIA, DGX B200 · NVIDIA, DGX B300 · NVIDIA, H200 · NVIDIA, Vera Rubin NVL72 · AI Engineer NYC 2026, Revolut talk · accessed 9 November 2026.

Method note: specifications were re-read on the cited pages on 9 November 2026. All memory sizes, node counts and cost figures are our arithmetic from those specifications and from stated assumptions. They are not vendor sizing guidance, and we ran neither model. The 1 TB RAM figure for Muse Spark Open is reported by TechCrunch citing Engadget and is not on Meta's page.