Deploy open-weight and private models on infrastructure you control

Serving-stack choice is made per deployment, against the model, the hardware and the traffic shape, and recorded in the deployment’s release record. <serving stacks supported in Swfte-managed deployments — founder to fill>.

If you already have a tuned vLLM or SGLang deployment, bring it. Connect fronts any OpenAI-compatible endpoint, so the governed path (routing, approved-model policy, cost tracking, logging) does not depend on which engine sits behind it.

Weights cost roughly two bytes per parameter in bf16, one byte in FP8 and about half a byte at 4-bit. That is plain arithmetic: a 70-billion-parameter model is about 140 GB of weights in bf16, about 70 GB in FP8 and about 35 GB at 4-bit, before KV cache and activations. An 80 GB H100 therefore cannot hold it in bf16 on one card, and can only hold the FP8 version with very little room for batching. Our GPU reference lists per-card memory so you can do this sum for your own model.

Mixture-of-experts models are the common trap. Only a fraction of parameters is active per token, which cuts compute, but every expert still has to sit in memory because the router picks different ones for each token. Active-parameter count is a speed claim, not a memory claim.

Quantization changes the model. The evaluation you ran on the bf16 checkpoint says nothing certain about the FP8 or 4-bit artifact you actually serve, particularly on refusal behavior and non-English text. Treat each quantized artifact as a new candidate and put it through the same gate.

Connect is Swfte’s universal model gateway: one API in front of many providers, with smart routing, automatic failover and cost tracking. For a private deployment it plays two roles. It is the single place where the approved-model policy is enforced on every request, and it is how you move traffic between an open-weight model on your own GPUs and a fallback without rewriting agents.

Scale on the signals that actually saturate an inference server: queue depth, time to first token and KV-cache utilization, not CPU. Keep a floor of warm replicas for latency-sensitive routes, because loading hundreds of gigabytes of weights is slow. Route low-risk, high-volume work to the smaller or cheaper model and keep the harder slice on the stronger one; the routing rule is policy, so it can be reviewed like any other change.

Routing is also your rollback lever. When the old and new revision are both deployed, shifting weight between them is a configuration change rather than a redeploy.

Models run on dedicated infrastructure: single-tenant, in a region you choose, with options from an isolated VPC to bare metal in your own data center. Data residency options and private, on-premises and hybrid deployment are part of the platform’s stated direction, and the exact regions and facilities available to you are agreed per engagement. <EU regions and facilities offered — founder to fill>.

Swfte provides the technical controls, governance mechanisms and evidence you need to deploy AI within your applicable regulatory, security and policy requirements. The exact posture depends on your use case, jurisdiction, deployment and configuration.

Swfte does not hold a SOC 2 report or an ISO 27001 certificate and does not sign HIPAA BAAs today; a SOC 2 Type I audit is in preparation, and the trust page has the current status. Your own regulatory position depends on your use case, jurisdiction and configuration.