AMD Instinct MI300X: specs, pricing and what it can run
The credible non-NVIDIA option: 192GB at 5.3 TB/s. The constraint is software maturity — ROCm support in serving stacks trails CUDA.
Specifications
| Spec | AMD Instinct MI300X |
|---|---|
| Architecture | CDNA 3 |
| Released | 2023 |
| Memory | 192 GB HBM3 |
| Memory bandwidth | 5,300 GB/s |
| Dense FP16 | 1,307 TFLOPS |
| FP8 tensor cores | Yes |
| Board power | 750 W |
| Interconnect | Infinity Fabric (896 GB/s) |
| Segment | datacenter |
What fits in 192 GB
Weights only, at 85% of nominal capacity to leave room for the KV cache and activations. Long-context serving needs materially more headroom than this table implies.
| Quantisation | Largest model (weights only) |
|---|---|
| FP16 | ~87B parameters |
| FP8 / INT8 | ~175B parameters |
| 4-bit | ~350B parameters |
Rental cost
Indicative on-demand pricing runs roughly $1.80–$4.50 per GPU-hour depending on provider, region, and commitment. Spot and reserved capacity sit well below that band. These are ranges rather than quotes: street prices move week to week, so check live cloud price feed before budgeting.