What fits in 96 GB of VRAM
Data updated . Last reviewed the same day, when this catalog snapshot was written.
Usable memory here is 91.2 GB, which is 95% of 96 GB. 43 of 49 catalog models fit at Q4, Q8, or FP16. No GPU in this catalog ships with exactly 96 GB. The list is the memory budget, not a product roundup. 4 of the fits do not fit the 64 GB tier.
Models
Sorted by the highest quant that fits, then by headroom. Tok/s is blank because this tier has no catalog GPU.
| Model | Best quant | Estimate | Headroom | Est. tok/s |
|---|---|---|---|---|
| Llama 3.2 1B Instruct | FP16 | 3.0 GB | 88.2 GB | — |
| Llama 3.2 3B Instruct | FP16 | 7.7 GB | 83.5 GB | — |
| Phi-3.5 mini | FP16 | 9.2 GB | 82.0 GB | — |
| Phi-4 mini | FP16 | 9.2 GB | 82.0 GB | — |
| Gemma 3 4B | FP16 | 10.3 GB | 80.9 GB | — |
| Mistral 7B Instruct v0.3 | FP16 | 17.4 GB | 73.8 GB | — |
| Qwen2.5 7B Instruct | FP16 | 18.3 GB | 72.9 GB | — |
| Gemma 4 E4B | FP16 | 19.2 GB | 72.0 GB | — |
| DeepSeek R1 Distill Llama 8B | FP16 | 19.3 GB | 71.9 GB | — |
| Llama 3.1 8B Instruct | FP16 | 19.3 GB | 71.9 GB | — |
| Qwen3 8B | FP16 | 19.7 GB | 71.5 GB | — |
| Ministral 3 8B | FP16 | 21.4 GB | 69.8 GB | — |
| Gemma 2 9B | FP16 | 22.2 GB | 69.0 GB | — |
| Gemma 4 12B | FP16 | 28.7 GB | 62.5 GB | — |
| Gemma 3 12B | FP16 | 29.2 GB | 62.0 GB | — |
| Mistral Nemo 12B | FP16 | 29.4 GB | 61.8 GB | — |
| Ministral 3 14B | FP16 | 33.5 GB | 57.7 GB | — |
| Phi-4 | FP16 | 35.2 GB | 56.0 GB | — |
| Qwen3 14B | FP16 | 35.4 GB | 55.8 GB | — |
| Qwen2.5 14B Instruct | FP16 | 35.4 GB | 55.8 GB | — |
| gpt-oss-20b | FP16 | 50.2 GB | 41.0 GB | — |
| Mistral Small 3.2 24B | FP16 | 57.6 GB | 33.6 GB | — |
| Gemma 4 26B-A4B | FP16 | 61.9 GB | 29.3 GB | — |
| Gemma 2 27B | FP16 | 65.3 GB | 25.9 GB | — |
| Gemma 3 27B | FP16 | 65.8 GB | 25.4 GB | — |
| Qwen3.6 27B | FP16 | 66.7 GB | 24.5 GB | — |
| Qwen3.8 27B | FP16 | 66.7 GB | 24.5 GB | — |
| Qwen3 30B-A3B | FP16 | 73.3 GB | 17.9 GB | — |
| GLM-4.7-Flash | FP16 | 74.9 GB | 16.3 GB | — |
| Gemma 4 31B | FP16 | 75.1 GB | 16.1 GB | — |
| Qwen3 32B | FP16 | 78.6 GB | 12.6 GB | — |
| DeepSeek R1 Distill Qwen 32B | FP16 | 78.6 GB | 12.6 GB | — |
| Qwen2.5 32B Instruct | FP16 | 78.6 GB | 12.6 GB | — |
| Qwen3.6 35B-A3B | FP16 | 86.3 GB | 4.9 GB | — |
| DeepSeek V4 Flash | Q8 | 4.5 GB | 86.7 GB | — |
| Mixtral 8x7B | Q8 | 50.7 GB | 40.5 GB | — |
| gpt-oss-120b | Q8 | 63.7 GB | 27.5 GB | — |
| Llama 3.3 70B Instruct | Q8 | 75.0 GB | 16.2 GB | — |
| DeepSeek R1 Distill Llama 70B | Q8 | 77.7 GB | 13.5 GB | — |
| Qwen2.5 72B Instruct | Q8 | 80.2 GB | 11.0 GB | — |
| Llama 4 Scout | Q4 | 65.4 GB | 25.8 GB | — |
| Mistral Small 4 | Q4 | 74.0 GB | 17.2 GB | — |
| Mixtral 8x22B | Q4 | 87.5 GB | 3.7 GB | — |
Next misses at Q4: Qwen3 235B-A22B (144 GB), Llama 4 Maverick (243 GB), Llama 3.1 405B Instruct (244 GB), DeepSeek R1 (405 GB), Mistral Large 3 (407 GB).
GPUs at 96 GB
No catalog GPU has exactly 96 GB of VRAM.
Questions
- How many catalog models fit in 96 GB of VRAM?
- 43 of 49 fit at Q4 or better inside 91.2 GB usable (95% of 96 GB).
- Which catalog GPUs have 96 GB of VRAM?
- None. No GeForce or Radeon row in this catalog is exactly 96 GB.
- Is the tokens/sec column a benchmark?
- No column is filled in, because no catalog GPU has 96 GB to borrow a bandwidth from.