Models

49 open models. Q4 and Q8 use a stored GGUF file size when one exists, otherwise the parameter formula. Open a model for the machines that fit.

Search

Family

Weights

Showing 49 of 49

DeepSeek V4 Pro
DeepSeekMoE
V4 Pro is about 1.6T weights. Included so the checker can say no honestly.
Params
1599B
Q4
959 GB
Context
1.0M
49B active per token. Memory uses the full weight count.
DeepSeek R1
DeepSeekMoE
Full R1. Hundreds of billions of weights; not a single consumer GPU.
Params
684B
Q4
405 GB
Context
131K
37B active per token. Memory uses the full weight count.
DeepSeek V4 Flash
DeepSeekMoE
V4 Flash MoE. About 13B active, 291B stored, so Q4 still wants a large unified-memory box.
Params
291B
Q4
175 GB
Context
1.0M
13B active per token. Memory uses the full weight count.
DeepSeek R1 Distill Llama 70B
DeepSeek
70B R1 distill. Needs a 48GB-class machine at Q4.
Params
70.6B
Q4
45.2 GB
Context
131K
DeepSeek R1 Distill Qwen 32B
DeepSeek
R1-style reasoning in a 32B dense model. Q4 fits 24GB.
Params
32.8B
Q4
22.0 GB
Context
131K
DeepSeek R1 Distill Llama 8B
DeepSeek
R1 reasoning distilled into Llama 8B. Same memory as Llama 3.1 8B.
Params
8.03B
Q4
6.0 GB
Context
131K
Gemma 4 31B
Gemma
Dense Gemma 4 31B. Q4 fits 24GB; FP16 does not.
Params
31.3B
Q4
19.7 GB
Context
262K
Gemma 3 27B
Gemma
Gemma 3's large dense model. Q4 is at home on 24GB.
Params
27.4B
Q4
16.5 GB
Context
131K
Gemma 2 27B
Gemma
Larger Gemma 2. Q4 needs about 16.3 GB usable, so a 16GB GPU comes up just short.
Params
27.2B
Q4
16.6 GB
Context
8K
Gemma 4 26B-A4B
GemmaMoE
Widely downloaded Gemma 4 MoE. About 3.8B active, 25.8B stored.
Params
25.8B
Q4
17.3 GB
Context
262K
3.8B active per token. Memory uses the full weight count.
Gemma 3 12B
Gemma
Gemma 3 12B. Q4 fits 12GB cards; Q8 wants 16GB.
Params
12.2B
Q4
7.3 GB
Context
131K
Gemma 4 12B
Gemma
Apache-licensed Gemma 4 12B with a 256K advertised context.
Params
12B
Q4
7.7 GB
Context
262K
Gemma 2 9B
Gemma
Gemma 2's mid size. Short 8K context, easy Q4 fit on 8GB.
Params
9.24B
Q4
6.8 GB
Context
8K
Gemma 4 E4B
Gemma
Efficient Gemma 4 around 8B. A practical laptop model.
Params
8B
Q4
5.1 GB
Context
131K
Gemma 3 4B
Gemma
Small multimodal Gemma 3. Comfortable on 8GB at Q8.
Params
4.3B
Q4
2.5 GB
Context
131K
GLM-4.7-Flash
GLMMoE
Mixture-of-experts. The safetensors files total 31.221B parameters. The model card calls it a 30B-A3B model, so the active count used here is that 3B designation. Context is config.json max_position_embeddings (202,752); the card introduction does not restate it. Attention is MLA, so the KV estimate uses the compressed cache, not a per-head KV.
Params
31.2B
Q4
18.8 GB
Context
203K
3B active per token. Memory uses the full weight count.
gpt-oss-120b
gpt-ossMoE
Larger gpt-oss MoE. You still load about 117B weights at Q4.
Params
117B
Q4
63.1 GB
Context
131K
5.1B active per token. Memory uses the full weight count.
gpt-oss-20b
gpt-ossMoE
OpenAI's smaller open-weight MoE. Weights are 20.9B; active parameters are a few billion.
Params
20.9B
Q4
11.8 GB
Context
131K
3.6B active per token. Memory uses the full weight count.
Llama 3.1 405B Instruct
Llama
Meta's largest Llama 3.1. Only half-terabyte Macs in this catalog have a chance at Q4.
Params
406B
Q4
244 GB
Context
131K
Llama 4 Maverick
LlamaMoE
400B-class MoE with 17B active. Memory follows total weights, not the active count.
Params
402B
Q4
243 GB
Context
1M
17B active per token. Memory uses the full weight count.
Llama 4 Scout
LlamaMoE
Mixture-of-experts. All 109B weights load; 17B are active per token. The 10M context is not covered by the 20% overhead.
Params
109B
Q4
65.4 GB
Context
10M
17B active per token. Memory uses the full weight count.
Llama 3.3 70B Instruct
Llama
Dense 70B. Q4 needs a 48GB-class pool; consumer 24GB cards do not fit it.
Params
70.6B
Q4
42.5 GB
Context
131K
Llama 3.1 8B Instruct
Llama
The default local Llama. Q4 fits on 8GB. FP16 needs a 24GB card.
Params
8.03B
Q4
4.9 GB
Context
131K
Llama 3.2 3B Instruct
Llama
Small Llama that still fits comfortably on an 8GB GPU at Q4.
Params
3.21B
Q4
2.0 GB
Context
131K
Llama 3.2 1B Instruct
Llama
A tiny instruct model for laptops and 8GB cards.
Params
1.24B
Q4
0.8 GB
Context
131K
Mistral Large 3
MistralMoE
675B total, about 41B active. The Hub listing has no safetensors total, so params use the published 675B figure.
Params
675B
Q4
407 GB
Context
262K
41B active per token. Memory uses the full weight count.
Mixtral 8x22B
MistralMoE
Large Mixtral. Q4 needs on the order of 85GB usable.
Params
141B
Q4
87.5 GB
Context
66K
39B active per token. Memory uses the full weight count.
Mistral Small 4
MistralMoE
119B MoE with about 6.5B active. Memory tracks the full weight set.
Params
119B
Q4
74.0 GB
Context
262K
6.5B active per token. Memory uses the full weight count.
Mixtral 8x7B
MistralMoE
Eight 7B experts. You load about 47B weights even though fewer are active.
Params
46.7B
Q4
27.5 GB
Context
33K
12.9B active per token. Memory uses the full weight count.
Mistral Small 3.2 24B
Mistral
Dense 24B. Q4 is the usual choice on a 24GB GPU.
Params
24B
Q4
15.7 GB
Context
131K
Ministral 3 14B
Mistral
14B Ministral 3. Q4 needs a 12GB card or a 16GB Mac.
Params
13.9B
Q4
9.6 GB
Context
262K
Mistral Nemo 12B
Mistral
12B Mistral with a 128K context. Q4 fits 12GB GPUs.
Params
12.2B
Q4
8.8 GB
Context
131K
Ministral 3 8B
Mistral
Small Ministral 3 from December 2025. Q4 fits an 8GB GPU. Q8 needs 12GB.
Params
8.92B
Q4
6.3 GB
Context
262K
Mistral 7B Instruct v0.3
Mistral
The classic small Mistral. Still one of the easiest 8GB-card models.
Params
7.25B
Q4
5.4 GB
Context
33K
Phi-4
Phi
Microsoft's 14.7B reasoning model. Short 16K context. Q4 fits 12GB.
Params
14.7B
Q4
10.6 GB
Context
16K
Phi-4 mini
Phi
Phi-4's small sibling. Q8 still fits 8GB.
Params
3.84B
Q4
3.6 GB
Context
131K
Phi-3.5 mini
Phi
Small Phi with a 128K context. Easy on an 8GB laptop GPU.
Params
3.82B
Q4
5.6 GB
Context
131K
Qwen3 235B-A22B
QwenMoE
Large Qwen3 MoE. Q4 still needs well over 100GB usable.
Params
235B
Q4
144 GB
Context
33K
22B active per token. Memory uses the full weight count.
Qwen2.5 72B Instruct
Qwen
Dense 72B Qwen. Same memory neighborhood as Llama 3.3 70B.
Params
72.7B
Q4
46.7 GB
Context
131K
Qwen3.6 35B-A3B
QwenMoE
Mixture-of-experts. The safetensors files total 35.952B parameters. The model card says 35B total and 3B activated, with a 262,144-token native context. The 3B figure is the card's activated count, not a separate tensor sum.
Params
36B
Q4
22.3 GB
Context
262K
3B active per token. Memory uses the full weight count.
Qwen2.5 32B Instruct
Qwen
The 32B class. Q4 fits a 24GB card. Q8 needs about 40GB usable.
Params
32.8B
Q4
22.0 GB
Context
131K
Qwen3 32B
Qwen
Dense Qwen3 32B. Plan on a 24GB card for Q4.
Params
32.8B
Q4
21.9 GB
Context
33K
Qwen3 30B-A3B
QwenMoE
MoE that activates about 3B parameters but loads the full 30.5B.
Params
30.5B
Q4
19.4 GB
Context
33K
3B active per token. Memory uses the full weight count.
Qwen3.6 27B
Qwen
Dense model. The safetensors files total 27.781B parameters. The model card states a 262,144-token native context, and says that window can be extended further. Fit figures on this site are estimates.
Params
27.8B
Q4
17.4 GB
Context
262K
Qwen3.8 27B
Qwen
Dense 27.8B with a long advertised context. Q4 is a 24GB-card model.
Params
27.8B
Q4
17.0 GB
Context
262K
Qwen2.5 14B Instruct
Qwen
Mid-size Qwen2.5. Q4 wants about 9GB usable.
Params
14.8B
Q4
10.6 GB
Context
131K
Qwen3 14B
Qwen
Dense Qwen3 14B. Q4 fits a 12GB GPU with a little room.
Params
14.8B
Q4
10.3 GB
Context
33K
Qwen3 8B
Qwen
Qwen3's small dense model, with a reasoning mode. Native context is 32K.
Params
8.19B
Q4
6.2 GB
Context
33K
Qwen2.5 7B Instruct
Qwen
Compact Qwen2.5 instruct. A strong fit for 8GB and 12GB cards.
Params
7.62B
Q4
5.2 GB
Context
131K