Qwen 3.8 27B, released by Alibaba’s Qwen team in August 2026 under the Apache-2.0 license, runs locally on Apple Silicon Macs at usable speeds using GGUF quantization. The 4-bit quantized version (Q4_K_M) is 16.5 GB on disk and needs approximately 20 GB unified memory — within reach of any Mac with 24 GB RAM or more.
What is Qwen 3.8 27B?
Qwen 3.8 27B is a 28-billion-parameter dense multimodal model from Alibaba’s Qwen team. It processes text, images, and video with a native 262,144-token context window, extensible to 1,000,000 tokens via YaRN RoPE scaling. Architecture: 64 layers using a hybrid of Gated DeltaNet (linear attention) and Gated Attention, classified internally as qwen3_5. It was the #1 trending model on Hugging Face in September 2026 with 4.96 million downloads in its first month.
Qwen 3.8 27B outperforms Claude Opus 4.6 Max on SWE-bench Pro (61.7 vs 53.4), LiveCodeBench v6 (90.3 vs 88.8), and IFBench (79.5 vs 62.5), according to benchmarks published on the Qwen 3.8 27B model card on Hugging Face.
How much RAM does Qwen 3.8 27B need on a Mac?
RAM requirements depend on quantization level. Apple Silicon Macs use unified memory shared between CPU and GPU, so “VRAM” and “RAM” are the same pool. These are the GGUF file sizes from unsloth/Qwen3.8-27B-GGUF on Hugging Face — actual runtime memory is approximately file size + 2-4 GB overhead for context and KV cache.
| Quantization | File size | Min RAM needed | Quality vs BF16 |
|---|---|---|---|
| Q2_K_XL | 9.8 GB | ~14 GB | Noticeably degraded |
| Q4_K_M | 16.5 GB | ~20 GB | Best speed/quality tradeoff |
| Q5_K_M | 19.8 GB | ~24 GB | Near-lossless |
| Q6_K | 22 GB | ~26 GB | Minimal loss |
| Q8_0 | 29 GB | ~33 GB | Effectively lossless |
| BF16 | 54.7 GB | ~59 GB | Full precision |
Recommendation: Q4_K_M at 16.5 GB is the standard choice. Fits on any Mac with 24 GB unified memory (M2 Pro, M3 Pro, M4 Pro, M4 Max, or any Max/Ultra chip). Unsloth’s Dynamic v3.0 quantization claims over 10% better accuracy at the same file size versus standard GGUF quantization.
Which Macs can run Qwen 3.8 27B?
| Mac | Unified RAM | Best quantization | Viable? |
|---|---|---|---|
| MacBook Air M2/M3 (8 GB) | 8 GB | None | No |
| MacBook Air M2/M3 (16 GB) | 16 GB | Q2_K_XL (degraded) | Barely |
| MacBook Pro M2 Pro/M3 Pro (18 GB) | 18 GB | Q4_K_M (tight) | Yes, with limited context |
| MacBook Pro M3 Pro/M4 Pro (24 GB) | 24 GB | Q4_K_M | Yes — sweet spot |
| MacBook Pro M3 Max/M4 Max (36 GB) | 36 GB | Q8_0 | Yes, near-lossless |
| MacBook Pro M4 Max (48 GB) | 48 GB | Q8_0 + long context | Yes, full capability |
| Mac Studio M2 Ultra (64 GB+) | 64-192 GB | BF16 full precision | Yes, no quantization needed |
How do you run Qwen 3.8 27B with Ollama on a Mac?
Ollama is the fastest path. Two commands:
# Install Ollama (if not installed)
brew install ollama
# Pull and run Q4_K_M from unsloth
ollama run hf.co/unsloth/Qwen3.8-27B-GGUF:Q4_K_MOllama serves an OpenAI-compatible API at http://localhost:11434 by default. To use it programmatically:
curl http://localhost:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "hf.co/unsloth/Qwen3.8-27B-GGUF:Q4_K_M",
"messages": [{"role": "user", "content": "Explain vector quantization in 200 words."}]
}'How do you run it with LM Studio?
LM Studio provides a GUI. Open LM Studio, search for unsloth/Qwen3.8-27B-GGUF, select Q4_K_M, click download. Load the model and start chatting. LM Studio also exposes a local API server at http://localhost:1234/v1 compatible with OpenAI client libraries.
How do you run it with llama.cpp directly?
For maximum control over inference parameters:
# Clone and build llama.cpp
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make -j
# Download the GGUF
huggingface-cli download unsloth/Qwen3.8-27B-GGUF \
Qwen3.8-27B-UD-Q4_K_M.gguf --local-dir ./models
# Run with Apple Metal GPU acceleration
./llama-cli \
-m ./models/Qwen3.8-27B-UD-Q4_K_M.gguf \
-ngl 99 \
-c 8192 \
-p "Explain vector quantization in 200 words."-ngl 99 offloads all layers to the Metal GPU. -c 8192 sets 8K context. Increase -c for longer conversations at the cost of more memory.
What inference speed should you expect on Apple Silicon?
Speeds vary by chip and quantization. These are approximate token generation rates for Q4_K_M based on community reports and Hacker News benchmarks (September 2026):
| Chip | Tokens/sec (Q4_K_M) | Tokens/sec (Q8_0) |
|---|---|---|
| M2 Pro (16 GPU cores) | ~8-10 t/s | Not enough RAM |
| M3 Pro (14 GPU cores) | ~10-12 t/s | Not enough RAM |
| M4 Pro (20 GPU cores) | ~14-18 t/s | ~8-10 t/s |
| M4 Max (40 GPU cores) | ~25-30 t/s | ~15-18 t/s |
| M2 Ultra (76 GPU cores) | ~35-40 t/s | ~25-30 t/s |
For reference, comfortable conversational speed is approximately 10+ tokens/sec. Code generation tasks are less latency-sensitive — 5 t/s is workable.
How does Qwen 3.8 27B compare to other local models on Mac?
| Model | Parameters | Q4 size | Apple Silicon speed | Key strength |
|---|---|---|---|---|
| Qwen 3.8 27B | 28B | 16.5 GB | ~14-18 t/s (M4 Pro) | Coding + vision + 262K context |
| Llama 3.3 70B | 70B | ~40 GB | ~8 t/s (M4 Max) | General reasoning |
| Gemma 3 27B | 27B | ~16 GB | ~15-18 t/s (M4 Pro) | Google ecosystem |
| Mistral Large 2 | 123B | ~70 GB | ~4 t/s (Ultra only) | Multilingual |
| Phi-4 14B | 14B | ~8 GB | ~25-30 t/s (M4 Pro) | Fits on 16 GB Macs |
| DeepSeek V3 | 671B | Not viable | CPU only | Needs server hardware |
Qwen 3.8 27B hits the best tradeoff for 24 GB Macs: large enough to match 70B models on coding benchmarks (SWE-bench Pro 61.7 vs Llama 3.3’s ~49), small enough to run at conversational speed.
What about Qwen 3.8 Flash Next — is that better for local use?
Qwen 3.8 Flash Next is a 180B-parameter Mixture-of-Experts model that activates only 6B parameters per token. It outperforms the 27B dense model on every published benchmark (DeepSWE 1.1: 58.7 vs 42.2, SWE-bench Pro: 62.5 vs 61.7, according to the Qwen 3.8 Flash Next model card on Hugging Face). However, the full model is 180B parameters on disk. GGUF quantizations exist but the Q4_K_M is approximately 100+ GB — requiring a Mac Studio Ultra with 128 GB+ RAM. For most Mac users, the dense 27B is the practical local choice.
Does Qwen 3.8 27B support thinking mode locally?
Yes. Qwen 3.8 27B generates <think>...</think> blocks by default. Three reasoning effort levels are available: xhigh (default), medium, and low. In Ollama, thinking is enabled by default. To disable it for faster responses, set enable_thinking: false in the system prompt or use the reasoning_effort: low parameter. Thinking mode uses the recommended sampling settings: temperature 1.0, top_p 0.95, top_k 20.
Sources
- Qwen 3.8 27B model card — architecture, benchmarks, sampling parameters, license. Hugging Face, August 2026.
- Qwen 3.8 Flash Next model card — MoE architecture, comparison benchmarks. Hugging Face, August 2026.
- unsloth/Qwen3.8-27B-GGUF — quantization levels, file sizes, compatible inference engines. Hugging Face.
- Ollama documentation — local model serving, OpenAI-compatible API.
- llama.cpp repository — GGUF inference engine with Metal GPU support for Apple Silicon.
Corrections: editorial@hypogray.com.