Running Qwen 3.6 Locally on a Mac Mini M4 with 16GB RAM

Two days ago Qwen open-sourced Qwen 3.6-35B-A3B — a 35-billion parameter Mixture of Experts model that only activates 3 billion parameters per token. It's Apache 2.0 licensed, ships with a vision encoder, and is reportedly competitive with much larger models on agentic coding benchmarks. GGUF quantizations were up within hours.

Here's the thing: a model of this size can run on a $599 Mac Mini M4 with 16GB of RAM. Not a toy demo. Jock.pl reported 17.3 tok/s, zero swap and 81% memory free on that exact machine. Two caveats: that was his test, not mine, and it used the previous release, Qwen 3.5-35B-A3B, which has the same 35B total / 3B active size. This post is about how to set it up, and which tools to use.

The naive math says it shouldn't fit. The standard formula for estimating model memory (from BentoML):

Memory (GB) = Parameters (B) × (Bits per weight / 8) × 1.2 overhead
35 × 4 / 8 × 1.2 = ~21GB

21GB for a Q4 quantization. That doesn't fit in 16GB. So how does it work?

The key is the MoE architecture. "35B-A3B" means 35 billion total parameters, but only 3 billion active per token. The model uses 256 total experts with 8 routed + 1 shared active per inference step. The remaining experts sit idle. This is what makes the --mmap trick possible: llama.cpp memory-maps the model file, and the OS only pages in the weights for the currently active experts. Jock.pl reports that the shared layers (attention, embeddings, norms) stay resident in RAM at about 4-6GB. The expert weights come from the SSD when the router picks them.

Reported by Jock.pl (Pawel Jozefiak, April 3, 2026): on a base Mac Mini M4 16GB, Qwen 3.5-35B-A3B ran at 17.3 tok/s decode, with 81% memory free and zero swap. His exact setup: Unsloth's UD-IQ3_XXS quant (13GB on disk), llama-server with --mmap, --n-gpu-layers 0 (CPU only), --ctx-size 16384, --flash-attn on, 8 threads. The same post says his first try through Ollama loaded 26GB into memory, froze the machine, and produced no tokens.

There are four main ways to run LLMs locally on a Mac. Here's how they compare for running the 35B-A3B on 16GB specifically:

*mlx-vlm has a PR in progress for tool calling support.

One important detail: Ollama 0.19 (released March 30, 2026) shipped an MLX backend. Ollama reports that it nearly doubles decode speed, from 58 tok/s to 112 tok/s, in their own test of Qwen 3.5-35B-A3B (NVFP4 on 0.19 against Q4_K_M on 0.18, on Apple M5-family hardware). But it requires more than 32GB of unified memory. On 16GB, Ollama falls back to the llama.cpp backend (see the warning in the Ollama section below). LM Studio doesn't have this gate, so it at least lets you try MLX on 16GB.

This is the only path with a published result on a 16GB Mac mini, so it is the one I recommend. The commands below use the same quant as Jock.pl’s test. Metal GPU acceleration is on by default on macOS. Jock.pl turned it off (--n-gpu-layers 0, CPU only), and I have not seen a published 16GB number with Metal on.

# Build llama.cpp
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build
cmake --build build --config Release

# Download the Qwen 3.6 GGUF (UD-IQ3_XXS, 13.2GB)
pip install huggingface_hub
huggingface-cli download unsloth/Qwen3.6-35B-A3B-GGUF \
  Qwen3.6-35B-A3B-UD-IQ3_XXS.gguf \
  --local-dir models/

# Run with mmap — this is the key flag
./build/bin/llama-cli \
  -m models/Qwen3.6-35B-A3B-UD-IQ3_XXS.gguf \
  --mmap \
  -c 4096 \
  -n 512 \
  -p "Write a FastAPI endpoint with input validation"

Note on the quant: use UD-IQ3_XXS (13.2GB). That is the quant behind the 17.3 tok/s number, CPU only, on Qwen 3.5. The UD-Q4_K_M file in the same Unsloth repo is 22.1GB, bigger than the RAM, and I have found no published measurement of it on a 16GB Mac.

What's happening: --mmap tells llama.cpp to memory-map the model file instead of loading it all into RAM. The OS pages in weights on demand. Because only ~3B parameters are active per token, resident memory can stay under 16GB. In Jock.pl’s run, 81% of RAM stayed free and there was zero swap. The rest of the model file lives on your SSD and gets paged in only when an expert is activated.

You can also run it as an OpenAI-compatible API server for use with coding agents:

# Start the server (Jock.pl's flags: CPU only, 16K context)
./build/bin/llama-server \
  -m models/Qwen3.6-35B-A3B-UD-IQ3_XXS.gguf \
  --mmap \
  --n-gpu-layers 0 \
  --ctx-size 16384 \
  --flash-attn on \
  --threads 8 \
  --port 8080

# Now any tool that speaks the OpenAI API can use it:
# aider, opencode, aichat, etc. → http://localhost:8080/v1

GGUF files are also available from bartowski if you prefer different quantization levels.

If you don't want to build anything from source, Ollama handles everything — download, quantization, API server — in one command. Under the hood it uses llama.cpp. Be careful on 16GB, though: in Jock.pl's test, Ollama loaded the whole 26GB model into memory instead of paging it, and the machine froze.

# Install Ollama, then:
ollama run qwen3.6:35b-a3b

Ollama downloads the GGUF, picks Q4_K_M by default (about 22GB, more than the RAM), and starts an OpenAI-compatible API at http://localhost:11434. You can connect coding agents directly:

# Launch with opencode
ollama launch opencode --model qwen3.6:35b-a3b

# Or with OpenClaw
ollama launch openclaw --model qwen3.6:35b-a3b

Ollama exposes models to anything that speaks the OpenAI API format — aichat, aider, opencode, and many others. Point them at http://localhost:11434/v1.

I have no measured Ollama speed for this model on 16GB, so test it before you rely on it. The MLX-accelerated path requires more than 32GB, so on 16GB Ollama falls back to the llama.cpp backend.

LM Studio deserves a special mention because it does not gate MLX behind 32GB, unlike Ollama. InsiderLLM says MLX uses roughly 50% less memory than Ollama for the same model at the same quantization, and a DEV.to guide reports about 2x faster generation. Both are about Qwen 3.5.

Download the app, open the models page (Cmd + Shift + M), search for "Qwen3.6-35B-A3B", and filter by "MLX". The 4-bit quantization from mlx-community is 20.4GB, more than the RAM. On 16GB, the safer choice in LM Studio is the GGUF UD-IQ3_XXS file from Setup 1.

Kai Wern's guide reports 81.79 tok/s generation with LM Studio running the MLX 4-bit Qwen 3.5-35B-A3B on a Mac mini M4 Pro with 64GB. On 16GB it is a much tighter fit. In the same guide, an mlx-vlm run of the 4-bit model reported 20.5GB peak memory, and Kai Wern recommends at least 32GB. InsiderLLM says the model fits on 16GB thanks to MLX's lower memory use, but I have not found a published MLX measurement on a 16GB Mac. Measure before you rely on it.

To use LM Studio as a local API server (for coding agents), switch to the Developer screen (Cmd + 2) and toggle the server to Running. It serves on http://127.0.0.1:1234/v1.

MLX is Apple's own ML framework and the fastest inference path on Apple Silicon. If you don't need tool calling — just direct chat or batch generation — this gives you the best tok/s.

# Install mlx-lm
pip install mlx-lm

# Run the 35B-A3B
mlx_lm.generate \
  --model mlx-community/Qwen3.6-35B-A3B-4bit \
  --max-tokens 200 \
  --temp 0.7 \
  --prompt "Write a Python function to merge two sorted lists"

# Or start an OpenAI-compatible server
mlx_lm.server --model mlx-community/Qwen3.6-35B-A3B-4bit --port 8080

On 16GB, keep in mind that the 4-bit MLX weights are 20.4GB, and I have found no published MLX measurement on a 16GB Mac.

Since Qwen 3.6-35B-A3B is a vision-language model, you can also use mlx-vlm to process images:

# Install with torch dependency (avoids transformers errors)
brew install pipx
pipx install "mlx-vlm[torch]"

mlx_vlm.generate \
  --model mlx-community/Qwen3.6-35B-A3B-4bit \
  --max-tokens 200 \
  --temperature 0.0 \
  --prompt "Describe this image"

The catch: mlx-lm doesn't support tool calling yet, so you can't use it as a backend for coding agents that need to read/edit files. For that, use llama.cpp, Ollama, or LM Studio.

These are not my measurements. Only the llama.cpp row is a real measurement on the base Mac Mini M4 (16GB, 10-core CPU, 10-core GPU). It is from Jock.pl, on Qwen 3.5-35B-A3B at UD-IQ3_XXS, CPU only. The MLX rows are estimates, and the Ollama row is what Jock.pl saw when he tried it. I plan to replace this table with my own runs on Qwen 3.6. Own Mac mini measurements: not yet.

*Estimate, not a measurement. The 25-35 tok/s range is InsiderLLM's extrapolated figure for Qwen 3.5-35B-A3B on a 16GB M1 Pro. For a measured reference on bigger hardware, Ante Kapetanovic measured Qwen 3.5-35B-A3B on an M4 Max with 128GB: about 130 tok/s with MLX through the Python API, and about 71 tok/s with llama.cpp.

For comparison, Ollama reports 1810 tok/s prefill and 112 tok/s decode with the 0.19 MLX backend, for Qwen 3.5-35B-A3B in NVFP4 on Apple M5-family hardware with more than 32GB. The 32GB threshold is real — if you're buying a Mac specifically for local inference, the 32GB upgrade pays for itself.

But 17 tok/s, if it holds for Qwen 3.6 on your setup, should be usable for interactive chat. Whether it is fast enough for tool-calling agents is my guess, not something I have tested. It's slower than an API call, but it's free, private, and offline.

The full Qwen 3.6-Plus flagship (1M context, top-of-the-line benchmarks) was released on April 2, but it's API-only through Alibaba's DashScope. No weights, no GGUF files, no local option. The GitHub repo is up but only for the 35B-A3B variant.

For local inference, 3.6-35B-A3B is it — and given its benchmark numbers relative to its active parameter count, it's far more than a consolation prize.

I have not run Qwen 3.6-35B-A3B on a 16GB Mac mini myself, and I have no own benchmarks. This is the setup I would start with, based only on the sources on this page:

• Engine: llama.cpp (llama-server) with --mmap. It is the only path with a published 16GB Mac mini result: Jock.pl measured 17.3 tok/s with zero swap, on Qwen 3.5.
• Quant: Unsloth UD-IQ3_XXS (13.2GB), the same quant as in that test. UD-Q4_K_M is 22.1GB and the MLX 4-bit weights are 20.4GB. Both are bigger than the RAM.
• Flags: start from Jock.pl’s: CPU only (--n-gpu-layers 0), --ctx-size 16384, --flash-attn on, 8 threads. Then try Metal offload and compare.
• Not Ollama: Jock.pl’s Ollama attempt loaded 26GB, froze the Mac, and produced no tokens.
• Coding agents: point aider or opencode at http://localhost:8080/v1. llama-server speaks the OpenAI API, so this should work, but I have not checked tool-calling quality at this quant.

Two caveats. The published number is for Qwen 3.5, not 3.6, at the same 35B total / 3B active size. And it was a CPU-only run, so 16GB numbers with Metal are still open.

• Hugging Face — Qwen 3.6-35B-A3B (model card, benchmarks)
• Unsloth — Qwen 3.6-35B-A3B GGUF quantizations
• bartowski — Qwen 3.6-35B-A3B GGUF (alternative quants)
• Ollama blog — MLX backend announcement (0.19 release)
• Qwen docs — running with llama.cpp
• Jock.pl — 35B on Mac Mini 16GB (benchmark source)
• Kai Wern — Run Qwen locally on your Mac (LM Studio + Ollama + MLX walkthrough)
• Ante Kapetanovic — Ollama vs llama.cpp vs MLX benchmark
• InsiderLLM — MLX vs Ollama speed test
• DEV.to — MLX on Apple Silicon for 2x performance
• GitHub — 16GB VRAM Local configs and launchers
• Will It Run AI — Qwen VRAM requirements guide
• Unsloth — Qwen local setup docs
• r/LocalLLaMA — Explain MoE like I'm 25