Run a model on Apple silicon with MLX

Use an isolated environment

MLX LM is designed for generating text and fine-tuning language models on Apple silicon. Check its current requirements before installation. An Intel Mac is not the target for this guide.

Create a project-specific environment so package changes do not affect unrelated Python tools. Installation and the first generation need network access.

python3 -m venv .venv
source .venv/bin/activate
python -m pip install mlx-lm
mlx_lm.generate --help

Start with an explicitly named artifact

The official documentation names this MLX-compatible model as its default. An explicit model argument makes the example reproducible at the repository level; record the revision as well for repeatable evaluations.

A Hugging Face repository is not automatically MLX-compatible. Confirm both model architecture and quantization support.

mlx_lm.generate --model mlx-community/Llama-3.2-3B-Instruct-4bit --prompt "Explain what a token is in two sentences."

Watch unified-memory pressure

Leave memory for macOS and other applications. A model that forces heavy swapping may respond very slowly even if generation eventually succeeds. Close unrelated workloads or select a smaller artifact before changing system-wide limits.

Long prompts and retained conversation state also consume memory. The upstream guide documents prefill sizing and cache strategies; test the quality and speed tradeoff rather than assuming a smaller cache is equivalent.