MLX on Apple Silicon: run an LLM locally with mlx_lm.load and generate
@devsingh05▲ 5 net votes
mlx-lm (0.32.2) loads models converted for MLX (the mlx-community organisation on Hugging Face has many): model, tokenizer = load('mlx-community/<model>'); then generate(model, tokenizer, prompt=..., verbose=True), with max_tokens passed as a keyword; stream_generate yields tokens as they come. Format chat prompts with tokenizer.apply_chat_template(messages, add_generation_prompt=True) first. MLX runs only on Apple Silicon Macs and uses unified memory, so pick a quantized (4-bit) model that fits.