Running AI Locally on a 24GB M4 Mac: What's Actually Possible at Home

For most Australians dabbling with ChatGPT, Claude or Gemini, AI still lives somewhere in the cloud — a subscription, a login, and a long list of things you can't say without tripping a content filter. But quietly, over the past 18 months, a different story has been unfolding on desks and kitchen tables: people running surprisingly capable large language models entirely on their own hardware, with no internet connection, no monthly fee, and no terms of service.

The unlikely hero of that shift isn't a specialist AI workstation. It's the Mac. And more specifically, the mid-range M4 with 24GB of unified memory — a machine you can walk into a JB Hi-Fi and buy for the price of a decent secondhand car deposit.

Why unified memory changed the game

The technical reason Apple Silicon punches above its weight for local AI comes down to architecture. On a typical Windows gaming PC, the GPU has its own VRAM — usually 8, 12, maybe 16GB if you've spent serious money — and that's the hard ceiling on how big a model you can load onto the graphics card. System RAM is plentiful but slow to access from the GPU.

Apple's M-series chips throw that split out. The CPU, GPU and Neural Engine all share the same pool of high-bandwidth memory. As XDA noted in its piece on Apple's sleeper advantage for local LLMs, this means a Mac with 24GB of unified memory can effectively dedicate the vast majority of that to a model, where a PC with 32GB of system RAM and a 12GB GPU is still capped by the smaller number when it comes to fast inference.

SitePoint's 2026 hardware comparison for local LLMs reaches a similar conclusion: dollar for dollar, Macs occupy a sweet spot for hobbyists who want to run mid-sized models without building a multi-GPU rig that sounds like a hairdryer and eats 600 watts at idle.

What a 24GB M4 can actually run

The honest answer is: more than you'd think, but with caveats. Rough rules of thumb for quantised models (the compressed versions most people actually run locally):

For the overwhelming majority of practical home uses — drafting emails, rewriting documents, answering questions about your own notes, helping with code, generating ideas — the 7B-to-14B range is genuinely good now. Not GPT-4-class, but easily good enough to replace a lot of routine cloud AI use.

The software finally caught up

Hardware was only half the story. The other half was tooling. Two years ago, getting a model running locally meant wrestling with Python environments, CUDA versions and command-line incantations. Today, apps like LM Studio, Ollama and Jan have collapsed the process to roughly: install, click a model, chat.

As the Chinese tech publication 36Kr put it in its review of so-called "AI PCs", after surveying the field of dedicated AI laptops, the Mac has effectively been there the whole time — quietly running the same workloads with less marketing and fewer copilot keys. The Neural Engine, MLX (Apple's machine learning framework) and Metal acceleration have been doing the heavy lifting in the background for years.

For an Australian buyer staring at the laptop wall, that matters. You don't have to learn a new ecosystem or buy a model with an NPU sticker. You just need enough RAM.

Why 24GB is the new sensible floor

Here's where the buying advice gets sharper. Apple's base M4 configurations now start at 16GB, which is fine for everyday computing but tight for serious local AI — you'll be limited to smaller 7B models with shorter context, and macOS itself wants a healthy slice of that pool.

Step up to 24GB and the experience changes meaningfully. You can run a 13B model with a long context window and keep your browser, IDE and Slack open.

It's also notable that Apple has been reshuffling the high end. Inkl recently reported that Apple has quietly axed the 128GB Mac Studio configuration amid supply pressures — a reminder that the bleeding edge of local-AI-capable Apple hardware is volatile and expensive. For most people, chasing 128GB is overkill. The 24GB-to-48GB band is where genuine value lives for at-home AI work.

The real reasons to bother running models locally

If cloud AI is cheap and fast, why go to the trouble at all? A few reasons that genuinely hold up:

What you give up

It would be dishonest to pretend local AI is a clean replacement for the frontier models. A quantised 13B model running on your Mac is not GPT-5 or Claude Sonnet. It will hallucinate more, reason less reliably on hard problems, and lack live web access unless you bolt that on yourself. Coding assistance is decent but not magical. Long, complex reasoning chains often fall apart.

The right mental model is to treat local AI as a competent generalist assistant that lives on your laptop — somewhere between a very good autocomplete and a junior research aide — and keep a cloud subscription for the heavy lifting if you need it. Many people end up doing exactly that: 80% of their casual AI use shifts to the local model, and they keep one paid plan for the gnarly stuff.

The bigger picture

What's quietly happening with the M4 generation is more interesting than any single product launch. The combination of unified memory, mature open-weight models from Meta, Mistral, Alibaba and Google, and one-click software like Ollama means a normal person — not an ML engineer — can now keep a competent AI model entirely on their own machine.

That has implications well beyond hobbyist tinkering. Small Australian businesses can run AI over sensitive client data without sending it offshore. Schools can give students hands-on AI experience without account signups. Researchers in remote areas can work without bandwidth dependence.

The hardware to do this used to cost five figures and live in a server rack. Now it fits in a backpack and runs on battery. A 24GB M4 isn't the most powerful AI machine you can buy — but it might be the most important one, because it's the first that makes "just run it yourself" a realistic answer for ordinary people.

Related on Bleen

Sources