How open-source AI is closing the gap on expensive proprietary models
For most of the generative AI era, the story has been the same: a handful of well-funded labs release a shiny new flagship model, benchmarks light up, and the rest of us pay per token to use it. But quietly, and then not so quietly, that dynamic has been shifting. Open-source and open-weight models — the kind you can download, tinker with, or run on your own hardware — are catching up on the tasks people actually use AI for. And on some of them, particularly retrieval-heavy workloads, they are matching much more expensive proprietary systems at a fraction of the cost.
If you have seen a headline claiming an open model beat a top-tier commercial one at a tiny sliver of the price, you are not imagining it. The pattern is real, and it matters more than the benchmark leaderboards suggest.
What "retrieval" actually means — and why it's the sweet spot
When people talk about retrieval in the context of AI, they usually mean retrieval-augmented generation, or RAG. Instead of relying purely on what a model "remembers" from training, a retrieval system fetches the most relevant chunks of information from a database — company documents, a knowledge base, a personal notes archive — and hands them to the model as context before it answers.
This matters because raw model intelligence is only half the equation for most real-world use cases. A support chatbot needs to cite the right refund policy. A legal assistant needs to quote the correct clause. A research tool needs to point at the correct paper. In all these cases, the model's job is less "know everything" and more "read the right thing and summarise it accurately."
That is precisely the kind of task where the very biggest, most expensive models are overqualified. You don't need a frontier reasoner to summarise a paragraph it has just been shown. You need a competent, reliable reader — which is exactly what modern open models have become.
Why open models are suddenly competitive
A few things have converged over the past couple of years to close the gap.
- Better base models. Meta's Llama family, Mistral's releases, Alibaba's Qwen line, DeepSeek and others have pushed open weights to a level that was unthinkable in 2022. Many of them post scores on standard benchmarks that would have been state-of-the-art for a proprietary model not long ago.
- Smarter fine-tuning. Techniques like LoRA and QLoRA let developers cheaply adapt a general model to a specific task — say, answering questions about Australian tax law — without retraining from scratch.
- Cheap, fast inference. Providers like Groq, Together, Fireworks and Cerebras offer hosted open models at prices that can be one or two orders of magnitude lower than the top proprietary APIs. Running Llama or Qwen through one of these services often costs cents where a frontier model would cost dollars.
- Retrieval offloads the hard part. When the answer is already in the retrieved context, the model doesn't need to be a genius. It needs to be accurate and follow instructions. Small, cheap models are good at that.
The result is a stack where the expensive part — the general reasoning muscle of a frontier model — becomes optional for many everyday tasks.
The 100x cost gap is not a typo
The pricing chasm between the top proprietary APIs and hosted open models is genuinely enormous. Depending on the model and provider, you can pay somewhere between fifty and a hundred times less per million tokens by running an open model instead of a flagship closed one. For a business running millions of customer queries a month, that is the difference between an AI feature being a rounding error and being a line item the CFO wants to talk about.
And the cost story compounds. Cheaper inference means you can afford to send more context, run more retrieval passes, or use ensemble approaches where several small models check each other's work. Techniques that would be financially insane with a top-tier proprietary model become routine when each call costs a fraction of a cent.
What this means for everyday users
Most people will never fine-tune a language model or spin up a vector database. So why does any of this matter to a regular Australian scrolling through the app store looking for something useful?
Because the economics of the tools you use are about to change.
- More AI in more places, for less. When it costs almost nothing to add a smart assistant to a product, more products get one. Expect to see AI features showing up in local Australian apps — from small business accounting tools to community platforms — that could never have justified the cost a year ago.
- Better privacy options. Open models can be run on-device or in a private cloud. For sectors like healthcare, law and government, where sending data to an overseas API is a compliance nightmare, this is a genuine breakthrough. A GP clinic in regional NSW can, in principle, run a capable model behind its own firewall.
- Less lock-in. If your favourite productivity app is built on an open model, you are less exposed to a single vendor changing pricing or deprecating a feature. Developers can swap the underlying engine without rewriting their product.
- Australian sovereignty benefits. The federal government has been vocal about the need for sovereign AI capability. Open weights are the practical foundation for that — you cannot have a truly sovereign system if the only working models sit inside three American companies.
The caveats — because there are always caveats
Open-source parity is not universal. On the hardest tasks — long-horizon reasoning, complex code generation, multimodal understanding of tricky images — the frontier proprietary models still lead, sometimes by a lot. If you are asking an AI to plan a research programme or debug a subtle race condition in a large codebase, you probably still want the expensive one.
There are also softer issues. Open models can be easier to jailbreak, come with less polished safety tuning, and shift the responsibility for guardrails onto whoever deploys them. "Cheaper" is not the same as "consequence-free."
And benchmark victories deserve scepticism. A model that wins on a particular retrieval evaluation may still trail on another. The honest read of the current landscape is not "open models have won" but "open models are good enough for a rapidly growing share of what people actually do with AI."
The takeaway
The interesting story in AI right now isn't which lab has the biggest model. It's that the floor is rising fast. Capable, cheap, downloadable models are absorbing more and more of the workload that used to require a premium API call. For retrieval-style tasks — the bread and butter of business AI — the gap between "good enough" and "the best" has narrowed to the point where cost, latency and control often matter more than raw intelligence.
For everyday users, that translates into more AI features, in more products, from more providers, with better privacy options and less exposure to a single vendor's whims. The flashy frontier models will keep grabbing headlines. But the quiet revolution is happening one open-weight release at a time.
Related on Bleen
Sources
- No external sources were used for this article — it is a general explainer based on publicly known trends in open-source AI development.