What open AI models are actually good enough for.

There are two dishonest answers to this question. One says open models have caught up with ChatGPT and Claude, which any client will disprove within an afternoon by handing it their hardest problem. The other says they're toys, which is contradicted by the fact that a lot of production work now runs on them quite happily. The truth sits between those, it's measurable, and it's more useful than either.

How far behind are they, honestly?

Two independent measurements agree, which is more than you usually get in this field.

Epoch AI tracks the distance between the best open-weight models and the frontier closed ones. Since January 2026 it has averaged about four months, or 8 points on its capability index. That's a comparable distance to the step between two consecutive flagship releases from the same lab.

Artificial Analysis, which scores models on a composite intelligence index, put the highest-scoring open model (Kimi K3) at 57 in August 2026, against 61 for the highest-scoring closed model (Claude Opus 5). A four-point gap on a scale where the leaders cluster tightly.

So the defensible line is this: open models sit about one release cycle behind, and that gap has stayed roughly where it is for eighteen months. They have not caught up, and they aren't falling away either. Anyone who tells you the first thing is selling you something, and anyone who tells you the second hasn't looked recently.

The task-by-task answer, which is the useful part

A four-point average gap tells you very little on its own, because it lands unevenly across the work. Here's where it lands.

Genuinely solved on hardware you own:

• Summarisation. Meeting notes, long email threads, reports, call transcripts. This is the strongest local use case and it's where most projects should start.
• Structured extraction. Pulling fields out of invoices, forms and contracts into clean JSON. Often better locally, because you can constrain the output at the token level so invalid output becomes impossible rather than unlikely, and most cloud APIs don't expose that control.
• Classification and routing. Tagging, triage, sorting enquiries to the right person. A small model fine-tuned on your own examples routinely beats a much larger model that has only been prompted, and it's cheap to run at volume.
• Transcription. Local speech to text is fast, accurate and keeps the audio in the building, which matters a lot in clinical and legal settings.
• Translation, OCR and document parsing. Unglamorous, high volume, and a good fit for a machine that's already paid for.
• Question answering over your own documents. The limiting factor here is almost always the retrieval and the document preparation, not the model.

Where frontier cloud models still clearly win: hard agentic coding across a real multi-file repository, long-horizon reasoning, and complex multi-step tool use. Not marginally. Clearly.

The mechanism is what makes that split credible rather than a matter of taste. A single-pass transformation, such as reading a document and producing a summary, absorbs a small quality gap invisibly, because there's one step and a slightly weaker model produces a slightly less elegant answer that's still correct. An agentic task chains twenty steps together, and a small per-step disadvantage compounds. A model that's 2% worse per step is dramatically worse at finishing the whole job. That's why the same four-point gap is imperceptible on one task and decisive on another.

“Open weights” and “runs in your office” are different claims

This is the most common overclaim in the local AI conversation, and it's worth being precise about because it wastes real money.

Kimi K3, the open model scoring 57 above, has 2.8 trillion parameters. It's freely downloadable and it will not run in anyone's small office. The same goes for the largest DeepSeek and GLM releases at full precision, which need multiple hundreds of gigabytes of fast memory. Downloadable and self-hostable are separate properties, and the models that top the open leaderboards are frequently the ones you can't host.

What you can run on a single good machine is a tier below that, and it's still very capable: models in the 9 billion to 35 billion parameter range handle everything in the solved list above. That's the tier the cost arithmetic in what local AI actually costs is built around, and it's the tier that matters for a small business.

On licences, the news is good and mostly unremarked. Qwen, Gemma, Mistral and gpt-oss ship under Apache 2.0, and recent DeepSeek and GLM releases under MIT. Both permit ordinary commercial use without conditions worth arguing about. Meta's Llama, by contrast, carries a more restricted community licence with usage conditions and naming requirements attached, and Llama hasn't had a new open flagship since May 2025. Which means the old safe recommendation, just use Llama, now points at something well behind the field and with a more awkward licence than the alternatives. An Australian business never has to accept a restricted licence to get excellent local AI.

The China question, answered precisely

Most of the strongest open models today come from Chinese labs, and the instinctive reaction is that using one sends your data to China. That instinct is pointing at a real risk and aiming it at the wrong object.

A downloaded model is a file of numbers. Static arrays of weights, not an executable. It has no capacity to open a network connection, and it cannot transmit anything anywhere, because there's nothing in it that runs. You can confirm that with a packet capture, or more bluntly by pulling the network cable out and watching it keep working. That isn't a policy assurance or a vendor promise; it's a property of what the file is.

Two things do survive local deployment, and both deserve a straight answer. The first is alignment behaviour: models trained in China are noticeably evasive on politically sensitive subjects. That's irrelevant to a quoting assistant and a genuine capability defect if your work involves China-related research or due diligence, so test it against your actual subject matter rather than assuming either way. The second is supply-chain integrity: where you download the file from, whether you verified its checksum, and whether the format can execute code when loaded. Use the verified vendor repository, check the hash, insist on safetensors format, and filter the host's outbound traffic. Those are controls you'd want on any model from any country.

The Australian Government has drawn exactly this distinction in writing. PSPF Direction 001-2025, issued on 4 February 2025, required government entities to remove DeepSeek products, applications and web services from their systems. It then carved out, expressly: “This does not include open-sourced Large Language Models (LLM) where the entire codebase is available for inspection, the model is deployed locally on a government system, and appropriate mitigations are in place.” The hosted service was prohibited. The locally deployed open model was permitted, with conditions.

Which leads somewhere counterintuitive and correct. A firm pasting client information into a US-hosted chatbot has a live cross-border disclosure question to answer today. The same firm running a Chinese-origin open model on a machine in its own office has none, because nothing leaves the building. On data sovereignty grounds specifically, the second arrangement is the more protective one. The difference between residency and sovereignty is worth understanding before you form a view on that.

The safety gap, which is the genuine catch

Here's the part local AI advocates tend to skip, and it's the one that should change what you build.

TechCrunch reported on 4 August 2026 on a SaferAI evaluation which found that GLM-5.2, one of the strongest open models available, refused none of the offensive cyber or biology tasks it was given. Claude Opus 4.7, on the same evaluation, refused so consistently that the assessment couldn't be completed. That's a wide gap in refusal behaviour between the open and closed tiers.

The reason a normal business should care has nothing to do with cyberweapons. Safety training and business-appropriateness training are the same training. A model that has been taught to decline harmful requests has also been taught to decline the customer email that's defamatory about a competitor, the response that makes a discriminatory assumption, and the confident answer to a question it should have refused. A model with weak refusal behaviour is more likely to produce all three, to a real customer, in your name.

The practical consequence: if a local model touches customers, it needs a guardrail layer in front of it, along with review steps on anything that goes out. That's a real line item in the build, not an optional extra, and any proposal that omits it is underquoting. It's also the reason JDCS keeps a human approving anything customer-facing, whatever model is underneath. More on how that gets built on the local and private AI page, and on AI consulting if you want the decision before the build.