Short answer
Choose on control and cost at your real volume, not on benchmark scores. Proprietary API models win on low or spiky volume, zero maintenance, and frontier reasoning. Open-weight models win when data must not leave infrastructure you control, when volume is steady and high enough to beat per-token pricing, or when you cannot accept a vendor changing the model under a working system. Most companies end up mixing: a small controlled model for high-volume routine steps, a frontier model for the few steps that need judgment. Test both against your hardest real cases before deciding anything — twenty is a rule of thumb from LYVIA's own engagements, not a published benchmark.
The difference that actually matters
Strip the ideology out and one structural fact separates the two options: where the model runs. A proprietary model is a service you call — your data goes out, an answer comes back, and the vendor controls the version, the pricing and the availability. An open-weight model is a file you can run on a machine you rent or own, which means nothing leaves unless you send it and nothing changes unless you change it.
Every practical consequence follows from that. Privacy posture, cost curve, latency, the ability to freeze a version — none of them are separate criteria, they are all downstream of where the thing executes. Framing the choice as quality is what leads companies to pick on a leaderboard and then discover the constraint that actually mattered three months later.
"Open source" mostly means open-weight, and the license matters
The vocabulary is misleading and it has commercial consequences. Most models described as open source publish their weights — the file you run — without publishing the training data or the full recipe. That is more accurately called open-weight, and it is still enormously useful; it just is not open source in the sense a software team would mean.
What matters more than the label is the license attached. Some are permissive. Others carry conditions: restrictions above a user threshold, limits on specific uses, or requirements about attribution and downstream licensing. Before a model goes anywhere near a product you sell, someone has to read the actual license text rather than the blog post announcing it. This is a five-minute task that occasionally saves a rebuild.
Quality: the gap, and where it still shows
As of August 2026, the honest position is that the gap has narrowed sharply for the work small businesses actually automate, and remains real at the frontier. Classification, extraction from documents, summarizing, drafting a reply from context you supply — capable open models handle these well. Long multi-step reasoning, unusual edge cases, and tasks where the model must hold a lot of context and stay coherent are still where the largest proprietary models tend to lead.
Public benchmarks are a poor proxy for your decision. A model that ranks highly in general can fail on the particular shape of your documents, and a smaller one can be perfectly sufficient. Build the test set from your own hardest real cases and run both candidates against it — the same regression discipline described in our guide to prompt engineering for business. It converts an argument into a measurement.
Cost: where the crossover really sits
Self-hosting does not remove cost, it changes its shape. API pricing is per token with no floor: run nothing, pay nothing. Self-hosting is a fixed cost — a rented GPU or hardware you bought — that you pay whether or not any work arrives, plus the time of whoever patches, monitors and upgrades it.
- Low or spiky volume favors the API, decisively. Paying for idle capacity is the most common way a self-hosting decision turns out worse than the thing it replaced.
- Steady, high, predictable volume is where self-hosting starts to win — repetitive classification over thousands of items a day is the classic profile.
- The maintenance line is the one that gets forgotten. In our own engagements it is the part that turns a favorable spreadsheet into a break-even one, particularly in companies with no dedicated infrastructure person — a pattern from our client work rather than a published figure.
- Small models change the arithmetic more than hosting does. Routing a routine step to a cheaper model often saves more than moving hosts, and costs a day rather than a quarter.
Whichever way it goes, measure it against your realistic monthly volume before committing, using the same discipline as any other automation investment — set out in our guide to measuring AI automation ROI.
Data: when self-hosting is a requirement, not a preference
This is where the decision is most often made emotionally and should be made contractually. The question is not whether you would prefer your data to stay in-house — everyone would. It is whether something obliges it: a client contract with a data residency clause, a regulator, a sector rule, or a commitment you made in a tender.
Absent such an obligation, business tiers from the major providers generally commit to not training on customer content and offer regional processing options, which satisfies most companies once someone actually reads the terms. Where an obligation does exist, self-hosting stops being a trade-off and becomes the specification. Either way, the underlying discipline is the same one we apply on every build: know exactly which fields leave, where they are processed, and for how long — the written data decision described in our AI implementation checklist.
Control: the model that changes under you
An underrated argument for open weights has nothing to do with privacy or cost. A hosted model can be updated, deprecated or retired on the vendor's schedule, and a system tuned against one version can behave differently against the next without anyone touching your code. For a workflow that has been validated carefully, that is a real operational risk.
Running your own weights lets you freeze a version and upgrade deliberately. If you stay on an API, the mitigation is to pin the model version where the provider allows it, and to rerun your test set whenever a version changes — treating a provider's upgrade as a change to your system, because it is one.
The answer most companies land on: both
Framed as a binary, this question produces bad decisions. Framed per step, it produces obvious ones. A workflow is a chain, and each link can call a different model.
- High-volume, well-defined steps — classification, extraction, routing, tagging — to a small model, self-hosted or cheap.
- Judgment steps — the ambiguous reply, the unusual document, the decision with consequences — to the strongest model available.
- Anything touching data that must not leave — kept local by design, with only the sanitized remainder sent outward, if anything is sent at all.
Building it that way requires the workflow to be split by decision rather than written as one monolithic prompt, which is the same design that makes agents debuggable — covered in our guide to building AI agents without code.
Deciding in an afternoon rather than a quarter
The decision does not deserve a quarter of deliberation. Four questions settle it for most companies, in this order.
- Is there a written obligation that data must not leave your systems? If yes, the question is answered and the rest is implementation.
- Is the volume steady and high, or spiky and low? Spiky and low means API, and revisiting the question is cheap later.
- Do your hardest real cases pass on a smaller open model? Twenty is a rule of thumb from LYVIA's own engagements, not a published benchmark. If they do, the frontier model is a luxury for that step.
- Who maintains it in six months? If nobody can name that person, self-hosting is a decision to make later, not now.
Where the model runs is an infrastructure question as much as a model one, and the surrounding choices — hosting, storage, monitoring — are covered in our guide to AI infrastructure for business.
Frequently asked questions
What is the real difference between open-source and proprietary AI models?
Where the model runs and who controls it. A proprietary model is reached through an API: you send data out, you get an answer back, and the vendor decides when the model changes. An open-weight model is a file you can download and run on infrastructure you choose, which means nothing leaves unless you send it and nothing changes unless you change it. Everything else — quality, cost, speed — follows from that one difference.
Are open-source models good enough for business use now?
For the tasks most small businesses automate — classification, extraction, summarizing, drafting from supplied context — yes, and as of August 2026 the gap has narrowed considerably. Where frontier proprietary models still tend to lead is long multi-step reasoning and unusual edge cases — recheck it before locking in a model, because it keeps shrinking. The practical test is not which is better in general but whether the open model handles your hardest real cases; twenty is a rule of thumb from LYVIA's own engagements, not a published benchmark.
Is self-hosting an open model actually cheaper?
Only above a certain volume, and the crossover is higher than people expect because the cost moves rather than disappearing. API pricing is per token with no floor; self-hosting is a fixed cost for hardware or a rented GPU that you pay whether or not anything runs, plus the time of whoever keeps it running. Low or spiky volume favors the API; steady high volume is where self-hosting starts to win.
Do we need to self-host to keep our data private?
Not necessarily. Business tiers of the major providers generally commit contractually to not training on your data and offer regional processing, which is enough for most companies. Self-hosting becomes the answer when a contract, a regulator or a client requires that specific data never leaves infrastructure you control — that is a requirement question, not a preference one.
What does "open source" actually mean for AI models?
Less than the phrase suggests. Most models described as open source release the weights under a license with conditions attached, without releasing the training data. That is better called open-weight. It matters commercially: some licenses restrict use above a size threshold or for certain purposes, so the license terms need reading before a model goes into a product.
Can we mix both in the same system?
That is usually the right answer. Route the high-volume, well-defined steps — classification, extraction, routing — to a small model you control, and reserve the expensive frontier model for the steps that genuinely need judgment. Chained workflows make this natural, because each step is a separate decision about which model to call.
If you want this decided against your own cases and your own volume rather than a benchmark table, that comparison is part of how we scope a build. Book a call.
