Private AI for small business, explained by a live deployment.
Private AI means the model runs on hardware you control and your documents never leave your network. This page explains what that actually requires, who genuinely needs it, what it costs, and how it differs from a ChatGPT subscription — using the architecture and the measured figures from a regulated lender running it in production today.
What is private AI?
Private AI is artificial intelligence that runs on infrastructure you control, using models that read your documents without those documents being sent to an outside service. The usual implementation is on-premise: a physical machine in your building, bought in your name, running open-source models locally.
The word “private” is used loosely in this market, so it is worth being precise about what it does and does not mean. It does not mean a vendor promising not to train on your data. It does not mean a checkbox in an enterprise plan. It means the file does not leave your network, and you can prove it.
On our deployments, on-prem is the default for regulated data, and in normal operation nothing about a client file is sent to an outside service. Where a cloud model is genuinely the right tool, identifiers are stripped before anything is sent — which is a different architecture from sending the document and trusting a policy.
How is private AI different from a ChatGPT subscription?
A consumer subscription is a seat on somebody else’s server. That is not a criticism; for a great deal of work it is exactly right, and it costs a fraction of a build. The difference is what happens to the document.
When your bookkeeper pastes a client’s financials into a consumer chatbot, three things are true at once: the work gets done, the file has left your control, and there is no log you can read afterwards. The first is why staff do it. The second and third are why a regulated business cannot.
A private deployment changes the arithmetic rather than the interface. Staff get the same usefulness, on your hardware, with an audit trail behind every question. We cover the capability trade-offs in detail in ChatGPT vs. custom AI, and the deployment decision itself in where your AI data actually goes.
Who actually needs private AI?
Not everyone, and the honest version of this page says so early. If nothing in your documents is regulated or confidential, private AI is insurance against a risk you do not carry, and a subscription will serve you better for less.
The businesses that genuinely need it share one property: a specific class of document that cannot leave their control. In practice that is three groups.
Lenders and financial institutions. Borrower financials touch every step of the work — intake, credit, eligibility, pricing. This is the case our published deployment was built for: an SBA 504 and CDFI lender whose borrower data could not leave a controlled environment, which made feeding it to a cloud AI a non-starter.
Healthcare practices and clinics. The patient record is the constraint. Scheduling, intake and coding are all workflows AI handles well, and all of them touch the record. See AI for healthcare and wellness.
Legal and professional services. Privileged client files carry an obligation that a vendor’s terms of service does not discharge. See AI for professional services.
The test is not company size or revenue. It is whether you can name the document that cannot leave.
What does a private AI system actually consist of?
Five controls, in the order data meets them. This is the architecture on our security page, and it is the one running in production at a regulated lender rather than a policy aspiration.
1. Local models on your own hardware. The machine sits in your building. For regulated work, no client file leaves the network in normal operation.
2. PII stripped before any model reads a document. Microsoft Presidio — an open-source PII engine you can inspect yourself — detects and removes names, Social Security numbers and account numbers first. Identity is stripped before ingestion, not redacted after. The model sees the work, not the identity.
3. Everything stored is encrypted at rest. Documents, extracted data and work products, all of them, where they sit. A copied disk is a useless disk.
4. An append-only, tamper-evident audit log. Every action is appended to a log where entries can be added but never edited or deleted. When an auditor asks what happened, compliance can prove it rather than assert it.
5. A person signs off on every consequential action. Nothing sends, files, posts or pays until a human approves it. The system shows its work — what it did, what it is asking for, the source it relied on — and a person approves, returns or escalates it. The machine prepares; people decide.
The build that carries controls 1 and 2 is the private AI server and its data privacy filter.
Does private AI actually work in production?
This is the question most pages on this topic cannot answer, so here are figures from a system that is running rather than a diagram of one that could.
MARCUS is a private AI back office built for B:Side Capital, an SBA 504 and CDFI lender: 14 agents across 7 departments, built entirely from the institution’s own process documentation, running on-premises. It is published with the client’s permission, and every figure below is reconstructable from the tamper-evident audit log.
Over the first 90 days of full-fleet operation: 1,240 staff hours of preparation returned. 93% of staff using it weekly by week six, with no new app, no new login and no training deck. Zero borrower identifiers sent to any outside model. 100% of consequential actions approved by a person before anything was sent, filed, posted or paid.
On the privacy machinery specifically: 2.1 million model calls processed entirely on-premises in those 90 days. 1.9 million personal identifiers detected and stripped. 100% of prompts and attachments cleared the filter before reaching any model, because no bypass exists. Detection recall measures 99.8% on a monthly seeded red-team test — measured, not claimed. There have been zero PII incidents since the filter went live.
Two of those numbers matter more than the headline. The adoption figure, because a private system nobody opens returns nothing regardless of its architecture. And the incident count, because it is the number a board actually asks about. The full scorecard shows the rest.
What does private AI cost to buy and to run?
There are two costs and they behave differently.
The build is delivered through an AI Implementation Sprint at $18,000–$60,000, quoted fixed in writing before work begins, with hardware quoted as part of that scope rather than billed as a surprise afterwards. If a scoped workflow is not live in your operation within 90 days, we keep building at no charge until it is. An AI Readiness Audit at $3,500–$8,500 tells you first whether this is your right move.
The running cost is different in kind from a subscription. On owned hardware, model usage is electricity rather than per-token billing, which is why volume behaves so differently at scale — 2.1 million model calls in 90 days does not produce a 2.1-million-call invoice. What remains is tools, maintenance and monitoring; what AI automation costs to run covers that monthly reality in full.
One number worth stating plainly because vendors rarely do: we buy the hardware at cost, in the client’s name, at zero markup. You own the machine.
What are the honest limitations?
A local open-source model is not a frontier model. For reading documents, drafting, classifying and answering from your own records — the work most businesses actually need — it is sufficient. For open-ended general reasoning, the frontier models are ahead, which is exactly what the privacy filter exists to make available: cleaned text goes out, identifiers stay in.
Private AI is also not a compliance certificate. It is an architecture that makes compliance provable. The proof comes from the audit log, and a log nobody reads is just a file.
And it is a build, not a purchase. If your first instinct is that this sounds like a lot for a business your size, that instinct is often correct, and the audit is the cheap way to find out.
Where should you start?
Ask three people what they pasted into a chatbot last week, and do not punish the answer. If the answers include something that cannot leave your building, you have found your first workflow, and the rest of this is scoping.
Private AI, asked plainly.
01Is private AI the same as on-premise AI?
In practice, yes. Private AI means the model runs on hardware you control and your documents are not sent to an outside service. On-premise is the usual way to achieve that: a machine in your building, bought in your name, running open-source models. The distinction that matters is not the word — it is whether a file leaves your network.
02Do we need private AI, or is a ChatGPT subscription enough?
If nothing in your documents is regulated or confidential, a subscription is almost certainly enough and you should not buy a build. Private AI earns its cost when a specific class of document cannot leave your control — borrower financials, patient records, privileged client files. The test is not company size. It is whether you can name the document.
03What does private AI actually cost to run each month?
Running cost is tools plus model usage, and on owned hardware the model usage is electricity rather than per-token billing. The build itself is delivered through an AI Implementation Sprint at $18,000–$60,000, quoted fixed in writing before work begins. Our published guide to monthly running costs covers the ongoing side in detail.
04Is a local model good enough, or do we lose capability?
For reading documents, drafting, classifying and answering from your own records, local models are sufficient — that is the work most businesses actually need. Where a task genuinely needs a frontier model, a privacy filter strips identifiers first and sends cleaned text out, so the capability is available without the identities travelling with it.
05How do we prove to an auditor that data never left?
With a log, not an assurance. Every action writes to an append-only, tamper-evident audit log: entries can be added, never edited or deleted. In our largest published deployment, 100% of prompts and attachments cleared the privacy filter before reaching any model, and there were zero PII incidents since the filter went live.
06Who is accountable when the system gets something wrong?
A person, because nothing fires on its own. Send, file, post and pay all wait for human approval — the machine prepares and people decide. In the deployment above, 100% of consequential actions were approved by a person before anything was sent, filed, posted or paid.
30 minutes. A straight answer.
A senior advisor walks your workflows and tells you whether private AI is worth doing — including when the answer is not yet.