Primer

Local LLM or cloud API: a five question test.

A healthcare practice has been handed two PowerPoints. Both vendors promise AI that saves hours every week. One runs in the cloud. One runs on a server you would own. The deck on the table will not tell you which is right for you.

The hook

Where this shows up.

Every week, an operations director somewhere is told to decide between a cloud hosted AI tool and a local one. The cloud tool is cheaper this month. The local tool keeps the data in the building.

Both vendors will frame their answer as the obvious choice. Neither will ask the five questions that actually settle the decision. Here they are.

The misconception

That cost is the deciding factor.

What most people believe

Cost almost never settles this decision cleanly. The cloud option looks cheaper in month one and breaks even somewhere between month nine and month eighteen depending on your usage volume.

The decision is about data sensitivity, usage shape, and operational maturity. Cost is a tiebreaker, not a verdict.

The better model

Five questions, in order.

What actually works

Run through these in order. The answer to question one alone determines the path for most healthcare and legal practices. Questions two through five are tiebreakers.

01
Does any of the data ever touch a protected category?
PHI, attorney privilege, classified information, trade secrets your competitors would pay for. If yes, default to local unless the cloud vendor signs a BAA AND you have read their data flow diagram AND you are comfortable with it.
02
Is your usage steady or spiky?
Steady predictable usage favors local (you pay once for hardware, then nothing). Spiky unpredictable usage favors cloud (you pay only when you use it).
03
Do you have someone who can keep a server alive?
Local AI needs someone to restart it, patch it, and respond when it breaks. If that person does not exist, the local option creates more work than the cloud option saves.
04
Is your network reliable enough that a cloud outage stops the work?
A practice that loses internet for two hours per month and uses AI for clinical documentation cannot afford a cloud only setup. Local works offline. Cloud does not.
05
What is your time horizon?
A three month pilot favors cloud regardless of the answers above. A five year deployment shifts the math toward local because the cumulative cost of cloud eventually exceeds the cost of owning the hardware.

Three decisions

Make these calls differently this week.

If any data is sensitive AND you have IT support, default to local.

The compliance burden of cloud is high enough that local is usually the cheaper option once you account for BAA negotiation, audit overhead, and the residual risk of a vendor breach.

If usage is unpredictable AND data is non sensitive, default to cloud.

Pay for what you use. A small marketing team summarizing public articles does not need a GPU server in the closet.

If you are piloting for under three months, always start with cloud regardless.

Local hardware procurement and configuration alone takes four to six weeks. By the time you would have it running, the pilot would be over. Pilot in cloud, migrate to local when you commit to a long term deployment.

How ByteWorthy uses this

What this looks like in our work.

We start every client engagement with this exact conversation. The answer is different for a healthcare practice with PHI than for a real estate agent automating follow up. Both can be right.

When we recommend local, we spec the hardware, handle procurement, and get the model running with your team. When we recommend cloud, we configure the BAA, set up the integrations, and lock the configuration.

See our in house infrastructure service

Keep reading