Most law firms we talk to have already decided they want AI. What stopped them was not skepticism about whether it works. It was the moment somebody asked where the documents go.
On-premise AI is the answer to that specific question, and only that question. It is worth understanding what it is before deciding whether the tradeoffs are worth it for your practice.
What "on-premise" actually means here
A physical machine sits in your office. On it runs an open-weight language model, typically in the 70-billion-parameter class, along with an index of the documents you choose to give it. Your team talks to it through a browser on the office network.
There is no API call to anyone. The machine does not need an internet connection to answer a question. If you unplugged the office router, it would keep working.
The confidentiality analysis changes completely when there is no third party. You are not vetting a vendor's security posture or negotiating a data processing agreement. The file never left your office, so there is nothing to disclose and nobody to trust.
What it is good at
A local 70B model handles the work that eats junior time and does not require frontier reasoning:
- Answering questions across a large set of case documents, with the passage it drew from
- Summarizing depositions, discovery productions, and long correspondence chains
- First-pass contract review against your own checklist of terms
- Drafting routine correspondence in your firm's established format
- Finding the specific paragraph across thousands of pages when you remember the substance but not the file
That last one is the capability firms end up using most and expected least. Search finds the word you typed. A model finds the passage that means what you described.
What it is worse at than the cloud tools
Being straight about this matters more than selling you the upside.
A 70B model running locally is not as strong as the best hosted frontier models on hard reasoning. On genuinely novel legal analysis, a complicated multi-step argument, or unusual jurisdictional questions, you will notice the gap. It is a very capable assistant, not a substitute for an experienced associate.
It also has whatever knowledge was in its training data and nothing newer, unless you feed it. It cannot look up a case decided last week. For current law, it works from documents you supply, which means your research process still starts somewhere else.
And like every language model, it can state something false with complete confidence. Every output still gets read by a lawyer. That is not a limitation specific to local models, but a local model does not make it go away either.
What it costs
Our install is $2,500 one time, which covers the on-site setup, the base model, and indexing up to 50 documents to prove it works on your material. Managed service, where we handle model updates, security patching, uptime monitoring and a custom workflow each quarter, runs $3,500 to $5,000 a month.
Ongoing electricity is roughly $9 a month. There is no per-seat licence and no usage metering, because there is no vendor charging you per token. Once the hardware is yours, using it more costs the same as using it less.
If you want the model to speak your firm's specific language, contracts, clause library and house style, a fine-tuning run is $800 to $1,500 and happens overnight.
Buying a machine does not make your firm compliant with anything. Your professional responsibility obligations are about your whole practice: who has access, how it is supervised, what you tell clients, how you handle a breach. Hardware is one input to that, not a certificate.
We sign a data sovereignty agreement before touching a file: no remote access to your data, no telemetry, no phone-home. We work on the system, never on your client files. What we cannot do is take responsibility for your professional obligations, and any vendor telling you otherwise is selling something.
When this is the wrong answer
Do not buy this if the work you want AI for does not touch confidential material. If you want help with marketing, internal process documents and general drafting, a business-tier cloud subscription costs a fraction of this and does the job better.
Do not buy it if nobody at the firm will own it. It is infrastructure. Someone has to care that it is running, the same way someone cares that the document management system is running. Our managed tier exists because most small firms do not have that person, but you should be honest about which situation you are in.
Buy it when the work you most want to automate is exactly the work you are not allowed to send anywhere. That is the case it was built for.
Find out whether your firm is actually the right fit
The free AI Audit maps which of your workflows would benefit, how much of that touches privileged material, and whether on-premise is justified or overkill. The plan is yours either way.
See what the audit covers