AI for accounting firms, without sending client data anywhere
The AI I build for accountants and Italian tax assistance centres (CAF) runs on a server inside the firm: it classifies incoming documents, extracts the fields you need, searches tax authority guidance and drafts replies to clients. Tax returns, payment forms and correspondence stay where they are.
Why keep the AI inside the firm
An accounting firm works on data its clients hand over under professional confidentiality: income, dependants, medical expenses, outstanding debts. Pasting it into a public chatbot means transferring it to an outside provider, often outside the EU, under a contract the firm has to have read and assessed under GDPR.
A local language model (an open-weight LLM with 8 to 32 billion parameters) installed on the firm's own server does the same work on documents and text with no outbound connection. It is not the largest model on the market, and it doesn't need to be: classifying a tax payment form, reading a tax code or summarising a circular are narrow tasks where a well-instructed model of this size is reliable.
The flow, from incoming document to client reply
Automatic document classification
Every incoming file (certified email, email, shared folder, scan) is recognised: tax return, F24 payment form, income certificate, company record, tax authority notice, invoice, correspondence. The document lands in the right case file, with a label the operator can correct.
Structured field extraction
From the classified document the model extracts fields as validated JSON: tax code, VAT number, tax codes, amounts, reference period, deadlines. Formal checks (tax code checksum, totals, dates) are plain code, not AI: if a field doesn't add up, the case goes to review instead of passing through.
Search across tax guidance
Circulars, rulings and published answers from the Italian Revenue Agency, plus the firm's internal notes, are indexed in a semantic search store (RAG). The operator asks a question in plain language and gets an answer with references to the document and paragraph it came from, to be checked before use.
Draft replies to clients
For recurring requests (missing documents, deadlines, outcome of a check, explaining a notice) the system drafts a reply in the firm's tone, with the case data already filled in. Signing and sending stay with the operator.
Many operators at once, in tax season
Parallel requests, not a queue
The model runs behind an inference server with continuous batching (vLLM, for example): requests from several operators are processed together on the same GPU instead of waiting one after another.
Heavy jobs off-hours
Classification and extraction over the day's batch of documents can run overnight or at lunch. During the day, capacity stays free for search and drafting, which are the interactive requests.
Sized on the real peak
Before choosing hardware, we measure: how many concurrent operators on the worst day, how many documents a day, how long a typical document is. Model size and number of GPUs are decided on those numbers, not on a hunch.
Graceful degradation
If load exceeds capacity, interactive requests take priority and batch jobs slow down. No operator is left staring at a spinner.
What the model has to learn, and what it only has to look up
This is the most common mistake in AI projects for accountants: thinking you need to "train" the model on tax guidance so it knows it. It doesn't work well and it ages badly, because the rules change every year. To make the model know your documents you use RAG: the text stays in a store, the system retrieves the relevant passages for each question, and the model answers citing them. A new circular is added to the store, with no retraining.
Fine-tuning is for something else: style, format and classification. With a few thousand extraction examples corrected by the firm's own operators, a light fine-tune (LoRA) on an 8-32B model takes a few hours of compute and teaches the model to always return the same JSON, recognise the documents the firm actually handles, and write in its register. The data stays on the same server during training too.
Public chatbot vs accounting software with cloud AI vs private on-premise AI
| Public chatbot | Accounting software with cloud AI | Private on-premise AI | |
|---|---|---|---|
| Where the data goes | The provider's servers, often outside the EU | The software vendor's cloud | The firm's own server |
| Knows the firm's cases | No: you paste the text | Only data inside that software | Yes: documents, archive, internal notes |
| Output format | Free text | Set by the vendor | Set by the firm (JSON, fields, labels) |
| Cost | Per-user subscription | Per-user or per-module fee | Hardware and initial project, then maintenance |
| Best when | Generic questions with no client data | The firm already uses that software and it's enough | High volume, many operators, data that must not leave |
Stefano Salvucci
Engineer registered with the Order of Engineers of Rome, 12+ years across software development, architecture and automation, co-founder of BusinessinCloud. I design and put AI systems into production: from choosing the model and the hardware to integrating with the folders, certified email and accounting software the firm already uses. I work as a fractional CTO: we decide together what is worth automating, and I build it.
AI for accountants and tax centres: common questions
What is the best AI for accountants?
It depends on what it has to do and where the data is allowed to be. For generic questions a public chatbot is fine. For working on client tax returns, payment forms and correspondence you need a system that stays inside the firm, with an 8-32B open-weight model chosen and instructed for those specific tasks. The model is chosen by testing it on the firm's real documents, not on leaderboards.
Does client data leave the firm?
No. Model, search store and documents sit on a server at the firm or tax centre, with no calls to outside AI services. Fine-tuning happens on the same machine. The usual GDPR work remains (records of processing, access control, backups), but without a new outside provider to transfer data to.
Is an 8-32 billion parameter model enough?
For classification, field extraction, document search with RAG and draft replies, yes: these are narrow tasks. The model doesn't need to know the tax rules by heart, it reads them from the store. Where longer reasoning is needed you move towards 32B; where speed with many operators matters you move towards 8B.
What hardware is needed?
A server with one or more GPUs, sized on the number of concurrent operators on the peak day and on document volume. For a 5-50 GB archive, disk space is not the issue: GPU memory is, because it decides which model you can run and how many requests you can serve in parallel. We define it after measuring the real load.
Does AI replace the professional's review?
No. It classifies, extracts and proposes; the operator checks and signs. Fields that fail formal checks go to review, and every answer about tax rules shows the source it came from.
Does it integrate with the accounting software we already use?
Usually yes, through shared folders, certified email, exports and the software's APIs where available. The goal is for operators to find the extracted data where they already work every day, not in yet another application.
How do we start?
With a free 30-minute intro call. You tell me the firm's volumes, documents and bottlenecks, and I tell you whether on-premise AI makes sense for you or whether something simpler will do.
Let's talk about your firm
30 minutes, free: we assess whether and how on-premise AI can help your practice.