AI Toolbox
Personal data stays on a local model
A company that gives its teams AI tools needs to know which tool may see which data, what it costs and whether it does the job. This demo answers that for the support team of a made-up e-shop. Before any model reads a customer e-mail, a gateway checks it for personal details: those e-mails go only to an open model running on my PC's graphics card, the rest to Claude within the team's rules and monthly budget. One MCP server gives each team only the tools it is allowed and logs every call. On 36 test e-mails graded by code, Claude got all 36 right and the local Gemma 32, so the dashboard keeps Gemma in pilot and every draft and ticket waits for a person. A second local model, gpt-oss-20b, got 26 and is marked rejected.
A demo AI toolbox for a made-up e-shop: a gateway keeps e-mails with personal details on a model in the PC and sends the rest to Claude, with team budgets, an MCP server and an eval.
How it works
The routing: before the first model call, the gateway sorts each e-mail into personal, internal or public data. Personal goes only to Gemma 4 12B on my PC's graphics card; internal and public go to the team's Claude model while its monthly budget lasts. If a tool result in a cloud run turns out to hold personal details, the run stops and the e-mail runs again locally.
The tools: one MCP server gives each team only the tools the registry allows (find an order, search and read the wiki, open a ticket, check its status) and logs every call, refused or not, with personal details replaced by their kind.
The agent: it reads one e-mail, looks up the order, finds the rule in the wiki, picks one of 13 decisions and opens a ticket where the wiki asks for one. Code checks the draft (the decision, the cited paragraphs, every date and number, no request for a card or password by e-mail) and sends a failed one back once. Nothing goes to a customer: drafts and tickets wait for a person.
The eval: 36 made-up Czech test e-mails, graded by code. Claude Opus 5.5 and Haiku 5.5 got 36/36, at $0.0183 and $0.0006 per e-mail. Gemma 4 12B got 32/36 (88.9 %), under the registry's 90 % rule; gpt-oss-20b got 26/36, with 28 decisions right (77.8 %). The gateway as support would use it, personal e-mails on Gemma and the rest on Opus, got 33/36 at $0.0073 per e-mail.
The registry: one file lists each tool's status, the data it may see, its licence, cost and the evidence behind it. By its own rule Opus and Haiku are approved and gpt-oss-20b is rejected. Gemma stays in pilot even though personal e-mails can only go there, so a person checks every draft.
Limits and decisions
- A made-up shop, not a real support desk. Every order, customer, wiki page and team is made up, and the demo runs only on my PC. Nothing was sent to a customer, and the dashboard is not online.
- Not an independent test. The same AI agent wrote the system and the 36 test e-mails, and the e-mails were written knowing the privacy filter, so its catch rate here says little. How good the reply drafts read and how much staff time they save are not measured.
- The local model is not approved. Gemma misses the 90 % rule, yet e-mails with personal details may only go to a local model. The registry records that conflict instead of hiding it: until a local model passes, a person checks every draft and ticket. The second local candidate, gpt-oss-20b, did not pass either (77.8 %).
- Local times on a shared card. Gemma's run shared the graphics card with another local job and a game, so its 20.1 s median per e-mail is not a clean speed test. gpt-oss-20b had the card to itself (4.8 s median), so the two times cannot be compared.
What's inside
- Before the first model call, the gateway sorts each message into one of three data classes
- A cloud run stops if a tool result turns out to hold personal details
- The local model is Gemma 4 12B (QAT Q4_0, Apache 2.0) in llama.cpp on the PC's RTX 5070 Ti 16 GB…
- One MCP server offers the support tools (find an order, list, search and read the wiki, open a…
- The support agent reads one e-mail, looks up the order (code has already counted the days and…
- The eval
Built with: Claude API, MCP, llama.cpp, Python, Claude Code
Previous project: CRM Campaign · Next project: Job Outreach · All projects