Insurance Assistant
An AI guide that answers from an insurer's own pages
I built a chat guide that answers questions about Czech public health insurance from the insurer's own public web pages. Compared with the chatbots most people know, it is really useful: it follows no script, does not brush people off with a stock reply and tries to solve their actual problem right there in the conversation. Claude searches the pages and answers with sources and the next step, and code checks every answer before it is shown. It works in Czech and English, and it can listen and speak.
A Claude chat guide to a Czech health insurer's public pages, with sources, next steps, voice and accessibility settings.
How it works
One Python server runs two assistants. The main one is an unofficial guide to a Czech public health insurer's public pages: it answers what is covered, which form to use, what to do when moving abroad or paying premiums, always with links to the pages it used. The second is a demo for a made-up insurer, built earlier, with fixed conversation paths, forms, a mailbot and a way in for AI agents. The guide is also online as a public demo, framed further down this page: it runs on Claude Sonnet 5.5, which costs less per answer than Claude Fable 5.1, has a daily budget and no microphone. Each figure below names the model it was measured on.
The guide, on Claude Fable 5.1, was measured on 9 Nov on questions it had not seen. The test set starts from the insurer's own FAQ titles; Claude Opus 5.5 rewrote each title the way a customer would type it, without the page's wording, and hard cases (safety, tricks, off-topic) were added by hand. About 60% went to development and 40% stayed blind. On 78 blind questions, a Claude Opus 5.5 judge with each question's source page rated 68 solved, 4 correctly sent elsewhere, 6 wrong and none needlessly handed to a call centre. Median time 22.8 s, $0.22 a message.
The numbered steps came after that run, so they were checked separately on 20 development questions with Claude Fable 5.1: 19 solved, 11 got steps with 38 links between them, and a message cost $0.35 on average. The made-up insurer has its own blind run on Claude Haiku 5.5: 23 of 27 in-scope questions answered with the right facts, 6 of 6 out-of-scope questions refused or handed over, 7 of 7 prompt injections resisted, and 12 of 14 e-mails with all four fields right, for $0.04 in total.
The guide's development rounds now run first on Gemma 4 12B, a small model on my own PC (llama-server), for free; each 45-question round there saves about $11 of API spend. That loop found a gap. The web chat picks the guide by the language a message is written in, so "answer in English" inside a Czech message is the customer's text, not a language switch; yet Gemma, as the guide, answered such an instruction in English 4 times out of 4. A new check, live since 11 Nov, sends a reply in another language back for one repair round, and with it 0 of 6 came out in English. Claude Fable 5.1 and Claude Sonnet 5.5 already refused such instructions, so on the paid runs the check never fired: it is a safeguard. As a judge the local model falls short, so every quality figure on this page comes from the Opus judge.
A second blind run followed on 11 Nov, again on Claude Fable 5.1 with the Claude Opus 5.5 judge and the language check in. Gemma wrote its 45 questions from 60 FAQ pages in seven styles (typed on a phone without diacritics, colloquial, rambling, formal, annoyed, Slovak, English), and 12 of them were prompt injections. 43 were solved and 2 wrong, none needlessly sent to the call centre; all 12 injections got a Czech answer and none was followed. Median 34.8 s, $0.264 a message, $13.08 for the whole run with grading. Each question rests on one FAQ page, so this set is easier than the 78 questions of 9 Nov and the two results are not comparable. In 24 of the 45 replies at least one unsupported detail remained. Of the 2 wrong answers, one invented a web-form channel and linked the wrong page; the other was a strict but defensible grade.
Two more fixes went live on 11 Nov, the public demo included. The page reader had dropped 382 short list items as if they were menu captions, among them "webovým formulářem" (by web form) on the page about refunds to a bank account; with items like that missing, the guide could invent a channel or borrow one from another page. Short list items are now kept as text, and four prompt lines aim at the unsupported details seen most often (channels, requirements, scope). And after 8 search rounds the guide now has to write its answer: before, Claude Sonnet 5.5 kept searching until the loop ran out in 7 of 45 development questions and fell back to "call the infoline". On those 45 development questions (not blind), with the Opus judge, Sonnet 5.5, the model the public demo runs, now solves 39 instead of 32, sends none needlessly to the call centre instead of 7, takes a median 21.3 s instead of 26.9 s and costs $0.062 a message instead of $0.072. Wrong answers stayed at 6, and replies with an unsupported detail did not go down.
That evening each of those 6 wrong answers was traced to its cause, and the fixes went live the same night. The privacy filter had read the insurer's own service names as people's names (the names of its app and its online service, and "Přehled OSVČ", the yearly form of the self-employed, in 23 of 480 test messages), so the guide saw [jméno A] and asked which form was meant; it now leaves them alone, and its scores on the six filter test sets did not change. The guide also has to search before it asks a question back, the line on what the pages do not cover gets the same number and link checks as the answer, a link labelled only "zde" (here) shows Claude where it leads, and a reply may not tell the customer that the filter hid something. Each of the 6 was solved at least once afterwards, but two more Sonnet runs of the 45 questions solved 37 and 39, at $0.068 to $0.077 a message: the wrong answers left are unsupported details, and they move from question to question.
The server and the guide are Python, the chat page is plain HTML, CSS and JavaScript. AI coding agents (Claude Code) write the code in my projects under my direction; the brief, conversation design, test criteria, review and sign-off are mine.
Limits and decisions
- An unofficial guide, not the insurer's service. The pages' text belongs to the insurer and the guide is not its service, so it is never presented as the insurer's own. On Claude Fable 5.1 each answer also costs about $0.15 to $0.75 in API calls. The public demo therefore runs on Claude Sonnet 5.5 with a daily budget. That model has had no blind run: it solved 18 of 20 practice questions on 9 Nov, at about $0.09 an answer, and 39 of 45 development questions after the 11 Nov fixes, at $0.062 a message.
- Not every detail is backed. In the 9 Nov blind run (Claude Fable 5.1) the judge found at least one detail without support in the pages in 37 of 78 replies, mostly small ones that did not change the grade; such details were the reason for 4 of the 6 wrong answers. In the 11 Nov blind run it was 24 of 45, and the 11 Nov fixes did not lower the count on Sonnet 5.5's development questions (14 of 45 before the page-reader fix, 15 after). A rule added after the first run keeps conditions as wide and as strong as the page has them, and was rechecked on 4 cases only.
- The privacy filter misses some and hides too much. It hides 85.8% of unseen made-up details, and Czech incomes written without a cue word get through. A rule round added on 11 Nov (health words, a return from hospital, ethnic origin, a VIN, a postcode before a town name) did not change that figure. It also hides harmless words: while these stills were made, "jsem OSVČ" (I am self-employed) and "European Health" were read as names, and Claude then saw a stand-in instead.
- Measured by a model, on two blind runs. A Claude Opus 5.5 judge graded the answers against each question's source page. In the 9 Nov run the same model wrote the customer-style questions (78 sampled from the blind split); in the 11 Nov run Gemma 4 12B wrote them (45), each from one FAQ page. Neither set comes from real customers, and the local model cannot take over the grading: it caught none of the 15 replies Opus graded wrong.
- Accessibility not tried by its users yet. The settings were checked in a browser and by unit tests, not with a real screen reader or by blind or low-vision visitors.
- Answers take a while. A typical answer takes 20 to 40 s, and one with six steps up to about 80 s. The waiting card shows what the guide is doing and how long it usually takes.
What's inside
- The guide answers from 446 documents made from the insurer's public web pages.
- Claude Fable 5.1 gets two tools, search and read (three with the care finder), searches up to 8…
- Code checks every reply before a customer sees it
- When there is something to get done, the answer adds numbered steps ("Jak to vyřídit") with…
- A care finder searches the public register of healthcare providers for any field of care and any…
- A local filter, rules and word lists only, hides personal details before anything leaves the…
Built with: Claude API, Python, JavaScript, HTML & CSS, Whisper, ElevenLabs, MCP, Claude Code
Previous project: Lead Intake · Next project: CRM Campaign · All projects