RAG copilot
REGULATED B2B SERVICES · Compliance

A RAG copilot over 60,000 compliance documents that answers in seconds, with citations.

Answer accuracy on eval set

~94%

5 weeks · 1 PM + 2 engineers

Per-query turnaround

~3 days → < 1 min

Documents indexed

60k+

Answers cite source

100%

“It is the first internal AI tool the team trusts. Mostly because it shows its working and tells us when it does not know.”

Head of Compliance, regulated B2B services firm

Before

Where the team was when we picked this up.

  • Analysts were spending two to three days per query digging through PDFs, regulator websites and internal memos.
  • Off-the-shelf chat tools hallucinated rules that did not exist or cited the wrong jurisdiction.
  • Knowledge sat with three senior people. When they were on leave, the team slowed to a crawl.

What we built

Document pipeline

Cleaned and chunked the archive with a custom parser tuned for regulatory PDFs. Hybrid retrieval (semantic + keyword) so jurisdiction-specific terms do not get lost.

Answer model with citations

Every answer is grounded in retrieved passages and linked back to the source document and page. If the corpus does not support an answer, the copilot says so.

Eval harness

Built a question set with the senior analysts. We rerun it weekly so accuracy regressions get caught before users notice them.

What changed

  • Junior analysts now produce first-pass answers in a few minutes.
  • Seniors review and sign off rather than research from scratch.
  • When someone is on leave, the work keeps moving.

After

Same team. Same week. Different shape of work.

Stack

Anthropic ClaudeOpenAI embeddingspgvectorPostgresBM25Custom evals

Timeline & team

5 weeks · 1 PM + 2 engineers

Got a workflow like this one?

Book a working session. We will tell you whether this is a four-week build or something bigger, and what it would take to ship it.

Book a working session

Cookies. Sadly not chocolate chip.

We use cookies to keep the site working, understand what is useful, and avoid shouting ads into the void. You can accept all, reject non-essential, or choose your own settings.

More detail lives in our Privacy Policy and Terms.