← All workInternship · National Association of Insurance Commissioners

SERFF-AI

Fine-tuning a legal LLM on 100,000+ insurance filings, and building the test suite for a platform 20,000+ regulators use.

Role
Software Engineering Intern
When
Jun – Aug 2025
Where
Kansas City
Stack
SaulLM-7BFine-tuningPyTestPlaywrightPython
+30%

testing accuracy over generic LLM inference

100K+

insurance filings used for fine-tuning

20K+

regulators on the platform

Context

SERFF is the system US insurers use to file rates and forms with state regulators. SERFF-AI adds AI review on top of it, and it is used by more than 20,000 regulators. I had spent the previous summer at NAIC on data and delivery work, and came back as a software engineering intern.

Presenting SERFF-AI at NAIC

What I built

  • A fine-tuned legal model. I fine-tuned SaulLM-7B, a model pre-trained on legal text, on 100,000+ insurance filings. Against generic LLM inference, automated testing accuracy improved by 30%.
  • Test infrastructure from nothing. The platform had no end-to-end or unit test suite. I built both with PyTest and Playwright, covering the validation workflows regulators rely on, which reduced regressions after releases.