AI Agents in Compliance Reporting: Where It Breaks
Evidence gathering is a fire drill every cycle.
The loop, as it actually runs
- Requirement identified
- controls mapped
- evidence collected
- reviewed
- report assembled
- filed
Where it breaks
Not in the steps. In the joins between them:
- Evidence is gathered reactively
- Collection is manual across many systems
- Reviews happen too late to fix anything
- Reports are rebuilt from scratch each period
What that actually costs
Evidence gathering as a fire drill costs more than the hours it consumes, because the timing is what makes it expensive. Collecting evidence at the end of a period means discovering control failures after the period they occurred in, when nothing can be done except document them. The same collection running continuously would have surfaced the same failure while it was still fixable. You are therefore paying for the gathering twice: once in the scramble, and once in the findings that would not have existed if the scramble had happened earlier.
What we would build first
Continuous evidence collection goes first, before mapping or report assembly. Assembly is the visible pain — rebuilding the report from scratch each period — but assembly is fast once the evidence exists in a structured place. Collection running on a schedule against each system converts the fire drill into a background process and, more importantly, moves the moment you learn about a control failure to a point where you can still correct it.
It runs inside Google Drive, Airtable, Slack, Notion, Gmail — the tools already in use.
How this one goes wrong
An agent that reports controls as satisfied because it found the expected artefact is checking for the presence of evidence rather than for the control working. A log file exists, a review was recorded, an approval was captured — none of which establishes that anyone looked. Automated collection makes this failure easier, because volume creates the impression of rigour. The guard is to keep the agent scoped to gathering and to leave the assertion to a person: it collects, timestamps and flags what is missing, and a named human signs that the control operated. Software should never be the thing that says a control passed.
The individual jobs inside this loop — what starts each one and where it has to hand back to a person — are covered under admin & contracts.
Where it lands hardest: staffing & recruiting agencies, accounting & CPA firms, insurance brokerages, mortgage brokerage and medical & dental practices. Each of those pages carries the sector’s own economics — the three roles that absorb the work, and the season that decides when a build can start.
In most companies this work sits with accountants, HR specialists, payroll clerks, operations managers, medical office administrators, claims processors, loan processors, benefits administrators and management analysts. Each of those pages carries the wage arithmetic for that role, and says plainly where the numbers do not justify the spend.