Skip to content
Nash Software Services
NASHSOFTWARE SERVICES
← Writing
·Mike Nash

What I Find When I Audit AI-Assisted Builds

Most AI-assisted builds have the same security gaps. Authentication looks right until you trace it. Here's what I find when I audit before go-live.

AISecurityProductionAudit

If you're about to go live with a system an external team or AI-assisted agency built for you, there's a question worth being able to answer: have you actually looked at the authentication? Not "yes it's there" - traced it through, confirmed what happens at the edges, checked which endpoints bypass the middleware? Most of the time, honestly, the answer is no.

The stuff I find isn't subtle. JWT tokens with no expiry, endpoints that bypass the middleware entirely, session handling that would let someone enumerate users if they knew where to look. It has the right shape, the right variable names, the right comments. It passes a visual scan. It might even pass a review from someone who isn't looking for the specific thing that's off. But it wasn't built by someone who's seen what happens when these patterns fail in production, and that shows in the decisions. The gap is predictable once you know what you're looking for.

Authentication is usually the most visible gap. The other things I find regularly: data validation that stops at the frontend and assumes the backend is safe, error handling that leaks stack traces to the client, third-party integrations with API keys in places they shouldn't be. None of it is catastrophic on its own, but it adds up.

Look, the teams and agencies building these things aren't necessarily cutting corners. AI is fast, and fast gets you to something that works quickly. But a working demo and a production-ready system are different things - and the gap between them is hard to see when you're the one who built it. AI writes to the spec it was given. It doesn't have opinions about what the spec should have included.

Before you go live with something handling sensitive data, financial transactions, or anything you'd be embarrassed to explain to a customer - make sure someone with no stake in the outcome has had a proper look at it.

I don't know why this isn't standard practice yet. It seems fairly obvious in retrospect.

If this is where you are, the AI & Agent Production Audit is what it's for.

Share

Interested in bringing this kind of AI-assisted delivery to your team?

See AI Code Audit

Related posts