The Dunning-Kruger curve of AI on your data.

Somebody on your data team pointed an agent at the warehouse on a Friday. By Friday afternoon it was answering questions nobody had ever gotten around to building a dashboard for. By Monday there was a Slack thread with the phrase "we're basically an AI company now" in it.

I'm not making fun of that. It's genuinely impressive, and it genuinely took an afternoon. That's the problem.

The Dunning-Kruger effect is the observation that confidence runs highest when experience is lowest. You know just enough to think it's easy, and not enough to know why it isn't. Plot confidence against experience and you get the famous curve: a sharp peak, a long fall, and a slow climb back up to something you can actually stand on.

Running AI against production data follows that curve almost exactly. Connecting an agent to the warehouse takes an afternoon. Running one against production data takes a control plane, and most teams find that out from the bill.

The peak: "just give it a login"

The peak is built out of three perfectly reasonable shortcuts.

The agent needs to read data, so it gets a service account. Nobody knows yet which tables it will need (that's the whole point of an agent, it figures that out) so the service account can read everything.

The agent needs to answer questions, so it gets to write whatever SQL the question calls for. There's no limit on what one prompt can run, because nobody has stopped to ask what the worst query a prompt could produce actually looks like.

And the warehouse bills monthly, so nobody is watching the bill. Not out of negligence. The bill has always been a finance problem that shows up thirty days late.

None of these is a mistake on day one. On day one the agent is a demo, the demo is spectacular, and the demo runs a few dozen queries. Everything looks easy because, at that scale, everything is easy.

The valley: the invoice, then the incident

The valley doesn't arrive with an outage. It arrives with an invoice.

Agents ask the same question all day. Not in the same words (an agent will phrase a prompt a dozen ways) but underneath, it's the same tables and the same SQL, because there are only so many ways to ask for last quarter's revenue by region. A human analyst asks that once and bookmarks the dashboard. An agent asks it every time anyone asks it anything nearby, and every repeat is billed. The warehouse doesn't care that it computed this exact answer four minutes ago. It spins up and computes it again.

That's the first bill. The second thing in the valley is worse, and it doesn't come on an invoice.

Somewhere in your warehouse is a table nobody meant to expose. A staging copy with unmasked emails. A compensation table somebody loaded for a one-off analysis two years ago and never dropped. The service account can read it, because the service account can read everything, and eventually an agent, doing exactly what it was asked and looking for context, runs SELECT * on it and puts the result in a chat window.

Then somebody asks the obvious question: which agent ran that? And nobody can say. The warehouse audit log shows the service account. Every agent uses the service account. You have a perfectly accurate record that something happened, attributed to a login that everything shares.

That's the bottom of the curve. The first bill. The first incident. Confidence in AI on your data drops to roughly where it started, except now there's a steering committee.

The slope: put something in the query path

The way out of the valley is not better prompts. A prompt is a request, not a control. You can tell an agent not to touch the HR schema and it will mostly comply, right up until it doesn't.

The way out is to put something in the query path, between the agents and the warehouse where every statement has to pass, that does three things.

1. Cache the repeats, so they never reach the warehouse. Most of what agents ask has been asked before. A gateway that recognizes the repeat answers it from cache, and the warehouse never wakes up. The agent gets its answer in milliseconds and the warehouse bills nothing. The tenth identical question should cost zero, and with a cache in the path, it does.
2. Govern each statement with deny rules. A deny rule refuses a statement at the gateway before the warehouse is ever contacted. Match on the tables, the statement type, the SQL text, the user, the role, the headers the agent sends, the time of day. SELECT * on customer_pii from anything that isn't the approved extract job doesn't get slower or more expensive. It just doesn't happen. The agent gets back an ordinary SQL permissions error, the kind its driver already knows how to read, and moves on. Every deny rule is checked on every statement, and if the check itself fails, the statement is refused. A security control should fail closed.
3. Observe every query, attributed to its caller. Every statement that crosses the gateway is logged with who sent it: the user, a hash of the caller's token, the client address, the user agent. Give each agent its own credential instead of one shared service account, and "which agent ran that?" stops being an archaeology project and becomes a query.

None of this means rebuilding anything. The agents keep the same drivers and the same SQL. The only thing that changes is the hostname they connect to.

The plateau: agents at scale, on purpose

Look at the chart again. The plateau isn't as high as the peak. That's the honest part of the curve. You will never again feel the way you felt that first Friday, when it looked like the agent could do anything and cost nothing.

What you get instead is confidence you can defend. Answers at cache speed, with no change to your tools. A bill that stops tracking the prompt count, because more agents asking more questions doesn't mean proportionally more warehouse compute when most of those questions have already been answered. And every query has a name on it, so when something goes wrong (and something will) the first question has an answer.

That's what "at scale, on purpose" means. Not fewer agents. More of them, doing more, deliberately, because you can see what they're doing and you've drawn the edges of what they're allowed to do.

Skip the valley

Here's the thing about the Dunning-Kruger curve: everybody assumes you have to walk the whole thing. For personal expertise, maybe you do.

For infrastructure, you don't. The valley isn't a rite of passage. It's just what happens when the agents arrive before the control plane does. Nothing the first bill or the first incident teaches you is anything you couldn't have read off this chart.

And it's cheaper to start than most people assume. Point a connection at the gateway and nothing changes: queries pass through to your warehouse untouched and get logged, so you can see the repeats and the callers before you decide to cache or block anything. You turn it up when, and only where, you want.

So if you're on the peak right now, if someone on your team pointed an agent at the warehouse last Friday, congratulations. Genuinely. Now, before the invoice lands, put the control plane in before the rest of the agents arrive.