The AI P&L Gap Is Not a Technology Problem
A MIT study tracked 52 executive interviews, 153 leaders, and 300 public deployments. The headline: 95% of AI pilots show no measurable P&L impact. When I shared that finding in a full-day workshop this summer, nobody looked surprised. The room nodded. Then someone said something sharper than the paper itself: we keep delegating AI to the wrong people.
That’s the thesis. The gap between AI experiments and AI results is not a model problem. It’s an operating model problem. And it has five specific, fixable parts.
1. The Wrong Department Owns the Wrong Problem
When AI gets handed to IT like a software rollout, the people who understand the actual process aren’t steering it. IT can deploy — but IT cannot tell you where the friction and the money sit. Those are your subject-matter experts: the ops lead who knows why invoices stall, the underwriter who spots the case that needs more time. They need to own the use case. IT’s job is to make it possible: guardrails, credentials, deployment.
The AI vendors figured this out before most of their customers did. The largest frontier labs built their own forward-deployed engineering units this year — pairing deep technical capability with domain expertise on-site. That’s the model. Not IT owning AI. Experts leading, IT enabling.
Ask yourself: Who in your company has the permission, today, to actually build something?
2. Two Ladders — Don’t Confuse Them
I’ve run enough workshops to know the mistake before the room makes it: organizations map their maturity level onto their technical customization level and assume they’re the same number. They aren’t.
Maturity runs from chatting (individual prompts) → working (real tasks, blank screen every time) → reusing (shared context, personas, playbooks) → delegating (agents) → reinventing (redesigned processes). Most rooms raise hands at one or two. Level three is the first that compounds — because shared context accumulates instead of being re-created from scratch each time.
Customization is a different scale entirely: off-the-shelf → RAG → fine-tuning → custom model. A company at maturity level two should stay at customization level one for now. Off-the-shelf is the same model your competitor already pays for. Moving up the technical ladder is worth progressively less without a data strategy underneath it — and without the organizational maturity to absorb it.
Know which ladder you’re actually on before you plan the next rung.
3. The Data Asset You’re Not Building Today
This one came from a peer exchange in the YPO technology network. Two firms, same database. One trained a model on 18 months of cases, briefings, and decision patterns. The other kept using retrieval — searching for the answer each time.
The difference: RAG searches. A custom model knows.
The gap widens over time, quietly. The firm still searching is one fine-tune away from being lapped — by a competitor who started months ago, on data that felt routine at the time.
I tell every leadership team the same thing now: the model you’ll want in two years is hungry for the data you’re not capturing today. Looking something up is a fundamentally different act from having lived through it. Start the data strategy now — not when you’re finally ready for the custom model.
4. A Fallback You’ve Never Run Is a Hope, Not a Plan
A frontier model was withdrawn this year, came back throttled, and quietly rolled back to its predecessor in production. Teams noticed it in output quality — not in a vendor notice. That should concern any CEO running meaningful workloads on a single model.
Vendor independence isn’t enough. You need model independence.
That means: same task, run on a second model, tested on a schedule. Document which workloads switch, to what model, who decides, and how you’d know the difference. You wouldn’t accept a single supplier for a core component without qualifying a second source. The logic is identical here. Self-hosted, open-weight models are the strongest hedge against a vendor withdrawing access or repricing overnight. They’re not always the right primary choice — but they’re a real fallback. One you’ve actually practiced.
5. Governance That Only Says No Produces Shadow AI
I’ve watched it happen. A blanket AI policy gets issued. Within weeks, private accounts appear. People work around the restriction because the work still needs to get done. Governance that only says no doesn’t stop AI use — it just makes it invisible.
The better model is a corridor: define what a tool may do, not only what’s banned. Keep money leaving the company, customer-facing interactions, and anything irreversible outside the corridor. Widen it on a measured error rate — not on enthusiasm, not on vendor demos.
Guardrails, not gates. You won’t out-engineer a frontier lab’s containment. Limit the blast radius instead: permissions, connectors, memory, reach. Good governance is measured by how much your people may safely do — not by how much is banned. Write down what’s allowed.
The AI Operating Layer: Five Questions
This is the framework I now walk through with every leadership team before any AI investment discussion. I call it the AI Operating Layer — five questions a CEO should be able to answer in under a minute:
| Layer | The Question |
|---|---|
| Ownership | Which subject-matter expert owns each use case — not IT? |
| Maturity | Where is the organization on the maturity ladder, honestly? |
| Data | What decisions are you making today that leave no structured trace? |
| Resilience | Which workload would you move first if your primary model went offline tonight? |
| Governance | Where does your current AI policy only say no? |
If you can’t answer two or more of these in under a minute, that’s the work.
The Moment the Room Went Quiet
In July, I ran a full-day workshop with a leadership team that had already spent six months piloting AI. Good company. Serious, experienced people. When we mapped these five layers together, the conversation moved quickly — until we hit the data question.
Someone said: “We’ve been running customer interviews for two years. We’ve never stored the outputs in a structured way.”
That wasn’t a technology failure. It was a strategy assumption nobody had surfaced until we named the question directly. The data was walking out of the room every time. The future model — the one they’d eventually want — was going to be hungry for exactly what they’d been letting disappear.
It’s the kind of insight that’s obvious in retrospect and invisible until you look for it.
Five Things to Do Next Week
- Name one owner. Pick your highest-priority AI use case and assign a domain expert — not an IT lead — as directly responsible.
- Run the maturity ladder exercise. In your next leadership meeting, have each function honestly place themselves on the scale: chatting → working → reusing → delegating → reinventing.
- Map your data gaps. List three recurring decisions your team makes that leave no structured trace. Those become your data strategy starting points.
- Test your fallback. Pick one live AI workload and run it on a second model this week. Time it. Note what breaks or degrades.
- Rewrite one governance rule. Find one “no” in your current AI policy and convert it into a corridor: define what is allowed, with clear boundaries, instead of what isn’t.
The MIT finding isn’t a verdict on AI. It’s a diagnosis. Most pilots fail not because the technology doesn’t work — but because the organization hasn’t decided who owns it, what to build on, how to recover, or what’s actually safe to try. Each of those is a leadership decision, not a technical one.
Where does your AI operating model have the biggest gap right now — ownership, maturity, data, resilience, or governance?