The AI Skill Your Company Forgot to Teach
For the last few years, companies told engineers to use as much AI as they could. Unsurprisingly, devs took this blank check and ran with it. Amazon had to scrap its AI leaderboard because employees were competing to see who could rack up the biggest token spend.
This year the message flipped. Uber blew through its entire 2026 AI coding budget by April and now caps each employee at $1,500 a month per tool. Meta's CTO told staff that “token usage alone is not a measure of impact of any kind.”
Through it all, orgs forgot to teach engineers how to spend tokens the right way. In Stack Overflow's 2026 developer survey, three in ten respondents said AI use at work was optional and left to them, and only 24% said their employer had published approved tools or formal AI guidance.
The skill companies skipped is matching spending to impact.
Two habits burn most of the budget
Most surprise bills come from two habits:
- Long sessions that never get cleared. Every request resends the full conversation, so a quick question late in an all-day session pays for everything that came before it.
- The frontier model doing everything. That's the newest flagship model, usually the most expensive one, and it ends up handling routine work a cheaper model would do just as well.
Why context costs so much
A
Stanford study of coding agents found agentic tasks use about 1,000 times more tokens than code chat. Most of that cost is input, meaning everything the agent reads before it writes anything.
Put the expensive model where the judgment is
Not every part of a task needs the frontier model. Choosing an approach and weighing tradeoffs does. Writing code against a plan that's already settled usually doesn't.
My setup splits the work across agents with defined jobs: planning, design, security review, and PR review. Planning runs on a frontier model. Implementation runs on a mid-tier model that costs about half as much, and quality hasn't dropped.
Half the price is meaningful on its own. Pair it with the context habits below and the savings stack, because you're paying a lower rate on fewer tokens.
If your tool lets you define agents, give each one:
- A clear job, so it isn't guessing at scope.
- Instructions specific to that job. A security agent should already know your auth patterns, and a database agent should already know your schema conventions.
- The cheapest model that does that job well.
Separate agents also keep their context separate, so the files and logs a review agent reads don't pile up in your main session. Each agent still costs tokens to run, though, so five agents aren't automatically cheaper than one.
Keep a gauge in front of you
Context drives cost. Every turn resends what's already in the context window, so each turn in a long session costs more than the one before it. When my context gets heavy, I can watch my per-session spend climb faster with every turn.
I built a custom status line so I can see where I'm sitting at a glance. Whatever tool you use, have a gauge you can check while you work, not after the bill shows up.
What to put on your gauge
At minimum: how full the context window is, what the current session has cost, what you've spent today, and which model you're on. A burn rate per hour helps you catch a runaway session early.
When the context gets heavy, you have two options:
- Start a fresh session when you switch to unrelated work. Old context does nothing for the new task except cost money.
- Summarize the conversation when you're mid-task and need the thread. Tell the tool what to keep, like decisions made and tests still failing.
Summarizing isn't free, either. The tool has to read the whole conversation to summarize it, so summarizing a huge context is a big request on its own.
A gauge also makes it easier to follow the rule from August about restarting after two failed corrections.
|
|
Where can AI save you time?
My friends at Big Creek Growth put together a quick survey to spot the repetitive work you can hand off to automation.
|
|
Measure spend against what shipped
Token spend by itself doesn't tell you much. A dev spending $1,000 a month to ship large, complex features is a better use of budget than a dev spending $100 a month with nothing to show for it. The second dev looks better on a cap report, but the first dev is the reason the company pays for these tools.
The same holds within a single task. In the Stanford study, runs of the same task varied by up to 30x in tokens, and spending more didn't produce better results.
To measure your own spend against impact:
- Pick a unit your team already tracks, like features shipped, tickets closed, or incidents resolved.
- Track your agent spend per unit for a few weeks.
- Bring the numbers to your next one-on-one and talk about what the spend bought.
A cheaper run that needs a second pass to fix isn't cheaper.
Teach the people around you
Since orgs skipped the training, the devs who've figured this out have to teach the rest of the team.
- Commit your agent definitions and instructions to the repo so teammates start from the same setup. As I wrote in September, the repo is the only onboarding an agent gets.
- Share your gauge. Show teammates what you track and why, and help them set up their own.
- Walk people through a real session, including the points where you started fresh or switched models.
A cap tells you when to stop. It can't tell you which model to use, when to start a fresh session, or whether a feature was worth what it cost.
Those are judgment calls, and your team only gets better at them if the people who've figured them out share how.