When Not to Use AI
For a long stretch, AI was my default first move. A new feature? Ask the assistant how to approach it. Hard message to write? Let it draft. Design doc due in a couple of hours? Hand it a few bullet points and let it fill in the rest. It was fast, and fast felt like progress.
hat I wasn’t doing was asking whether the task was a good fit for AI at all. I shipped things I hadn’t really read. I produced documents that looked finished but that I couldn’t defend when someone pushed on them in the meeting afterward. The work got faster and thinner at the same time, and it took me a while to connect the two.
So I changed how I start. Before reaching for AI, I decide what it’s good for and what I need to work through myself.
AI multiplies judgment you already have. It cannot supply judgment you don’t.
Now I run a task through three questions before handing it over:
- Can you check the answer? If you can’t tell a good output from a bad one, going faster doesn’t help you.
- Was the thinking the point? Some work exists to produce a document. Some exist to help you understand something, and AI skips understanding.
- If it turns out wrong, how soon and how expensive? Cheap and obvious mistakes are fine to take a risk on. Costly ones that are not obvious are not.
AI Slop is a Comprehension Issue
The promise of AI is speed. You describe what you want, it produces something, and you move on.
That only works if the thing it produces is right, and checking whether it’s right is work AI doesn’t do for you.
On a task you know well, the check is fast. You read the output, catch what’s off, fix it. The tool saved you the typing, and you supplied the judgment. On a task you don’t know, there’s nothing to check against. The output looks reasonable, and “looks reasonable” becomes your entire quality bar.
A 2023 Stanford study tested this. Researchers had 47 people write security-related code, half with an AI assistant and half without. The group with the assistant wrote less secure code. They were also more sure their code was safe. They did worse and felt better about it.
The people who did best trusted the AI least and pushed hardest on their prompts. Several said they leaned on it most for the technologies they understood least.
The instinct is to lean on AI hardest where you’re weakest, because that’s where it feels like the biggest lift. But weak is exactly where you can’t grade what comes back. You don’t get an expert. You get a confident answer in an area where you can’t tell confident from correct.
GitClear’s 2025 analysis of 211 million changed lines points in the same direction. As AI tools spread, duplicated and copy-pasted code rose sharply, and refactoring fell so far that 2024 was the first year in which copy-pasted lines outnumbered reworked ones. Reusing code means understanding what’s already there.
Pasting around it doesn’t.
AI is safe for work you could have done yourself and chose not to. It’s risky on the work you’re handing over because you couldn’t do it, since that’s the work you have no way to grade.
Understanding Types of AI Work
Some work exists to make you understand something. The document it produces is just what’s left over.
A design doc is that kind of work. So is a postmortem. So is the message you send a teammate when you need to push back on their approach. The text was never the value. The value was you thinking it through until you understood the problem well enough to write it down.
Hand that to AI, and you get the text without the thinking.
It reads fine. It might read better than what you’d have written. But the understanding the document was supposed to produce never happened, and that shows up later. In the meeting where someone asks a question the doc should have already answered. In the decision you can’t defend because you never really made it.
The output looks like success. You have the artifact. You shipped it. But a design doc you didn’t think through is a problem in disguise, and the next person trusts it and builds on reasoning that was never there.
If you’re using AI to produce something whose only purpose was to change what’s in your own head, that’s the work to keep.
|
|
Where can AI save you time?
My friends at Big Creek Growth put together a quick survey to spot the repetitive work you can hand off to automation.
|
|
All AI Work is Not Equal
Not every mistake costs the same, and not every mistake announces itself.
The safest work for AI is where errors are cheap and loud:
- A throwaway script you’ll run once and delete.
- A first-draft email you’re going to reread before it goes anywhere.
- A function wrapped in tests that fail the moment something breaks.
Get those wrong and you find out fast, at almost no cost. The dangerous work sits in the opposite corner, where being wrong is expensive, and the error stays out of sight:
- A database migration that touches live data.
- A change to who can access what.
- An estimate that the next quarter of planning gets built on.
Here, the output looks finished, you move on, and the bill arrives weeks later when the problem is harder to trace and costs more to undo.
Carnegie Mellon researchers tracked 807 open-source projects that adopted an AI coding tool and compared them to 1,380 similar projects that didn’t. The adopters saw a sharp jump in development speed that faded within a month or two. The quality cost didn’t fade: static-analysis warnings rose about 30% and code complexity about 40%, and both stayed elevated. The speed was temporary. The complexity wasn’t.
The benefit lands now, and the cost lands later, so the trade looks better than it is. You’re borrowing speed from a future version of yourself who has to work in the mess, in a hurry, with no idea what changed.
Before you hand the work off, ask what happens if it’s wrong: how fast will you find out, and what will it cost by then?
AI is valuable for many different types of work. But you have to know which work before you reach for it.
Point it at work you understand, where mistakes are cheap and visible, and it makes you faster at being right. Point it at work you don’t understand, where the thinking was the point and the errors hide, and it makes you faster at being wrong while feeling more sure.
Better prompts don’t decide which of those you get. Judgment does. The same judgment that tells you when a plan is thin, when an estimate is too optimistic, when an answer is too clean to trust. AI won’t build that for you. It can help you learn the territory faster, but only while you keep doing the part that builds the judgment in the first place.
Which parts of your work are you willing to stop understanding? Answer that on purpose, before the reflex answers it for you.