Photo: Close-Up of Mechanical Metal Part · Bokeh by Tomas Heligr-Pyke
I’ve had a very fun, ongoing situation lately, and I’ve been debating whether I should even write about it. It involves someone who actually exists. They probably won’t read this. They probably would be offended if they did.
I’m leaving out their name and the details that would identify them. I’m not a knobhead. But the situation has got me thinking about vibecoding again, and about how much thinking we are willing to hand over to AI.
Someone I’ve worked with uses AI for day-to-day thinking and is proud of it. They also use the answers it gives them to question decisions I’ve made about the work.
Questioning me is fine. I expect that. The problem is that the understanding I would expect underneath those questions doesn’t seem to be there. We end up arguing over an answer from a model, when I need to know whether the person bringing it to me understands what they’re asking for.
I thought the fundamentals would be a fairly reasonable thing to expect from someone working in software.
How wrong was I.
I have worked in the coal mines by hand
I use AI constantly. I build a ridiculous amount of stuff with it. Anyone who reads this newsletter has probably worked that out by now.
I also know how the things I’m building work. I’ve built software by hand. I’ve planned systems, worked through the code and dealt with the infrastructure underneath it. I have worked in the coal mines by hand, and I still need that knowledge when I use AI.
Before I let it touch a codebase, I spend a lot of time planning what needs to happen. Afterwards, I still have to understand what it has produced. If something goes wrong, knowing the system gives me somewhere to start looking. I can investigate the problem and fix it instead of handing the whole thing back and hoping the next answer is better.
AI makes it faster to get the work done. It also gives me more work to check. That responsibility is still mine, however quickly the code appeared.
So when someone produces something they can’t properly explain, then expects me to be impressed because an AI built it for them, no way. I need to be able to trust the thing. I need to know that the person responsible can recognise when it is wrong.
And when they use another AI answer to argue with me about it, I need an explanation I can examine. Sending the problem through a model doesn’t automatically make the result correct.
The confidence is what gets me
I used to think some of the warnings about people offloading their thinking to AI were over the top. Surely people were still capable of looking at an answer and thinking about whether it made sense.
Meeting someone who seems comfortable giving that responsibility away has been sobering.
I can deal with someone not knowing something. There are plenty of things I don’t know. Learning the fundamentals takes time, and asking for help is part of that. I get frustrated when the missing understanding comes back at me as confidence, backed by a model answer that apparently settles the discussion.
It feels like I’m expected to argue with the AI through the person. They have an answer in front of them. I have the system in front of me, and I still have to deal with what happens if we make the wrong decision.
I don’t want to turn every disagreement into me declaring that I know better. I’ve already written about my ego, and I know I need to watch it. Being challenged is useful. If someone can explain what I’ve missed, I want to hear it.
But I need them to understand the explanation too. Otherwise we’re both looking at generated text, and somebody still has to work out whether it applies.
Give it another five years
The way my heart sinks over this probably tells you how much it bothers me. It makes me feel physically ill sometimes. I can see how easy it would be to keep accepting answers until you stop bothering to work things out yourself.
More sloppers. More AI jockeys accepting whatever the model says as gospel. People getting more confident in work they understand less and less.
I don’t know how common this will become. But give it another five years and I’m worried we’ll have a lot of people producing things they can’t explain, while being quite happy with the arrangement. It is starting to feel a bit like Idiocracy, except everyone has a chatbot to make them sound clever.
I’m starting to worry about an underclass of people who can get an answer to anything but have given up on understanding any of it. Seeing this happen in work has made that fear harder to dismiss. God help us all if we make it normal.
I have to keep applying the same standard to myself. I use these tools too much to assume I’m somehow immune. I still need to plan the work, understand the output and be able to deal with whatever goes wrong.
If you’re going to question how I’ve built something, explain what’s wrong with it. Give me something I can check against the actual code and the decisions we’ve made. I’m happy to find out I’ve missed something, but I need us to work through it. Another AI answer on its own doesn’t give me enough to trust the decision.