Software is becoming easier to write than to understand
AI coding tools have dramatically reduced the cost of producing software. But software engineering has never been just about typing code. The real bottlenecks have always been understanding, ownership, design, and judgement.
Now that prompt engineering has given way to harness engineering, software engineering has become less about writing code and more about directing the systems that write it. The challenge is less around writing the perfect prompt, and more around orchestrating agents, defining constraints, and managing context. We don't need to write code anymore and can orchestrate agents to do it for us.
The productivity gains are undeniable but there is a tension around how they are being interpreted. AI is being used to justify job losses by organisations that regard engineering as a cost centre. We risk losing hard-earned discipline around how code is produced and deployed.
The accountability gap
This also changes who can build software. This age of the "citizen developer" who can build software without engineers may finally be upon us. Democratizing software creation is largely a good thing. Product managers building prototypes and designers validating ideas earlier can improve delivery. The challenge begins when prototypes quietly become production systems without clear ownership.
Code is being generated at a greater rate than our capacity to understand it. We're about to find out what happens when we push a lot of code into production systems that nobody has reasoned about.
This even has a name now: comprehension debt. Code accumulates faster than understanding. This is the hidden cost of AI generated code. It may look immaculate and pass the tests, but developers miss out on the learning experience of building and debugging code themselves.
What should give pause here is that working code is no longer evidence of effort or careful design thinking. All it really means is that somebody has access to a healthy token budget.
This is an accountability issue that boils down to the "who fixes this when it breaks?" problem. You might be able to rapidly generate something that resembles working software, but who is accountable for keeping it running? How can you constrain development to ensure ownership and manage risk?
The risk of over-dependency
The hypothesis is that LLMs will eventually be able to handle as much mess as we can throw at them. This is quite the assumption as there are still cases where humans must step in where coding agents fail. This speaks to Bainbridge's "Ironies of automation". He observed that automating routine work tended to leave humans responsible for increasingly rare and complex failures, i.e. the areas where context and expertise really matter.
Good design is a hedge against over-dependency on AI. Advanced coding tasks rely on inference capability that is only available in frontier models. Even if tokens settle into becoming a low-cost commodity, it will still be difficult to plan and predict costs. Human cost is ultimately restrained by headcount, while token cost is more unpredictable. A process can take as long as it wants and burns as many tokens as it needs to.
AI is becoming another abstraction later that is analogous to the introduction of 3GLs such as Java or C. These have remained popular because they can express computational logic in a form that humans can understand. Most engineers have no idea about the underlying machine code or assembly languages and probably never need to. We're placing quite a big bet on agentic tools being able to provide a similarly expressive abstraction.
There is a danger of speculating what LLMs might be able to achieve in the future. The fallacy of ever-increasing potential is driving organisations to gut their productive capability in the belief that the machines are about to take over. Coding tools are not currently capable of software engineering without human oversight and perhaps never will be.
Even if LLMs can eventually solve every coding problem, design quality will still matter, especially for long-lived systems. Complexity increases cost for LLMs as much as it does for humans. The context window gets stuff with unnecessary structures, and the reasoning required to solve problems is elongated. Code quality may be a solved problem, but good design requires experience, nuance, and context that is hard to distil for LLMs.
What humans bring to the party
In some ways, the current furore around coding tools is based on a fundamental misunderstanding around how software is built. Code production has always been quick. It's the stuff that happens around code that takes time. The real bottlenecks are in the design thinking rather than typing.
The hard problems around software engineering don't magically go away because we can write code more easily. You still need experience, domain knowledge, and judgement built through hard-earned experience. AI will amplify this, but it won't replace it. If anything, engineering will become a more rarified discipline as coding tools do not allow less talented engineers to "vibe" their way to greater productivity.
Engineering skill is only one thing humans bring. Good design also needs collaboration and the diversity of thought that only comes from a group of people discussing a problem. Coding agents don't think. They can't bring critical thinking to a problem. Humans should be doing more than just checking the output of AI models. They should be orchestrating agents and driving system design.
Organisations risk hollowing out their engineering capability for short-term balance-sheet gains. There may be a case for rebalancing development teams, but replacing engineers with AI represents a false economy.
AI will almost certainly make every engineer dramatically more productive. But productivity has never been the limiting factor in software engineering. That's understanding, ownership, and judgement.
The AI wave will expose those companies who understand where engineering value comes from and those who view it as a cost centre. It's the organisations that regard AI as a lever for engineers rather than a replacement who will gain the most here.