​What this looks like in practice, not in a psychology lab but in the specific, concrete behaviors that distinguish good engineering from bad engineering, and good knowledge work from bad, is where the consequences start to impact organizations.

​

The ability to think in structure disappears

​Good engineering starts with breaking a hard problem into pieces small enough to reason about each one completely. Experienced engineering leaders consistently describe this as a skill that takes fifteen to twenty years to truly master. It's the skill that determines whether a system can be maintained, extended, and understood by others.

​

When the default move becomes "give the whole problem to AI and see what comes back," engineers stop practicing this skill. Instead, they start thinking in prompt-sized chunks rather than architecture-sized structures. The AI can produce the code, but when larger and larger chunks of logic are wholesale automated, humans stop developing judgment about how to divide and subdivide a complex problem. Which pieces to separate, which abstractions to create, and which clear and critical boundaries to draw between components. It is a slippery slope that starts with small pieces and then gets steeper as inattentive engineers fail to pay attention to what’s happening and get sucked into the loop.

​

The resulting change in practice is the difference between a surgeon who understands anatomy and chooses where to make the incision, and someone who asks a machine to "fix the patient" and then evaluates whether the result looks okay from the outside.

​

Engineers lose touch with what they're building and who they're building it for

​At first, there's a seemingly more subtle loss that goes beyond technical skill. The best engineers don't just know how to implement systems. They develop a deep understanding of what those systems are for. They internalize the user's workflow, the customer's environment, and the value proposition that makes a feature worth building. This domain understanding is what allows a great engineer to make the right judgment calls when the specification is ambiguous, the requirements shift, or an edge case the spec didn't cover arrives in production. The skill of sound judgement in the full context of the solution’s operating environment only forms through sustained engagement with the problem space.

​

When the prompting loop redirects that engagement toward the AI interface, the domain understanding stops forming by default. Engineers can still ship features, but they're increasingly disconnected from the world those features are supposed to serve.

​

The activities that build understanding lose their force

​In a healthy engineering organization, understanding doesn't form in a single activity; it builds across many: code reviews, where a reviewer questions a colleague's design choices. Architecture discussions where engineers defend their structural decisions. Pair programming sessions where two minds work through a problem together. Design reviews where someone asks, "Why this approach and not that one?" Teaching a new team member how the system works, debugging a production incident, and tracing the failure back through the logic to the root cause. Each of these activities forces active cognitive engagement with the system, and it's that engagement, not the activity itself, that produces understanding.

​

When AI generates any code, a critical distinction emerges. Catching defects, security flaws, and logical errors is a quality assurance function, and AI-powered review tools are rapidly becoming capable of performing it well. But quality assurance was never the primary way engineers built their understanding of a system. Understanding is formed through the process of engaging with design decisions. It involves questioning why something was structured a certain way, proposing alternatives, and defending a choice under scrutiny. The social dynamics of human collaboration aren't mere overhead to be automated away. The colleague who will read your review, the relationship that requires careful justification, and the productive friction of disagreement. They are the mechanism through which shared, system-level understanding forms.

​

The prompting loop quiets all of these activities simultaneously. Not because they're forbidden, but because the loop is faster and the organization rewards speed. Why schedule a design review when the AI has already produced working code? Why pair-program when one person with an AI tool can cover twice the ground? Why teach the new engineer how the authentication system works when they can prompt their way to a functional modification? Each shortcut by itself is a decision that appears rational. Collectively, however, they eliminate the cognitive engagement that was building the team's understanding of its own systems.

​

Imagine a publishing house where editors start accepting manuscripts from an anonymous machine. The manuscripts are competent, grammatically correct, structurally sound, and publishable. But no editor is sitting across from an author, challenging their assumptions, pushing them to rethink a weak chapter. The dialogue that turns a competent first draft into an exceptional second one has vanished. The quality ceiling drops and nobody notices, because the baseline competence of the output is higher than it used to be. The average manuscript is decent, so nobody sees that the exceptional ones have stopped appearing.​

 

The coupling between building and understanding breaks

​This connects directly to the flow dynamic described above. In traditional well-structured development, writing code and understanding the problem are the same activity. You cannot design a system that handles customer orders well without understanding how those customers actually behave. You cannot build an authentication system without understanding the security threat model. If done correctly, the code is a byproduct of comprehension.

​

When AI writes the code, that coupling breaks. You can ship an authentication system without understanding the threat model, because the AI produced output that pattern-matched against its training data. The feature works, but understanding was never formed in the engineer. The next time something needs to change in that system, nobody on the team will have the mental model to change it safely and effectively. Instead, everybody just keeps pulling the lever, and the loop continues to churn.

​

This is when debugging stops being a reasoning process and instead becomes forensic archaeology, reading code you didn't write, and reconstructing the intent behind design choices you didn't make. Working in a system whose architecture you didn't choose. Fixing an auto-generated system that is “almost there” is dramatically slower, more error-prone, and produces zero transferable understanding. You are not learning how the system works; instead, you are excavating what an unpredictable process happened to produce.

​

Jeremy Howard encountered exactly this situation while building a product his company depends on. After spending weeks using AI tools to fix a complex piece of infrastructure software, over five thousand lines of interconnected code, he arrived at a working solution. But it was a working solution nobody understood. He describes the moment of recognition. "Here's a piece of code that no one understands.” Then he asked himself rhetorically, “Am I going to bet my company's product on it? The answer is I don't know, because no one's been in this situation." The code works, it ships, but the person responsible for it cannot explain why it works, which means they cannot anticipate how it will fail when the requirements change, the system load increases, or the environment shifts.

​

Architectural coherence degrades silently

​When developers stop decomposing problems from first principles and instead assemble AI-generated components, the system-level architecture gradually loses coherence. Each piece may work in isolation, but the overall structure reflects no unifying design vision. It reflects the statistical average of the AI's training data, which is to say, the average of everything that's already been built.

​

Jeremy Howard, drawing on a case he examined in the same MLST interview, illustrates this precisely. When Anthropic had Claude Code build a C compiler, the resulting source code contained idiosyncratic design choices copied from LLVM (a human-developed compiler), choices the original LLVM author now considers mistakes. The AI didn't make architectural choices; it made interpolations. The result was functional but unprincipled. It is long understood that unprincipled architectures collapse under their own complexity when conditions change.

​

This cumulative erosion was predicted decades before AI coding tools existed.

​

Fred Brooks, one of the most influential thinkers in the history of software engineering and architect of IBM's System/360, made the case in 1986 in his famous essay "No Silver Bullet." He posits that no single technology advance would yield more than a thirty percent improvement in software development, because the essential difficulty of software has never been typing code. It's managing conceptual complexity. Typing was never the bottleneck.

​

Understanding was. Understanding is exactly what the prompting loop displaces.