Engineering · 5 min read
AI can write code. Engineering still needs judgment.
A practical way to use generative tools without confusing convincing output with a reliable software system.
In this article
The difficult part is deciding what should happen
Consider a small inventory application. Generating a screen that subtracts a quantity from a stock total is straightforward. Deciding what happens when two employees reserve the final item, a payment times out, or a warehouse sends a correction is harder. Those decisions cross product policy, data consistency, user experience and operations. A plausible implementation can silently choose the wrong policy. It may compile and pass its own tests while still violating the business requirement.
Generative tools can help an engineer explore those choices. They do not remove the need for someone to identify the choices, resolve ambiguity with the people affected, and check the result against reality. That is the useful distinction between producing code and engineering a system. It is also why the relevant question is whether assistance improves the whole delivery process, rather than how many lines appear in an editor.
Read productivity studies in their actual context
Evidence does not support one universal productivity number. A June 2025 paper published by Microsoft Research combined three randomized field experiments involving 4,867 developers and reported an estimated 26.08% increase in completed tasks, with a standard error of 10.3%, when developers received access to a coding assistant. That measures a particular intervention and outcome across those workplaces; completed tasks are not a comprehensive measure of maintainability or long-term product value.
METR's early-2025 experiment studied 16 experienced open-source developers working on 246 tasks in repositories they knew well. Allowing the tools increased completion time by 19% in that setting. Its February 2026 update discusses selection and measurement problems in a follow-up study. These findings describe different tools, people and tasks. Neither establishes that every team becomes faster, slower, or replaceable. Treat impressive benchmark scores and personal feelings of speed as hypotheses to investigate in your own work.
Choose work that is easy to check
Useful starting points include drafting a parser for a documented format, translating a small function between languages, generating a migration checklist, explaining an unfamiliar module, and proposing edge cases for an existing feature. Give the assistant the relevant interfaces and constraints. Ask it to state assumptions before expanding scope. A narrow task with observable behavior gives the reviewer something concrete to accept or reject.
For example, asking for a CSV importer that preserves quoted newlines is more useful than asking for an entire customer platform. Supply representative files, specify the encoding policy, and require useful errors with row context. Then verify against examples the generator did not write. The same principle applies to documentation: an assistant can outline an onboarding guide, but someone must run the instructions on a clean machine before calling the guide correct.
Keep an independent definition of correct
A generated implementation and generated tests can share the same misunderstanding. If both assume a reservation is confirmed before payment, a green test suite tells you very little about the policy you actually needed. Write important acceptance conditions independently, preferably in the language of the user or operator. Review failure paths as carefully as the successful demonstration.
A useful review follows the data. Where does input enter? Which actor is allowed to change it? What persists if execution stops halfway through? Can an operation be retried safely? What will an operator see when it fails? For the inventory example, exercise concurrent reservations, duplicate callbacks, expired holds and service restarts. The engineer's contribution is choosing and interpreting these checks. More generated test cases do not compensate for missing the central invariant.
Measure assistance across the whole task
A sensible local evaluation includes understanding the request, preparing context, waiting for output, reviewing changes, testing, fixing defects and handling regressions. Counting typing time alone rewards work that creates expensive review. Compare tasks of roughly similar difficulty and note the developer's familiarity with the repository. Keep the process lightweight enough that people will actually record it.
Track a few outcomes over several weeks: time to an accepted change, review rounds, escaped defects, and the effort required to modify the result later. Ask developers where the tool reduced cognitive load even when elapsed time did not improve. Do not turn a small internal sample into a scientific claim. Use it to decide which workflows deserve more investment and which need tighter limits. A tool can be valuable for exploration while being a poor default for a sensitive maintenance task.
Responsibility belongs to the team shipping the work
Before introducing an agent, decide which repositories, credentials and environments it can access. Use the same boundaries you would want around any automated contributor. Sensitive production actions should require an appropriate deployment process. Review dependency additions, data handling and generated database changes. Check the tool's actual data settings rather than assuming every product has the same confidentiality policy.
The goal is competent people working with useful tools. Engineers still need to explain the architecture, diagnose failures, understand the dependencies and maintain the result after the conversation ends. If nobody can explain a generated subsystem, reduce its size or investigate it before shipping. AI assistance can make experimentation more accessible and repetitive work less tedious. It becomes reliable engineering when the team retains a clear problem definition, independent verification and ownership of the software's consequences.
Sources & further reading
Primary material behind the discussion and useful places to go deeper. The recommendations and worked examples in this article are our synthesis.
Working through a software or game-development problem of your own?
Talk to Winnitude! ↗Explore our tools ↗