AI Coding, Changing Development Methods Three Times

In late November 2024, I first opened an AI conversation window. I didn't have any special expectations; I just wanted to try it out.

Having developed software since 1999, I was familiar with the landscape of new tools emerging. Languages changed, frameworks changed, and functions that I used to implement manually could now be solved with a single library. I thought this was just another one of those changes.

However, this time, my way of working changed consecutively. Just when I thought I had gotten used to one thing, a tool would come along and take over that task.

At first, I copied code. Then, I handed over files to AI. Now, I spend my time creating criteria and environments for AI to continue working.

In the process, there were moments when I felt frustrated with studying, and I wondered if developers were really becoming unnecessary. On the other hand, there were days when I felt like I was grasping the most difficult parts of development without even writing much code.

I'm going through a strange period.

The Tool Took What I Learned

At first, I input "write this code" into the conversation window. When I got an answer, I copied it, pasted it into my editor, ran it, and if there was an error, I took it back to the conversation window. I spent a lot of time going back and forth between the two windows with code and error messages.

Then, I heard that I should use prompts well. In reality, it made a difference. If I vaguely described what I wanted to make, the result was also vague. If I explained the existing code and conditions, and even specified the desired output format, it was much more useful.

So, I studied. I put effort into assigning roles, providing examples, and dividing work orders. I thought that creating good questions was also a skill.

However, when I started using a coding agent that could read and modify files directly, a significant part of my work disappeared. I no longer needed to copy errors. AI read the execution results and made the next modifications. Instead of writing detailed instructions, I could have AI prepare requirements and even create a work plan.

I felt a bit frustrated.

The method I had just mastered was now being taken over by a tool. It felt like someone next to me was skipping straight to the next step without needing to learn what I had just learned.

However, as time passed, I realized that what had disappeared was just the way I used to work. The sense I developed during that process didn't disappear.

The experience of understanding failures, the sense of refining questions, and the experience of dealing with errors were all still useful. They were applied to creating validation conditions and dividing tasks.

The lifespan of tool usage was short, but the experience of understanding failures lasted longer than I thought.

Between Quick Creation and Completion

Developing with AI is really fast at first. The screen appears, buttons work, and data flows in. What used to take a long time now takes shape before my eyes. It's hard not to think about how much developers will be needed in the future when I see this.

I thought the same way.

However, as I delved into complex tasks, a different scene emerged. When I fixed one part, another part broke, and when I modified that, the original problem came back. Each modification seemed valid when explained, but the problem was that these modifications didn't fit together within the same system.

This was especially true when refactoring a desktop app that involved multiple technologies. The screen's memory state, the intermediate process's values, and the internal core's judgments were all slightly off. I had to clearly define what was the original and what was the copy, but I hadn't established a sufficient basis for this.

As I followed AI's modification suggestions, I lost sight of the overall picture. Honestly, I couldn't even explain the structure as a single diagram. So, I couldn't provide AI with consistent criteria.

What I felt then was that this wasn't a problem that could be solved by simply saying "AI is still dumb." I had been expecting AI to accurately fill in the boundaries and premises that I hadn't explained.

The Blank Space in "Make It Like Our Company"

This isn't just a problem with code structure.

If you ask to create a shopping mall, you can think of functions like product lists, shopping carts, payment, and delivery. However, if you say "make a shopping mall like our company uses," the story changes.

For example, let's say a certain company requires human confirmation before shipping certain products. On the surface, it seems like an unnecessary approval process. Automating it would make it faster. However, if there was a big loss due to incorrect shipping in the past, and the supplier's data is still unreliable, there's a reason for this process.

The same goes for payment failures. If you receive a failure message, can you just retry the payment? What if the actual approval was successful, but the response was delayed? You need to distinguish between displaying a failure on the screen and the actual transaction failing.

These rules aren't conveyed just by function names. They might be in policy documents, scattered in code, or known only to the person in charge. Old documents and current operating methods might differ.

AI can also read materials and understand this context. So, I don't want to say "AI doesn't know the domain at all." However, I shouldn't expect it to know all the circumstances from the start. When materials conflict, AI needs to decide what to use as the current standard.

Developers ask stakeholders, check exceptions, and coordinate different requirements. After understanding why a rule exists, they express it as code and tests.

This was also part of development in the past. Now that code is created quickly, the work that was previously done before coding is more visible.

Even Making Documents Didn't Work

At first, I thought that if I had documents, it would be solved. I wrote down the design and had AI read it, thinking it would follow the same criteria for the next task.

However, just having documents wasn't enough. It was unclear which document was the latest, the reasons for decisions were missing, and even when the code changed, the documents remained the same. The criteria I wrote down were just reference materials for the actual work.

So, I started changing how I operated documents rather than how I wrote them. I left behind why I chose a certain structure, what alternatives I considered and rejected, and examples of problems that occurred when rules were broken. When AI repeated the same mistake, I didn't just fix it with words each time; I turned it into guidelines and checks that could be referenced for future tasks.

In development, we call this common standard SSOT. To me, it meant "what do we look at to decide when we get different answers?" It's not about putting everything into one document. At the very least, I needed to know what criteria were valid and what changed when something else changed.

Next, I worked on the environment where AI operated. I decided when to read necessary materials, where to leave progress, what to pass to consider it complete, and where to start again in case of failure.

This is called harness engineering. In my experience, having a device that checks actual operation before saying "it's complete" was more crucial than creating a grand agent organization chart.

I also tried dividing roles and using multiple agents together. However, just giving them names didn't make them cooperate. Each brought plausible results, but they didn't match. In the end, the person connecting them had to redo the work. I had to decide what information to share and what standards to use.

And attaching many devices wasn't the answer either. Running heavy checks for every small modification and dividing simple tasks into many steps actually made it slower. I had to review whether the procedures that were necessary in the past were still needed now.

The condition to prevent duplicate payments must be kept. However, it's possible to change whether to run all checks every time to verify this. Distinguishing between the rules to keep and the methods to keep those rules became my task as well.

That Doesn't Mean a Developer's Position is Guaranteed

If I say all this, you might expect me to conclude that "developers won't become unnecessary after all."

I don't want to reassure you with that statement.

As implementation time decreases, it becomes possible to do the same work with fewer people. The fact that judgment is needed doesn't guarantee the current hiring scale and roles. I don't think that just because I've developed software for a long time, I'm safe.

However, one change that has become clear in my experience is that while the amount of code I write directly decreases, the scope of what I need to understand and check has actually expanded.

When AI makes something faster by adding a cache, I need to check when the value expires. I need to confirm that another user's information won't get mixed in and what happens when the original data changes. The questions don't disappear just because the result looks clean; they move on to operation.

So, the responsibility I'm talking about isn't just pressing the final approval button. It's about explaining why a result can be accepted, stopping when it's dangerous, and being able to fix it when it's wrong. Organizations need to have the corresponding authority and support. It's not an ability that appears just because an individual's name is listed.

As AI takes on more work, I also need to ask myself how much I can understand and handle.

However, there's still a problem that's difficult to answer. A significant part of my judgment comes from directly implementing, failing, fixing, and failing again. If AI takes over much of that implementation, where will future developers develop that sense?

Saying "just do good design" is too easy. You can't expect someone who hasn't experienced failures yet to design well from the start.

So