One aspect AI is weak in is controlling complexity. If you tell it to implement something it will go ahead and implement it, without considering how much complexity it adds to the system or weighing alternatives. An experienced engineer on the other hand may decide the feature is too minor relative to the complexity it adds, and may decide to not do the feature. Or he may make some clever compromises to get most of the functionality while keeping the codebase simple. AI is weak in this judgement, it doesn't spontaneously exercise architectural restraint. As a result the code may progressively become too complex even for AI manage, and it becomes whack-a-mole where you can't make a change without breaking something.
I feel this mostly is a side effect from lack of domain knowledge. Most of the time this has happened to me, it's because I myself did not cleanly know how a problem should be solved to begin with. If you have a clear picture of what you want, approximately what syntax goes where and why, thats really when LLMs shine in my experience.
Yeah but on the other hand, if you asked an LLM to implement a spec and it was like “I skipped this part because I didn’t like the complexity tradeoff” most people would be like “wtf why doesn’t Claude just listen to me”
I use openspec in addition with the /grill-me skill and it really helps clearing the path before starting to code. I think the goal of engineering, when using AI, is to maximize your value upfront instead of every five minutes.
If you tell it to implement something it will go ahead and implement it, without considering how much complexity it adds to the system or weighing alternatives.
Recently I asked Claude (Fable) to use multiple threads to speed up a computation that could take several seconds to run while the user was waiting. Instead, it found a way to start the computation earlier in the background while the user was doing other things, so that it would be finished by the time the user was ready.
I found it follows conventions and documentation well. So if you have a well designed core, it can easily add independent features without increasing overall complexity. Maybe it doesn't work in some very tangled domains like games, but some basic crud and saas stuff is pretty much a solved problem now with agents. They will trivially add features that humans would have pushed to a backlog forever as not worth the effort.
I have actually found decent success recently (especially with Fable) giving it an "added line budget". This does more than anything else to keep it on task. The trick is knowing an approximate line count beforehand, but sometimes the LLM will push back on what it should cut. But usually it just doesn't add all those "nice to have" redundancies and multipage comments and unit tests that check if a certain phrase is still in the error message.
Keeping a simple log of "accepted/rejected" avenues is basically all that's needed, maybe with a root style guide in the project. I've got several multi-day sessions in 5.6 Sol running without going off the rails in terms of complexity. After a while in this loop it actually starts to remind/berate itself to keep things straight-forward.
Just ask it to do it. You can ask it to do a line reduction cleanup pass before each commit. I don’t know why they default to adding complexity because they are capable of doing it better just by asking.
With previous engineering trends like blockchains and microservices, you could choose not to jump on the bandwagon. However the coding agents trend is different and is changing the very fabric (sry for Claudeism) of software engineering, for better or worse. I do know we will never go back to mainly programming through code again, that’s for sure.
Knowledge work has changed...But it doesn't solve company internal governance nor politics.
When it takes anywhere between 2 weeks and 3months to do anything ( including approvals for non prod access) in most organizations...Code has never been the problem.
I'm starting to think there is no AI bandwagon per-se - instead the bandwagon most people associate with AI-assisted development is more a bandwagon of sloppy code and low standards - which irresponsible use of AI enables but isn't a prerequisite for (outsourcing sweatshops have been practicing it long before the dawn of LLMs).
I think it's not an option. The benefits are just too large for me.
>I do know we will never go back to mainly programming through code again, that’s for sure.
I think, in some niches, i.e. where there's something not well represented in the training set, it still makes sense to write code by hand. But I am not sure that it will continue.
There are still companies who refuse to believe this and still put Senior+ devs through hell during an interview process with junior level algorithm memorization.
In all aspects there will be dinosaurs and deniers and there will be embracers.
I don’t particularly mind algorithmic interview questions (they’re at least self-contained, somewhat objective, and something you can prepare for), but most software engineering has always been done by people with no knowledge of algorithms or compsci fundamentals. What’s being lost is coding fluency, not compsci knowledge.
Better example, leetcode interviews became so popular because people were memorizing FizzBuzz and they needed an alternative (but said interviewers didn't fully understand the purpose of FizzBuzz and though "harder means better, right?").
Coding with AI has now introduced feature dopamine. At times this results in the system being prone to more failures because AI may have missed edge cases.
Also i am experiencing a decline in job satisfaction and i'm more prone to procrastination because I know the agents will do the work 10x faster than me. I am personally worried about this shift and I fear becoming less knowledgeable over time or not feeling the need to keeping up with new tech stack as agents do the work.
And you immediately give up your IP for someone else to use. The 4th option, if you have something in your mind worth building, is to just build the thing, without an LLM.
Sure if you use remote AI services, but any companies working on niche markets where they want to protect their IP, or they simply work with sensitive stuff, will rely on local AI instead.
There’s no company that has “no sensitive stuff”, from HR to financials to customer data to board presentations to engineering IP and they all, without exception , entrust their data to cloud services for at least a decade now. Even governments do that, although they sometimes use special regions.
About "implementing by words bit": I don't believe English is a great language to program.
It's not type-safe, not object oriented, not functional. Has poor tools to highlight syntax or navigate through "wordbase", doesn't fail fast. It has no tests and has too large room for machine or other humans to interpret it.
Very often it's easier for me to express my thoughts in Java, which is ironically known to be a "wordy" language. But it's nowhere close to wordiness of English.
You have to be able to write the software yourself in order to judge the results and get good software out. Otherwise the system claims the goals are met, the tests pass, and the human driving the system puts up a new PR. If they don’t know any better it must seem like the AI system is better than them and knows what it’s doing.
All the loops and agents don’t protect you from generating garbage.
Which sucks because then how are you supposed to improve your skills when you’re just getting the answers all day… answers you can’t verify?
People are more confident than they ought to be. Always have been. But AI throws gas on that fire.
And yes, there is also another sane and rewarding option: write everything just by youserlf without any assistance. Let's not forget about that one, shall we?
I'm not so sure. I think it's fair to say that only people who can output good art / code are qualified to evaluate the output of ai.
It's like how product people / C levels have absolutely no understanding of what makes for good code or a good engineering shop (aside from perceived costs.)
My personal experience is the larger the task you ask it to do, the less attention it pays to the details - for a very large task it seems more prone to missing test coverage, writing duplicate code, not refactoring where it should etc. So I try to split into smaller tasks where possible (also makes it easier to review).
I have agents that have been running nonstop for several days working through tasks. At some point I have to go to sleep, and wake up to more progress.
Even with preplanning and post hoc analysis thrown in, I am seeing way more than 2x return on my investment. Where are your numbers coming from?
I wonder how that progress is being measured. Lines of code or counts of PRs? Sure... but I thought the matter of measuring productivity by lines of code was already well-understood as being misguided.
I'm having trouble reconciling all that supposed productivity with the real world where software isn't getting better, delivered faster, or becoming cheaper - unlike virtually all breakthroughs in industrialization (printing press, weaving loom, etc) which led to a quick increase in at least one of such factors.
I'm not denying that AI helps with and excels at parts of the software development lifecycle, but from my experience those parts overall contribute to a small increase in output or merely shift the work elsewhere (where it may just not be part of whatever measurement is being used).
> You can’t just give a 3000-word, 4-page detailed dense spec and expect it to follow everything, and the larger the codebase, the less it can pack everything in, nor are the vast documents you can feed it worthwhile.
This point seems lost on a lot of principals. I’ve had very little success with these grandiose designs and change requests from RFCs/specs. The context windows just can’t keep it all together and very quickly the approach unravels.
I posit that the further ICs were from writing code at this point in their career, the more they suffer from AI psychosis. It’s the same ivory tower they were already on, just a different order they’re giving.
It really depends on how you use it. I’ve switched up how I interact with LLMs repeatedly over the years, as the technology has developed.
Now, it’s at the point where it’s like running a development team of very eager amnesiacs. I’ve found the trick is exhaustive documentation by a lead agent, and then having a fresh agent work as a coordinator across as many subtasks as the project sensibly allows. This way the individual components stay on spec, as does the ultimate integration. It’s only really this year that this workflow has started to actually function, and it still needs human supervision - but less and less over time.
I give it two years, tops, and everyone everywhere is building bespoke software because it’s trivially easy.
Recently I asked Claude (Fable) to use multiple threads to speed up a computation that could take several seconds to run while the user was waiting. Instead, it found a way to start the computation earlier in the background while the user was doing other things, so that it would be finished by the time the user was ready.
More features is always good, right? Let's build dropbox+gmail+netlify+spotify+youtube+hackernews+... \s
stragnely i dont miss that feeling and kind of dread going back to it
I think it's not an option. The benefits are just too large for me.
>I do know we will never go back to mainly programming through code again, that’s for sure.
I think, in some niches, i.e. where there's something not well represented in the training set, it still makes sense to write code by hand. But I am not sure that it will continue.
Say who?
In all aspects there will be dinosaurs and deniers and there will be embracers.
And one does not memorize algorithms.
So I don’t think AI will be much different.
almost every inference operator either has ZDR or an opt out from training
unless you think they're just lying and training on business users data
It's not type-safe, not object oriented, not functional. Has poor tools to highlight syntax or navigate through "wordbase", doesn't fail fast. It has no tests and has too large room for machine or other humans to interpret it.
Very often it's easier for me to express my thoughts in Java, which is ironically known to be a "wordy" language. But it's nowhere close to wordiness of English.
All the loops and agents don’t protect you from generating garbage.
Which sucks because then how are you supposed to improve your skills when you’re just getting the answers all day… answers you can’t verify?
People are more confident than they ought to be. Always have been. But AI throws gas on that fire.
Questionable output, generally shunned by artisans.
But possibly good enough for some.
It's like how product people / C levels have absolutely no understanding of what makes for good code or a good engineering shop (aside from perceived costs.)
Even with preplanning and post hoc analysis thrown in, I am seeing way more than 2x return on my investment. Where are your numbers coming from?
I wonder how that progress is being measured. Lines of code or counts of PRs? Sure... but I thought the matter of measuring productivity by lines of code was already well-understood as being misguided.
I'm having trouble reconciling all that supposed productivity with the real world where software isn't getting better, delivered faster, or becoming cheaper - unlike virtually all breakthroughs in industrialization (printing press, weaving loom, etc) which led to a quick increase in at least one of such factors.
I'm not denying that AI helps with and excels at parts of the software development lifecycle, but from my experience those parts overall contribute to a small increase in output or merely shift the work elsewhere (where it may just not be part of whatever measurement is being used).
This point seems lost on a lot of principals. I’ve had very little success with these grandiose designs and change requests from RFCs/specs. The context windows just can’t keep it all together and very quickly the approach unravels.
I posit that the further ICs were from writing code at this point in their career, the more they suffer from AI psychosis. It’s the same ivory tower they were already on, just a different order they’re giving.
Now, it’s at the point where it’s like running a development team of very eager amnesiacs. I’ve found the trick is exhaustive documentation by a lead agent, and then having a fresh agent work as a coordinator across as many subtasks as the project sensibly allows. This way the individual components stay on spec, as does the ultimate integration. It’s only really this year that this workflow has started to actually function, and it still needs human supervision - but less and less over time.
I give it two years, tops, and everyone everywhere is building bespoke software because it’s trivially easy.