Clojurians Log v2

There are a few sweeping statements in there. I wonder about his argument that increasingly accurate statistical models make it more difficult to se that something is broken. I agree that agents do not program like humans do. But the argument used seems to imply that humans cannot write amazing code with hard to find flaws in it. I learned (the theory of) programming at university from books and professors. I learned doing it on the job, from colleagues, from open source, from communities. I make mistakes, and I am also pretty sure that the resources I used to learn contained mistakes and flawed thinking as well. Whole programming languages have been developed that made things more complex, more difficult. Major systems built by humans contain major flaws. Isn't the real issue that there is a difference between letting an LLM replace a developer, or having a developer use an LLM as a tool on his belt? In the first case, there is a reasonable guarantee that requirements are unclear, code becomes unreadable, impossible to understand, and with flaws. In other words, the LLM's ability defines the quality of the solution. In the second case, isn't the ability of the developer that decides if the requirements and code delivered are readable, correct etc? It will be as flawed as the skills of the developer permit. It also implies that if the dev wants to be in control, he still has to learn things, has to built things and fail at it, improving his skill and knowledge along that road. And he cannot allow the LLM to take control, it is a tool, not a software engineer. Or is this too simplistic of a view?

> I make mistakes, and I am also pretty sure... That is the difference. LLM, Agents, they are not "pretty sure" of anything - they mimic human speech, but have no "concept" of "making mistakes" or being "sure", "unsure", "correct". For an AI, "correct" is a mathematical model, not a set of rules you ended up developing over time, internalizing as your own, and applying everywhere even without knowing you're doing it. I always mention that I was working with a Babashka project, and found an issue. A LLM agent found the correction was to make an async code sync - I accepted the suggestion, it wrote it anyway, I committed (I was at the "vibe-code" part of the project) and moved on. Later, when rewriting the project (it became quite a mess) I re-visited that commit because the bug resurfaced - I found that the "make code sync" commit was changing the line (.write conn buffer) to (when conn (.write conn buffer)) - nothing to do with async or sync or anything, and the LLM had no idea

Turns out... the issue was on Babashka, not on my side. The LLM confidently said it was an issue that it had to fix, tried a weird approach to fix it, that probably caused some delay in the code that essentially "fixed" for that specific case the issue, and it was "good to go"; but when I rewrote the whole project (because it was a mess, and because it was slow to the point of being unusable) the issue appeared again, and because I solved the "slow code paths" this "simple fix" didn't work anymore.

Isn't the limitation our own knowledge? If the LLM offers a solution, it is our responsibility to assess it. We can still be wrong, but the responsibility has to lie with humans. I feel that is where all of the problems come from. We give that responsibility away and assume all is well. Which we would never do with code written by humans. By keeping that responsibility, the LLM becomes a tool, instead of a replacement.

I'm with @alexander.vanelsas here: LLMs are tools and it's up to developers to use them properly. When we had the big push for "visual coding", folks who just blindly let that write "all" their code got into a mess. To me, this is the same, except we're using prose to drive development, not graphics, and we have much more control over LLMs and they can fill in the holes (that visual coding could not always do). I'd argue some of the 5GL push also worked this way: it was mostly a set of tools that couldn't provide 100% coverage for all cases, but people treated that the same (and, frankly, some of the "formal methods" stuff). I think a lot of devs dealing with LLMs today haven't been in the industry long enough to have experienced those "silver bullets" and their downsides (or blind spots).

What people also seem to forget is just how much more complex software is today than it used to be. LLMs are so much more capable than any other "automation" we've had access to in the past. Do they make mistakes? Of course. But past tooling did too, in its own way. The fact that LLMs can produce solutions at all for complex problems with minimal guidance is mind-boggling. The fact that they get things wrong shouldn't surprise us since so many developers also get things wrong.

I do think Mauricio has a good point about the conversational capabilities of LLM's which can easily give a feeling of 'it knows what it is doing'. This can easily be disproven by pushing back on a proposed solution. The LLM will (too) often immediately fold. I've found for myself that I can utilise the LLM better if I force it to ask me questions, instead of me asking it questions. But if I use it for something I have no knowledge of, then all bets are off. If I cannot reason myself about a possible solution, then I cannot really take responsibility for it.

VS Code just introduced a "Rubber Duck" mode: you ask Copilot to make a plan (as normal), and you can now ask it to use another model family to review the plan and find gaps/improvements (in addition to the "normal" round of developer pushback which y'all do, right? Right?!?). You could do that manually before but it was a bit of a pain in the ass. Another couple of recent additions are the ability to provide "steering" guidance while a model is "thinking" (planning, implementing, whatever), and the ability to split off an LLM session to ask questions (or spawn a new agent to do work), while the original work session continues, but retaining all that context. I think we'll continue to see harness improvements like that. In other words, the focus is on how to make these tools easier to use and easier to control/guide.

It's the same with everything it generates in my opinion. If you use it to make images, you have to accept they'll all have this AI-quality of being just a bit stranger looking but good enough. If you want it as good as a real artist, you'll have to fiddle with it like that as long as it takes to hand draw or stage/photograph. The same goes for writing a book with AI for example.

> If you use LLM to help you generate code, ... You have to lower your expectations of the outcome. I guess I would ask why anyone had "high" expectations in the first place? I expect it to produce sub-par results, and I'm frequently delighted when it exceeds my (low) expectations! 😄

For example now I focus more on the interfaces. What is the public API of this namespace, do I like the interface. Ok, then the LLM can implement the internals and I'm not going to try and make that look super clean and concise. Or I might design the interface myself. Or I go even higher level, is the API for this service is designed how I want? Or I design it myself. Is the DB model what I want/I design it myself. Or even higher level, is what the end user interacts with designed well and is how I want it. In all cases, you have to give up something to go faster, but if it's something hiding behind another thing, and no one is going to look under the covers, it's all good 😝

That said, I still sometimes am not sure, as the article says, it happens sometimes that when I interact with the result, through the end user interface, or API, or namespace public surface, etc. When I really go through it extensively, it's still often a bit quirky/janky. And if I try to sharpen that to perfection, it takes a long time, even with the LLM. A lot of that time is me thinking about how it should work in all the edge cases and combinations with everything else. When I write all the code by hand, I think about that stuff as I implement. When I do it with LLM I think about it all in the "testing/trying it out myself" phase. It's hard to really know if it would all add up to the same amount of time or not.

It sounds like you're trying to have LLMs generate bigger pieces of code than is "safe"?

What is "bigger pieces than it's safe"? I ask because yesterday, Claude helped me debug a very hard to find bug (it was in Babashka, so quite experimental). It did reproduce the error, and suggested a 1-line fix... that was wrong 🤷 (to be fair - no really wrong, just that it used locking around a WAY BIGGER part of the code so thread-safe code that was running in parallel would simply be sequential)

(Which is essentially how I use LLMs 90% of the time, I ask for options, opinions, explanations, to help debug things, but not actually to write things)

That comment was aimed at @didibus not you @mauricio.szabo 🙂

I assume you refused its wrong suggestion @mauricio.szabo?

(and, to be clear, I was talking about how big a piece of code you have it generate, rather than having it modify existing code -- my experience is that they're generally better at generating fresh code, like a new function, rather than trying to fix existing code, especially code you wrote vs code it wrote)

Yes, I refused 😄. I usually don't use the LLM to write the code for me, I prefer to use it to explain something to me, suggest alternatives to implementations, query for some pattern/function/conversion that already exist, these things

The difficulty I've had is management expectations leading to increased work pressure. The LLM is a fantastic tool but makes a mess when I have 5 projects running at the same time. There's a point where I no longer understand the output. I have coworkers who are running LLMs around the clock so there's multiple code reviews waiting every morning. If I slow down and go back to using the tool as an assistant it will lower the number of “points” I deliver in the sprint. I realise how broken this is but I need the money.

As someone still trying to find a job, I also feel this in interviews. I know "coding tests" are bad ways to evaluate a candidate, but it's not like a have a choice - and unfortunately, in multiple cases I felt that the interview became a "pay to win" experience - if you have money to be able to access multiple models, have access to the best ones, the code interview goes well. With a single exception, every interview gave me less than a hour to code a full feature...

Ugh! Sorry. I mean, the interview process was already broken in our industry, and I hear from many people who are looking for jobs that AI has made it worse in almost every possible way 😞 I have friends who have had online video interviews with AI-powered avatars, and they all tell tales of AI misunderstanding them and also treating correct answers as wrong.

I had the AI interviewer experience. It's horrible, and the way it is trained to speak... for example, it asked me about Docker, then said "oh, that shows solid knowledge about it" and I'm like... "shut up, you're a machine, I don't need or want this". But on the outside, of course I was just quiet and waited for the next question

I found that grilling skills are invaluable for this reason. Even for “trivial” things, I use grilling. It forces the AI to ask me questions and that gives me more opportunity to grasp what is going on before the AI acts.

@dumrat Could you elaborate on what you mean by "grilling" in this context?

With all due respect for those in need it a job and undergoing an AI interview, but if that is practice for companies hiring perhaps it is better to stay away as far as possible, unless of course you are allowed to use an AI to do the interview for you.

@seancorfield My reply was to the early on discussion on how to use the tools properly. Google Matt Pocock’s skills repo. Basically it provides a way to iterate on the plan before you do the iteration at the implementation level. It's along the lines of that Rich Hickey quip: “The cheapest and easiest stage to debug is in the requirement gathering/design phase” (iirc). I found it invaluable.

@dumrat Ah, thanks for the clarification: I wasn't sure what you were responding to and wasn't sure what type of "grilling" you were referring to 🙂

@alexander.vanelsas, I would do that if I was in other times. But unfortunately, this is probably one of the worst times for me to have no job; it's also the first company that gave me no compensation for the sudden end of the contract, and I also spent quite a lot of money this year, so my savings are low. I kind of need something fast right now...

I fully understand and appreciate that Mauricio. It is just a lament against an industry that seems to be led by complete idiots sometimes

Yeah, it really feels that way. Seems that we kind of forget from time to time all good practices when a shiny new toy appears, and then we're left of with dozens of failed experiments that we still need to make it work somehow, otherwise the whole world explodes.