Clojurians Log v2
ai-skeptics
the token squeeze has already started with moving to token based billing instead of subsidized subscription pricing... I don't think the "use AI agent loops for everything" will survive when you have to pay the actual cost of the compute. there's 800 billion in investments now in the air... that bill is coming due and the providers also need to make a profit from this.
Totally agree about the Readme. Same goes for all communication about a project that are meant for humans to consume. Look how #C06MAR553 lately has a lot of neigh unreadable walls of text. 😢 (Hence https://clojurians.slack.com/archives/C0353589RFC/p1778835392889539)
@pez exactly you can see literally on the last one posted today mdash everywhere lots of not A but B / A not B etc. llm marketing writing style. If you're going to generate everything including the slack communication then I have to assume your target audience is other agents. You're not reading the consumer is not reading happy days. The irony is it says the engineering is real. I'm sure it is
Devs at my workplace will work around the clock to get the most out of the usage windows. Then leave you with multiple slop PRs to review during actual work hours. Difficult to keep up with if you have any sort of life outside of work. I didn't see this vision of “productivity” coming when started out.
AI generated project readme:s make me feel stupid. At least with some experience I am now quicker to realize it wasn’t written for me and can stop trying to understand.
The way I've seen it expressed is that READMEs are the only part of your repo where the direct target audience is unambiguously another human being... and so the act of outsourcing that one avenue of human<->human connection to an automated tool belies a fundamental lack of care towards the reader (and by extension the rest of the codebase)
oh and it's almost irrelevant what the author's intent is since the damage is already done at that point, which is sometimes a tragedy - I see non-native speakers try and 'punch up' their English prose by piping it through chatGPT and the result is always a bland sludge that immediately triggers everyone else's slop-detectors (with all kinds of subtle tells that they're least likely to pick up on 🫤 )
I’ve let out some thoughts about LLM-facilitated hobby projects: https://blog.danieljanus.pl/now-what/
> Is it because someone is actually going to use your creation, in anger? Have you actually solved someone’s problem? Is that someone you, perhaps? To be fair, I think a healthy part of hacker culture is to just do things because they can be done.
Thank you, that's a really great perspective. It also rhymes with @andersmurphy's observation, some of the constraints in (non-AI) software engineering are useful, if you take them away you lose a forcing function which helps to drive good engineering.
> just do things because they can be done I think the article makes room for that? It's not saying you shouldn't do it, it's saying to be honest and introspective about what drives you and what your expectations are afterwards.
Agree with @henrik! But sometime I do build something because I think it solves someone’s problem, and then it is a lost opportunity to not let the potential users know this. 😃
Perhaps, fair enough. I sort of bristled at the interrogation that seems to demand some kind of ROI, even if it’s knowledge. I often think of the concept of karma yoga, which I’ve taken to interpret as “there’s no reward other than what you got by doing the activity itself”.
@henrik I think that should be rephrased as "because you want to do them". Dumb projects are the best. Building things because you can and there's no resistance now, I'm not sure is that useful.
Yeah, but now you're no longer doing the activity, claude is.
so the journey isn't the reward as you are teleporting to the destination
I think it still stands. If you’ve diminished the journey, that’s what you got.
I think of it as a different kind of journey.
> To be fair, I think a healthy part of hacker culture is to just do things because they can be done. Yep, and I think that ‘it can be done’ used to imply things that it doesn’t any more. Previously, if you did something, you would flex your doing-something muscles, which then made you somewhat different; with LLMs you just outsource that.
I don’t know, that’s very categorical: “if you do this, then you’re X, but if you don’t, you’re Y”. There are different ways of using LLMs, and not all of them need to leave you as a passive bystander. I’ll concede that it’s easy to fake “having done something” though, and it’s hard to tell one from another.
Agreed, it’s very much a spectrum
If your goal is a fast proof of concept or something throw away, then I can see the potential value of LLMs (depending on real cost). I've use them to generate throw away java code that I'd never want to write, but I can spot check and benchmark my clojure code against (to get a rough idea of the performance delta). Or playfully explore what the JIT does (by getting the LLM to inline everything etc etc). They can be ok adversarial rubber ducks etc.
But, the goal there isn't an artifact the LLM generates
It's a disposable experiment to test a hypothesis
(this makes me so sad, I use A -> B all the time, and now I need to change my style) 😭
Anecdotal. Yeah a friend who works at a company that is tokenmaxxing AI (where they are forced to use AI for almost everything) said the best thing he did for his personal projects was switch back to writing communication by hand even though english is his second language in his own words "people engage more with my broken english than perfect LLM prose". I'm dyslexic my writing can be pretty rough and I'm prone to adding extra words, missing words, bad punctuation and spelling mistakes (the kind that spellcheck miss). LLMs in theory are a tool I'd greatly benefit from. But, I write to understand and blog posts are a way of cementing that understanding. The upshot is my style is definitely my own.
I do wonder how these dynamics (covered in the article) will play out in writing, software, music etc. https://www.hamiltonnolan.com/p/go-ahead-and-use-ai-it-will-only
Ditto. Though I also wonder if any clear outcome will ever emerge? Like, 10 years in the future, all the AI-slopped house of cards will come crumbling down, and the AI-atrophied brains of those who built them won't know what to do next, and all the AI nay sayers will have their day. Or writing code will become a hobby akin to knitting or woodworking, while the real world 10Xs their productivity and output by pivoting from hand-slinging code to developing ever more well-designed agent orchestration frameworks leaving the nay sayers to yell at clouds. Then SGI. But seems just as likely to just perpetually evolve into what is essentially an argument about how much AI is the right amount of AI, with (one hopes) occasional productive discoveries littering the path.
Hihi, the (Slack) onboarding message wants to give me an AI summary — also, who aligned those buttons.
...and of course Slack makes this stuff pretty much impossible to turn off 😞
I always struggled with READMEs. When you get round to writing, after the implementation, you are so well versed in the project that I, for one, find it har to explain the project to help someone coming to the project fresh.
Try writing a rough draft for it before you write the code. Often that will help you gain a clearer idea of what you're building as well.
In any case, even if you do end up getting a lot of help with it, writing the first draft first is what will allow you to improve this skill over time. If you never struggle through the writing, you will not learn, and you'll stay beholden to the LLM.
I do write a brief readme to start with, and I've struggled through many times, and have rarely been satisfied with the result. I think part of the problem is lack of clarity about what makes a good readme.
Find some good ones (e.g. some you like), and try to understand why they're good.
I usually find I only read the first few lines. I'll try and note the ones that I have actually read and found useful.
I sometimes do Documentation First. I.e. I write the docs for the thing I am about to build as if it was already built. I then show the docs to myself and sometimes to others and ask if it makes sense and if it would be good if the project had this feature implemented this way. I then iterate on the feedback until the answer is either no or yes. These parts of Calva are the best documented, and also the best implemented.
You can even ask an LLM to point out confusing bits, contradictions, undeclared assumptions etc. They're quite good at rubber ducking on text 😊
@hugod also do not hesitate to ask for feedback on a readme, either one that is published or a draft. (Unless it is AI-generated, lol.)
Human feedback, I meant. And I also agree with @christian767 about AI feedback (though pulling feedback from an LLM takes more skill).
Yes, it's more useful to point out areas that need more work than it is to do that work.
I always lament how little feedback one receives on open source projects, and it seems that any use of AI now means I'll receive even less. My aim for using AI is as a tool to help get more of my ideas out, and to improve their quality. Putting up this project here was an attempt to get some feedback on how to improve the use of AI to help do that, and I knew it was going to get beaten on for AI use. Unfortunately there has not been one actionable comment, other than don't use AI. I can see why there are objections to thing like readme's, but even there most of the objections seem to be about the style of the writing, and value projections about what the use of AI means about the quality of a project.
AI is by definition a mediocrity machine. The only quality improvement you can achieve with it is the same that anyone else who can type a half coherent prompt can get. You will never be able to stand out by relying solely on AI.
You'll receive less regardless because there's way more noise in general because everyone is using AI. It's a market for lemmons.
Here's a great lightening talk from 2024: https://www.youtube.com/watch?v=XhKcelV7DBo "Why your AI generated talk abstract was rejected for NDC Oslo". It's about conference abstracts, but could just as well have been about open source projects. I was on the program committee with him that year, and it was a gruelling experience. I bet it's gotten much worse 🙈
> I can see why there are objections to thing like readme’s, but even there most of the objections seem to be about the style of the writing The writing style is a bit awful, but mostly it serves as a give-away that AI wrote it. The big problem is that AI-generated READMEs are bad at relaying what the project is about. At least that’s the case for me. I just feel stupid as a reader, wondering why I can’t understand.
The video I posted has a screenshot of a tweet/bluesky/whatever (how's that for meta, eh?!) which sums it up pretty well for me: "Writing is thinking, and if they outsource their thinking to a machine I'm not interested in hearing what they have to say."
@pez are you saying the scry readme doesn't relay what the project is about? if so i can take that and do something with it.
I haven’t read that readme yet. But I may make an exception to my “I won’t give feedback on AI-generated readmes” rule in this case. I have a lot on my plate right now, though, but will try to give it a read later.
Thanks @pez - don't worry about the whole thing; just let me know if it conveys what it is about.
I am realising though that I don't have the same level of skill and review coverage for the docs as for code. A positive take-away 🙂
I've skimmed roughly through the readme - it actually reads fine on the surface (and not immediately as LLM-slop the way so many other repos do with the obvious em-dashes and rocket emojis) but the thing which stood out to me was the actual content. Maybe I'm not in the target audience for this specific sort of tool, but the Why part of the readme felt particularly thin - just a bunch of generic sounding bullet points, an unverified claim that structured data is better for agents than 'parsing console text' ( was this benchmarked anywhere? I'd be interested to know - intuitively I don't see why that should be the case, it's all tokens-in tokens-out after all, so any wins of EDN over kaocha printlns would have to be on some fuzzy basis of saliency or token efficiency - none of this mentioned, and I wasn't motivated to dig any further into the logs ) And then most of the rest of the doc seems to be devoted to the 'What' in great detail - API contracts, data shapes, key semantics etc. precisely the sort of thing that the LLM can output mindlessly, conveys no extra information above the code itself, and tells me nothing about why I should want to care about any of it - Not that every readme is meant to be a sales pitch, I've just found it to be a recurring pattern with AI generated ones
(based on the state of the readme at 6aa309b, I just saw there were some changes made in the meantime.)
Hope that wasn't too harsh - interestingly @hugod I found that your comment above
> It solves a specific issue I was having with AI constantly re-running tests trying to narrow down relevant test information.
was far more informative than anything the LLM presumably wrote on your behalf in the ## Why scry? section. Which (forgive me if i'm way off base) sounded more like the agent reverse-engineering your intentions from the prompts you gave it, coming up with a generic sounding rationale which passed the sniff test and landed at the top of a README with much less deliberation than was expended on a Slack message?
Thanks for taking the time to do this - it is very helpful. The distinction between my earlier comment and and the description in the readme about parsing text output is exactly the sort of thing that I am blind to, after having worked on the project - they both describe the problem equivalently in my mind. I'm actively working on getting the long tail of the readme trimmed. Thanks.
As much as I don't like it, we may get to a point where we need to focus more on certifying the software which isn't vibe coded, similar to how we have certified organic produce and then the other stuff. That will instead make it easier for interested folks to say "Nice, let me take a look at this project which someone invested time and passion into building as a human expression." rather than wondering how much AI was used.