Clojurians Log v2

Unfortunately it wouldn't require much to prompt the LLM to produce a less easily detectable output. I am actually surprised that it is so uncommon to do this.

The discussion that's "not going anywhere" started literally yesterday. :) And an LLM summary, in terms of all the relevant concerns brought up in that yesterday topic, doesn't seem any different from a project that uses LLM. In other words, there doesn't seem to be anything actionable here for the admin team that i worth doing at this point.

Maybe a simple and easily enforceable rule would be to define a cap on the length of the announcements. Handwritten announcements are for sure less noisy

@p-himik I only meant that the other discussion/question is more complex to solve/agree upon and may have no solution. But the one I've posted seems to be more straightfoward (to me).

Handwritten announcements are for sure less noisy

@p-himik You seem overly focused on what the admin team can/must do; like it's the only possible solution. A guideline might be just that, a guideline. Other members can point to those guidelines in the thread if they spot a misuse. Again, I assume absolute most members of this community to be well-intentioned, so just asking people not to do something would already improve the average result.

I think the emoji heavy use is your disclosure 🙃

We got Goodhart's Law on steroids coming into play soon.

cjohansen's announcements do use emojis. :) There are no accurate metrics which can be used to decide whether an LLM has been used. @alexyakushev I cannot promise anything, but you write such a guideline then maybe the rest of the admins would be willing to discuss it. Personally, I abhor writing things that aren't comments. :) In every single paper where I'm a co-author, I haven't written a word.

You see, the problem is I know who we're talking about and what's the end game, so I'm curious to see where it's all going, albeit slightly horrified it's built on a mountain of slop

@p-himik Fair enough, I'm not advocating that somebody does the hard work instead of me. For now, just reading the room if that is something that more people care about, or that it's only few of us that are overly sensitive.

i generally agree with that. My eyes just refuse to read messages that 🚨 Need to parse a file you’ve never heard of? What if you do it faster With 56% improvement over thing i’ve never heard of And capabilities to read the whole file I can’t stand this obvious LLM garbage

@dpsutton Funnily enough, I used to complain about overformatted sludge back when people did it organically.

@p-himik As simple as appending "Don't post LLM output, describe in your own words." to the #C06MAR553 topic header may be a good first step.

This formatting reminds me of rewritten in rust 🚀

but i think that gets to a difficulty about it. People write terrible messages both with and without LLMs. And it gets hard to say “take this message down, i don’t like it” and above it is a well crafted message, with or without LLMs.

I know several people who use A LOT of emoji's and aren't on the genAI bandwagon, they used emojis way before that

@alexyakushev Channel descriptions have a hard length limit that we're already hitting there. And nobody reads them anyway - they mostly serve to point people to when a violation is obvious. @dpsutton But how many of those do we have, really? And how many of those aren't actually generated? I just scrolled back quite a bit, and I see some posts that look kinda like that but that were probably written by hand, maybe.

yeah. emojis, i’ve seen people start bolding keywords in stuff organically just for semantic emphasis now that it’s kind of an established pattern

"Why it works"

I re-read my babashka README.md I wrote 6 years ago and thought I was dealing with an LLM-generated text because of the bolding... something I avoid now since it looks like it's written by an AI

(see the Goals part)

The only thing that's actually enforceable in any capacity is explicitly listing measurable things that aren't allowed. "No emojis, no formatting beyond lists, no em-dashes ("because fuck 'em, that's why")", and so on. But then users of LLMs would simply give that prohibitions list to their LLM and that's it - we replaced slop with slightly groomed slop, cool.

I'm pretty sure @borkdude is an LLM -- the level of output is a dead giveaway. Maybe he's mythos? Ban him! Burn him at the stake!

@p-himik Same as how many instances did we have of people posting patch-releases into announcements. But that is considered noise enough to ask people not to do that. Yet screenfuls of slop output is not noise.

> But that is considered noise enough to ask people not to do that. That's because we actively enforce it!

We cannot enforce "no LLMs", not a chance.

(this isn't an AI pic, but something a student of mine photo-shopped 15 years ago)

"Could you please reword your announcement so it doesn't read like it was written by borkdude? Only he can have that flair on this server, thank you for understanding."

i also don’t think the community wants no LLMs

The community is (I think) pretty respectful/considerate. Even periodic reminder posts and/or an "Announcements written by humans, not machines" type header might be enough to establish channel etiquette for most people, regardless of their personal positions on the topic...

I feel this thread already has much more comments than we have AI-written announcements. :)

i’m down with periodic reminders of “remember we are not bots. so write messages for humans here, not the robots” type thing

big on encouragement, think rules are tougher

Some people thought the highlights in the Deref were written by an LLM (I initially suspected it a bit, I must admit) but the author said flat-out that he didn't. So I guess it's just really hard to detect it. I also recently had this outside of the Clojure community. I thought someone's blog post was AI generated but he wrote it years ago he said.

Anyway, I'm off to try and close out a feature branch before finishing up for the week. It's written by me, mostly... 😉

@p-himik Slop needs some organic timewaste to train on, amirite. But yeah, it's not that much of it. For now.

Oh, I just remembered another thing. We actually had a user here or some place else that did sound like a genuine LLM. They were almost banned, some people complained to the admins. But it turned out that they simply didn't know English and used an LLM to translate texts in their native tongue into English that reads well.

I’m currently using an LLM to profile different ways to reuse buffers when rendering images. pool of BufferedImages, pool of backing arrays, etc. It’s quite helpful in certain circumstances

yes. i remember someone posted in #C8NUSGWG6 and i told them that it looked like LLM output and made me not interested in reading the content. They said it’s all human, but helps in second/third language publishing

@p-himik I keep hearing that as some positive LLM usecase, but it's not. Guess how effective that person would be at learning English eventually? Why are we suddenly so against learning? Not knowing a thing is not an impediment, it is overcomeable – by practice, not by having a clanker speak for you.

We also did have some other person who was very LLM-usage-heavy. To the "don't ask me questions, ask your LLM about my projects" degree. And it seems they haven't posted in quite a while, so the situation organically resolved itself, without any guidelines.

Guess how effective that person would be at learning English eventually?

Why being polite is a prerequisite for interacting with others if curses and shouting gets the same point accross?

Being polite does not cost as much as learning a whole new natural language.

I think forcing English upon everyone is harsh. We should all learn Esperanto which is a neutral language.

Finally!

I think we can converge on some common ground with PIE

Might be some gaps need filling

I'm a fan of cranberry when it comes to pies and fillings.

I meant Proto Indo European but I guess we can just use pies, sure, I spent quite a bit of time perfecting my pumpkin pie

I know, but I thought you deliberately made that pun. :D

Heh. I had to look PIE up. I do have a friend who insisted on writing a masters thesis on reinforced concrete in Irish. He had to write a technical dictionary first. Could ask him to do the same for Clojure/programming?

It might take a few years though -- watch this space.

does not cost as much as learning a whole new natural language.

If a chatbot does your English heavy-lifting for you, it doesn't mean it saves you any time programming. :)

>> Handwritten announcements are for sure less noisy > They aren't in general. > They tend to be, but there are handwritten announcements that are quite lengthy. > Now imagine an admin team policing that maybe an LLM could police it 😛

Maybe an LLM could summariz.... eh ok, no, sorry. ;)

Slop emoji is the answer I've seen. Basically lets the community mostly flag content. You don't remove it but it warns other readers that the announcement might be slop. Common one is the 🤖 emoji. Obviously nothing stops people tagging non-slop content with it. But, it is mostly a hint that the content might not be worth your time.

That approach has a high chance of increased toxicity and people missing stuff because of mislabeling. A binary yes/no can never reflect the nuance.

It's not binary though

If something has 10 robots it means at least 10 people thought it was slop

The LLMs have already made the slack toxic

True, but it can also mean that 1 person thought so and 9 others cheered. Or 10 people took a glance and left it as a knee-jerk reaction because of emojis and em-dashes or whatnot.

> The LLMs have already made the slack toxic We deal with things like that on a case-by-case basis. No single solution will magically alleviate it. But every suggestion I have seen so far is IMO certain to make things worse.

I personally don't think enforcement is all that important. What I want is for a social community to have some stated policy on how we expect people to respect each others time. To quote Rush: > If you choose not to decide, you still have made a choice By having no policy for dealing with generated content at all it's essentially a free for all. Over time I suspect the skeptics will leave, and all you're left with is the slop. I also don't think the community guidelines sufficiently addresses this. Reading Rich's take on AI I can only assume the guidelines would have included some wording to deal with the development of the past few years had they been written now. But that's speculation on my part.

A policy means policing. So yeah, we will not have a policy any time soon, the way I see it. A guideline - maybe, but it's something that has to be good and has to be done by someone. > Over time I suspect the skeptics will leave, and all you're left with is the slop. This assumes that there will never be any change. But as I said - we already deal with things on a case-by-case basis, and we will continue to do so. If the skew becomes apparent, there will be effort to correct it. Don't extrapolate based on a few months and a few data points. And you don't seem to consider the opposite - with a bad policy/guideline/execution, people that use LLM might also start leaving. Some will cheer, especially people who call any LLM usage "slop". But IMO a blanket exorcism of LLMs would be a net-negative.

As for the emojis - the latest post in #C06MAR553 that has 🤖 under it looks perfectly fine to me. I see myself writing something like that, if I were to ever release a public project. Nothing in the repo mentions any kind of AI. It doesn't have CLAUDE.md or anything like that. If I were a person who looks at 🤖 and skips the announcement altogether, I would've missed on something I might've wanted to use.

I guess we disagree on the urgency then. In any case, not having clear expectations hurts "both sides" since it causes so much frustration.

I feel like I keep seeing the same arguments, the same hypotheticals, the same everything in this and in the other thread about stuff in #C06MAR553. And it's getting old for me. So instead of asking specific questions and not getting specific answers, I'll put it bluntly: if you care about having a guideline, please write such a guideline. We'll then discuss it here, discuss it in the admin chat, and decide what to do next. Nobody else will write that guideline but you, dear reader. The future of this server lies in your hands.

> having clear expectations hurts "both sides" since it causes so much frustration. How much? How do you measure it?

@p-himik come on, it is slop, gptdetector flags it.

GPT detecting platforms detect my writing, my wife's writing, my mom's writing as if it was written by an LLM. The rate of false positives is immense. And as it's been said at least twice, it's easy to tune an LLM to output something that reads as human-written.

Can you give an example of your writing that rates 100% ai by the detector? I'm interested to see it and change my mind.

It's 100% slop. It's perfect linkedin speak. Also if you use codex there's no commit hint and no claude.md. The point is the announcement is pure slop. Project, may or may not be. The last announcement by the same author was also not written by a human.

Finding that again and deciding whether it's something I'm willing to share here or even privately is quite a bit of work that I'm not willing to do. :)

So, trust me bro then?🙃

Up to you.

But I went back and actually checked that announcement with https://www.zerogpt.com/. It printed out vague "2.4% AI GPT". Checked the whole README - 1.3%. Your own writing, @alexyakushev, gives 15-20% "AI GPT" on the same service. :) So what are we talking about?

2.4% is not 100%, don't you think?

Exactly?.. The announcement that was marked as 🤖 does not go anywhere near 100%. So why was it marked as 🤖 ?

You wrote: "come on, it is slop, gptdetector flags it." Which one?

20% on my stuff might be from may be from my earlier dabbling with LLMs in 2023-2024. Dropped that since. Which article do you refer to?

I tried only the most recent 20 articles or so. Some copied in full, then got lazy and copied only the intro and the first chapter.

Just pasted the announcement of wandler into gptdetector (minus the code blocks) and it gives 100%>

Different services give different results. https://gptdetector.net/ is for some reason 502 for me.

Should check my posts, maybe I'm getting too mechanical

> minus the code blocks This is also an arbitrary decision. Some people will do it, some people won't, some will choose to exclude other parts. Trivial to imagine stuff like "the intro looks human-written, so I won't include it in the text to check that the rest of the text is AI-written". So different people will be getting drastically different results. And as I said, it's also trivial to fool those services - false negatives are easier to achieve than false positives.

Anyway, I'll leave the thread to y'all, @ me if someone misbehaves or writes that guideline.

I just scrolled back through all the #C06MAR553 for June, hoping that we could pick a criterion like "length of announcement" as a way to dial back the LLM-authored stuff... but, no, several very clearly human announcements are longer than the one folks are flagging as LLM-authored. One of Alex's Contrib library announcements and a couple of Howard's announcements -- very clearly all handwritten -- were substantially longer than the ones folks here are complaining about.

I asked. Yes drafted by claude and reviewed. https://clojurians.slack.com/archives/C06MAR553/p1782579898611519?thread_ts=1782460719.454369&cid=C06MAR553 Like I said it reads like claude. People underestimate how quickly the human brain can tune to detect claude. LLMs are incredibly repetitive in terms of style.