Clojurians Log v2

It's possible, yes. We've always known that Pro sponsorship is at the whim of Slack (now Salesforce) and almost every year we've had to find new people in the organization to talk to about getting sponsorship renewed for another year. Staff churn at Slack/Salesforce has meant that both the CEO and the head of Slack has changed each year. Several of the admin team have attempted to contact the current CEO, starting over a month ago, but no luck.

It might be interesting to note then that the https://clojurians-log.clojureverse.org/clojure stopped archiving last august

What are your thoughts on if this year might be the year to "bite the bullet" and try to coordinate a mass migration to somewhere else?

This problem is not going to go away, lets rip the bandaid off

I have ideas

If you have ideas that persuade many individuals, each making their own independent decisions, to freely choose to switch to use another service, such that it becomes the one that folks go to because most other Clojurians are there, be our guest! Despite the existince of other chat services with Clojure channels like Zulip (and probably others I am even less aware of), Clojurians have voted with their feet, to Slack. There is no top-down command here. (And note that they voted with their feet this way before we had the pro Slack service with full message history visible - only last 10K or so messages were visible, and older ones disappeared.)

Clojurians Zulip apparently has become the go-to place for Clojure data science tool development discussions.

> be my guest OK! Proposal: • convince alex, david etc to move #clojure-dev #cljs-dev #datomic and other keystone channels to some blessed place with the same admins as this community, this becomes the https://en.wikipedia.org/wiki/Focal_point_(game_theory) that clojurians slack is today • Then, coordinated social media blitz where various site admins and community influencers all put the word out) • Propose target platform is Discord because it is free, has bots and APIs, i suspect majority of the community already has accounts there from other OSS communities, and it also has a monetization model which might be useful in the future • Exiting specialist communities such as the data science zulip are free to carry on

Note that in step one, it might be difficult to convince alex, david etc to stop using Slack to communicate about Clojure, because that is the current epicenter of Clojure on-line chat. You might be able to convince them to also monitor the new thing, too, but note that you've just doubled their work of putting out chat announcements, e.g. like here in announcements, news-and-articles, clojure-dev, etc. Don't get me wrong -- I'm not saying Slack must be the one true Clojure place. Just trying to explain that a "top-down command" mindset isn't the answer. Persuading the majority via persuasion/sales/marketing is.

they need to believe the cost of losing past and future history is greater than transition cost, and we can reduce transition costs through leadership

zero work is doubled

I was reading through this article I wrote a while back, and unless something has changed since then, note that currently all discussion on at least most channels here is logged on Clojurians Zulip, and searchable with full history there, going back to 2019: https://github.com/clojurians/community-development/blob/master/clojurians-slack-archive-notes.md. I have not confirmed that recently.

Confirmed a few seconds ago that the most recent messages of ours above were copied to Clojurians Zulip #slack-archive channel, along with many other channels here (I did not try to determine whether all Slack channels are copied there.)

you are proposing that slack users continue to use slack but login to zulip to query history?

I am saying it is possible, and documented, and working. What they do is up to them, and I doubt I'm the salesman to try to persuade them to move away from Slack.

> they need to believe the cost of losing past and future history is greater than transition cost This Slack is almost ten years old. It has had full history for only the past 2-3 years. There have been repeated discussions over the past decade about "everyone moving somewhere else" but people just stay here -- even when there was no full search history. > we can reduce transition costs through leadership The Clojure core folks are not the leadership here. The main hub of Clojure activity used to be the "official" mailing lists but over time, almost everyone moved to Slack. The mailing lists still exist but are very quiet. That was an organic shift -- there was no "leadership" advocating for the change. When this discussion has come up in the past (almost every year!), even the moderator team here -- there's 11 of us at the moment -- have suggested folks move to wherever they are happiest. I've said multiple times that I'd be happy if everyone here moved to Zulip since a) that's the closest style of communication to Slack and b) I'm not a moderator there so it would remove my workload quite a bit 🙂

There is a Clojure Discord BTW -- it's linked from https://clojure.org/community/resources -- and it is well "maintained" and moderated but relatively quiet. I'm active on Zulip and Discord. I've recently rejoined the Clojure IRC folks (on libera.chat) -- when I remember to start irc -- and there are still a core of regulars there. I have accounts in all of the Clojure communities that I know about -- Slack has been far and away the most active for most of the last ten years.

@plexus What is the current status of the ClojureVerse log archives? @dustingetz noted that no new posts since August seem to be on https://clojurians-log.clojureverse.org/ but I think I recall you saying that @logbot is still archiving posts, it's "just" the website that isn't getting updated?

(sean i meant your leadership (or even our), core is busy i expect nothing from them)

But, I retract my questions, as at this point you have answered them

If we lose the Pro sponsorship this year, it will be interesting to see if that motivates people more to move elsewhere -- Slack is changing the terms of free accounts to actually delete content older than one year, so once we lose Pro older content would be gone even if we got Pro back later.

@dustingetz In the past, I've actively encouraged folks to move to Zulip. I'd love that. Although, that too relies on Zulip sponsoring hosting plans for OSS communities for full archive/search functionality. It's just an official policy over there rather than "at the whim of Big Corp, Inc." over here.

zulip is not a schelling point, the network effect in play is that people go where core goes

i suspect if we were to ask them nicely what they felt about the impending loss of history, we might find they also are interested in transitioning, if someone else did the leadership work

I am pretty sure core Clojurians go where the majority go, not the majority goes where core does, historically.

Some people really care about chat history, but history of where people have chosen to chat suggests that something about Slack has been more compelling than a long search history. I don't pretend to know what that is.

The majority could choose something else in the future, of course.

Clearly y'all have decided, I accept your decision

@dustingetz That sounds very snarky. No one has "made a decision". For quite a while after this Slack was created, core continued to use the mailing lists -- setup by the core folks! -- to post announcements and have discussions. They didn't "lead" anyone or anything to Slack.

sorry no snark intended, i was slightly frustrated but good now

its truly not my decision, it's yours, and i support you in figuring out what's gonna work

Hey, this is a frustration that some of us -- especially the Admin team here -- have had for a decade. And, no, it is not "our" decision. It's up to the community.

i reject that, community needs leadership, you are that leadership here

Heh, no we're not.

You can reject it all you want, but you can't just dump on us because Slack's history might go away and return us to the state we were in for most of that decade.

In case it isn't clear, it isn't any of the Slack moderator's decision, either. It is the majority of all Clojure Slack user's decisions, at least hundreds of people if not over a thousand. There is no top-down command here.

nobody is dumping, i provided productive contributions

You are saying "You ... waves hand vaguely at the room ... should do this work I'm suggesting!". The Admin team here -- and the Clojure core folks -- are extremely aware that our history exists at the whim of Salesforce. We lived here without history for years. If it goes away, many people will continue to live here I expect.

> convince alex, david etc to move #clojure-dev #cljs-dev #datomic and other keystone channels to some blessed place with the same admins as this community There are dozens and dozens of such channels here. There are many more core people than that - often specific for particular channels. Some of those people don't use Slack all that often, yet provide invaluable insight. I would deem convincing the majority of them to be a Christmas miracle, no less.

i did not use the word "you" in this context, in fact i said "our"

i'm getting frustrated again, so please kindly permit me a gentle exit

If you want to live in a community with history, focus on Zulip. If the hyperfiddle / electric community moves to Zulip, that would essentially be up to you as "leader" of that community. And that's true for all these channels.

Every "community" here is driven by its specific "leader". The SciClojure community decamped to Zulip some time ago. The Juxt team mostly use their own Discourse forum (certainly, the XTDB community is mostly focused on a specific Discourse forum, but they still monitor a few channels here).

Some other bits and pieces that I remember from previous discussions that are relevant (although there's no data with specific numbers backing anything up): • Some people don't want to move away from Slack because that's what they already use at work and then don't want to have to manage some other client and its window • Some people dislike available alternatives (I, for one, can barely stand Discord with its invasiveness - I had to remove the desktop client and I add a new uBlock rule every other time I go there to keep all the "Buy Nitro" and other crap at bay) • Some communities are indeed not present here at all. The whole Russian community, for example, is on Telegram (which I also dislike, but for different reasons) • And some just don't care at all - they're here because they're here and moving somewhere else is an chore that isn't necessary for them

its truly not my decision, it's yours, and i support you in figuring out what's gonna workFWIW, I think all or nearly all of the long-time slack admins have actively encouraged people to go elsewhere (myself included), with no success. IIRC we even talked at one point about the possibility of shutting down this slack to force the issue, but that seemed clearly unfair -- if people choose to be here for whatever reason, it's not reasonable for us to force them out by administrative fiat. I for one would love to see the community coalesce elsewhere.

To chime in here, I joined this slack in 2016 because this is where most everyone is. I was asked to help moderate because i was frequently active. I don’t think i’ve ever ushered anyone here except the network effects of there already being a lot of people here. I have no particular affinity for slack beyond it’s convenience and ubiquity. The only thing i really want is a way to engage with the Clojure community. I don’t care where. I have no motive to keep people here, or against people moving anywhere else. I am only here because we are all here.

Hey folks, just catching up here. I was not aware clojurians-log had stopped archiving. It seems indeed that the https://github.com/clojureverse/rtmbot started malfunctioning at some point. I'll see how much I can backfill via the conversations API, and make sure we archive again going forward. We've done this sort of things before. It's probably time we modernize some of this stuff. The token the logbot uses is something that was passed down to me close to 8 years ago, I actually don't have access to the configuration of that slack integration, but I'm sure it's a legacy thing at this point. Slack has repeatedly changed how you're supposed to do things and made APIs deprecated. We also use websockets to receive the messages, which is supposed to be a dev-time only thing and we should change that to HTTP callbacks. @seancorfield would it be possible to make me (temporarily) an admin so I can more easily look into this stuff. cc @mitesh

I’ve regularly used the Colurians log over the years (I think it’s a great solution), however I have often found that week/months of messages are missing and also whole channels. Although I haven’t done a rigorous assessment of it. It’s possible there are still lots of gaps - not just recent history. Given the number of issues with it over the years, I think it’s not just bugs in the archiving software but also the process in place to manage it and verify that it is working and the integrity of the data. I understand it’s volunteers /best efforts and I appreciate what is being done. If the Clojurians log worked perfectly then I think that could make it much easier to remain here on Slack if they take away Pro.

@plexus are you managing the archiving process for Clojurians log? Do you have comfidenc there are no gaps in the 10+ year history? Do we need a plan to confirm integrity? Or perhaps run a full bulk extract? Is the data only in HTML or available in other formats (eg json)? Anything we can do to help you?

The Gaiwan team maintains clojurians-log (as well as clojureverse). This is as you say a best effort, we make zero promises. There are certainly gaps, some intentional some not. We only archive channels where logbot has been added. If channels are missing then that's probably the reason. This is a social contract. Not all channels want to be archived. The primary archive is a private git repo containing json-lines files of slack events. I will consider offers for help if they are serious long term commitments. Otherwise we rather do it ourselves considering the amount of handover necessary. Many people have been involved over the years, but at the end of the day it always comes back to the Gaiwan team.

Another factor - AI training. One of the benefits of having history is to feed it into an LLM as a training set. One thing that complicates that is threading. Purely IRC like chats have only implicit threading at best. Threading here on slack is structural but people adhere to it only sometimes. I Clojurians-part-2 could benefit from enforcing conversation threading (I think zulip does this - don't recall if discord does). WRT community moving over, I just don't see it happening unless there's an weekly, automated announcement going out to a few main channels saying, "Hey all, if you frequent here, please frequent over there too." And I don't see that as too egregious or annoying for existing users here. I just guess there's not enough appetite here to make that public announcement. Nor a consensus on what that better-next-slack might be.

> an weekly, automated announcement going out to a few main channels saying, "Hey all, if you frequent here, please frequent over there too." If there were such an announcement, I would absolutely go out of my way to remove it from my feeds. I'm subscribed to around 30 channels, seeing that 30 times a week would be maddening. Even just once a week would make me come up with a way to remove it on the client side.

It's just once a week and I'm not suggesting every channel, just a handful. Maybe once a month would work

But people don't agree on slack-part-2 yet anyway

Yeah, I'd still be pissed, even with a rare announcement. :) I try all feasible avenues of removing from my life any info that I absolutely do not need. One of the main reasons why I will never switch to iPhone unless they permit ad blockers there. I'd rather forgo browsing the internet altogether than use it with ads.

Maybe we should have a community competition to hack up a slack clone in clojure. Only rule is they have to federate threads between each other

I think if we have a standalone website with nice archive viewer and search functionality then all the interactions can continue to exist in Slack. The data can be extracted centrally then anyone who wants to build such a website using the data can.

And speaking of history - I don't remember and couldn't find any discussions on having a bot that would search Zulip archives, without users having to go there themselves. Has it ever been discussed?

I would love to include this with the other clojure search tools I have. I think it would be especially neat to have semantic search that can pull up related conversations.