Fork me on GitHub
#clojureverse-ops
<
2021-02-10
>
borkdude15:02:39

@mitesh ^ does this ring a bell?

borkdude15:02:34

@rahul080327 suggested I'd ping you. not sure if I was supposed to ;)

👍 3
oxalorg (Mitesh)15:02:41

Fixed, it's back online! Need to find the time to find the root cause of this, basically we're running out of memory (most probably due to datomic)

borkdude15:02:16

seems to work again! 🎉 thanks!

clojure-spin 4
borkdude15:02:48

maybe generating static HTML can prevent this problem? it's an archive right?

sheepy 4
oxalorg (Mitesh)11:02:38

I can't say I haven't thought about it, but at the moment it's too much to port, and is deployed at a few more high volume slack servers. Rendering thread messages etc go back in time and update some old archives too. We also have some code related to querying for stats, showing graphs, etc. So I don't think the benefits outweigh the cost, atleast not at the moment! 🙈

borkdude11:02:53

I mean, the logs themselves do not change over time? So some things can just be static HTML?

borkdude11:02:10

> Rendering thread messages etc go back in time and update some old archives too. Ooh, aha..

borkdude11:02:22

Well, at least nice that you considered it :)

💯 4
oxalorg (Mitesh)11:02:56

Yeah I think we go back and delete messages too (if we get a delete event). Not a 100% sure about this. But incremental / selective page generation is not a trivial problem to solve 😛

oxalorg (Mitesh)11:02:30

Also the emojis, they also update for all old messages

borkdude11:02:39

I thought it was as simple as IRC logs, but it sounds that it's not :)

✔️ 4
plexus07:02:45

we do honor delete messages, yes. Once logs are one to two weeks old we can generally assume they won't change any more. The original version that I inherited generated static HTML, but it meant that for every small improvement we had to rebuild the whole set going back to 2015. It was also a mess of cobbled together javascript/python tools that nobody wanted to contribute to, se we redid it as a Clojure app. Being backed by a database does give us great flexibility, the only thing I would do differently is not have it be datomic. It's not a good fit for this, and is way too resource hungry for a non-profit project.

✔️ 6
💯 3