Clojurians Log v2

sometimes a library is just the right size and does just the right thing

Thanks @mpenet - I’ll certainly take another look once the volumes are there to make it a requirement. Complete makes sense and I certainly didn’t intend to describe it otherwise. There are still options and this is still one of them.

fyi tape was created after we had some deadlocks with durable-queue and needed something more stable, it worked quite nicely (used it at previous job). ChronicleQueue is far from perfect, they tend to break backward compatibility at every release, but back then I had no time to either start from scratch or fix durable-queue, and there were no other alternative ready to go. Maybe nowadays there are better alternatives, not sure.

Tape is « complete », rather than dead. I suppose I could upgrade the dependencies but that’s about it.

Currently no commits since two years ago, not working with later Java versions. For now just http client to Sendgrid will do fine.

Hmm, @slipset is the "owner" of that fork. I just updated and merged the Java 13 PR. We can probably get this updated and a new release cut -- but it'll have a different group ID since we don't have access to the Factual group on Clojars.

These seem two be two different concerns. 1. you want a library to solve a problem. 2. you want some hosted service? for 1 there are other options too, I guess, like kafka or rabbitmq or maybe some java lib that does a similar thing

I’m reviving an old startup I did, all in Clojure. But there’s a mailer queue that needed some attention and it used durable-queue (to avoid Kafka overkill (Kafka expert is my day job so I know it’s overkill 🙂 ). Same RabbitMQ or NATS, just too much for what will probably be needed. I even got into trouble on hackernews over this. Single node Kafka isn’t such a bad option now with KRAFT.

and AWS SQS isn't overkill?

My sentiments exactly Michiel! 🙂

Ah well Kafka it is then….

I looked into alternatives. Something named Chronicle-Queue came up, it's a Java lib

Thank you, I’ll take a look at that.

I’ll run Kafka native in a container, everything else is going into containers. Thanks @borkdude, appreciate you taking the time, most kind. 👍🙏

There seems to be even a Clojure-wrapper around Chronicle-Queue here: https://github.com/zalky/cues > For when distributed systems like Kafka are too much, durable-queue is not enough, and both are too slow.

Thinking about it a Kafka consumer just works perfectly, all the other API calls were retired.

I mean, if you’re running in aws, sqs is really easy to set up. I wouldn’t avoid it for any reason.

Wait, this is for a mailer. Yeah duplicate sends are a thing with sqs.

I’m fortuned enough to know that I don’t trust any system that says “exactly once” 🙂

They have a thing that dedupes by id within a timeframe iirc. So it’ll be mostly good. The way I solved it once was to lock around a row in dynamo, but that’s annoying af for the use case. Idk if you’re using a db I might just shove it in there. If your volume is low, I might just put it direct in the request handler.

Volume is low for now so I’ve just been thinking about a direct request, overhead is low.

Unless this is a key part of the business, I’d lean toward that. You know how to solve this later, it’s just annoying.

What's your concern about durable-queue moving to clj-commons? That at least means it has an active maintainer and will get fixes etc.