the borkdude way to enlightenment: I don’t understand this well, I’ll go ahead and implement it!

mkvlr revealing some internals 😄

Now with CLI (thanks @U3X7174KS) and with C-u M-| you can replace a region :)

If it is not Clojure, the Bork will make it Clojure.

Can it port Clojure to Emacs Lisp?

@U0PUGPSFR it tries ;)

oh you mean, the entirety of clojure? no

Looks like I could use something like this for my MQTT broker. Thanks!!!

Happy to do it, hope it helps!

hmm... I just noticed it said that you can store clj maps... but I would need to store raw bytes (with some meta data together in a map)

Oh interesting!

@U052852ES, I think I could make the encoder fully configurable with a single one liner. I'll have another look in the next day or two.

Thank you. But I have to say I have no idea when (or if ever) I'll look into this.

So don't feel pressured at all to do this.

@U050SC7SV, could not have done it without Tape. 🙂 Thanks for all the hard work on a great lib!

@U0HJK8682 I've got some errors. How to fix it? I'm on JDK 19.0.2 Mac M1.

Hi @U097654L8 , I'll have another look tomorrow but that initially looks like a Java version issue. ChronicleQueue does not support Java 19 yet: they https://github.com/OpenHFT/Chronicle-Queue/issues/1286, which are so far 8, 11 and 17.

@U052852ES, coming back to your question this morning, unless I've misunderstood it, I think the serialization layer already supports what you've described. You can wrap some bytes in a map with some metadata. Then write and read that back from the queue:

user> (def q (q/queue ::q))
user> (def a (q/appender q))
user> (def t (q/tailer q))
user> (def bytes-in (byte-array (map byte "data")))
user> (q/write a {:bytes bytes-in :metadata :meta})
83485574299648
user> (String. (:bytes (q/read t)))
"data"

Cool, thank you @U0HJK8682! 🙏

Very cool.

Are there any resources/docs/books on how people use and develop with queues? There is some material around queueing systems in relation with events, but especially for IPC I don't see a lot or I'm not searching correctly. I've only seen some very simple tutorials on how to build your own queue, how to use a queue API, etc, but nothing much around how to think and "architect" using queues. (sorry, I guess this is a diversion of the main discussion 😞 )

@U8ZE1VBSS, not a diversion, that's a great question! It sounds like you're not asking about how queues are used in algorithms, so I'll focus on program/systems design instead. You also mention you've seen material on "queueing systems in relation to events", which I interpret to mean event messaging in distributed, or "big data" systems. So technologies like Kafka, and its use cases. If so, I think you're right that this tends to dominate the discussion nowadays. That said, many of the concepts that apply to distributed systems also generalize to IPC, or really any concurrent computing problem. So for example, things like: 1. Change capture 2. Event sourcing 3. Decoupling consumers and producers 4. Managing process overload/backlog 5. Pub-sub 6. Implementing MVCC and other resource sharing strategies 7. Fault tolerance All these things generalize to IPC, database design, and other scenarios that are not necessarily distributed, and are good jumping off points for your search on queue applications. I would also make sure to modulate your search to include "logs" since they are related, have similar design patterns, and are sometimes used interchangably with queues. As for resources, if you haven't already encountered it, I found that "Designing Data-Intensive Applications" touched on logs and queues in many different contexts. And while its focus is not really queue or log specific, it is frankly a great read regardless. Also, a long time ago I found https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying to be a great introduction to how logs and queues fit into broader architectures like, databases, ETL, microservices, and distributed systems. It has supplemental links at the end. You also might want to post your question in the main #C03S1KBA2 channel. There's such a wealth of knowledge there, I'm sure you'll get some very non-distributed queue specific resources!

btw, conversation can be had in #C04H5DMM99S