Hah it seems that someone put things from my head on a paper! Anyway didn’t we have a call two weeks ago, did we?

Somewhat agree, but I have to say that these lists have a certain circle-jerky flavour to them. What did we really learn reading any of those bullet points? The statements are never qualified in any way. This exact blog post or something close to it floats to the top of Hacker News nearly every single day. It’s the kind of stuff I will read when I sit on the toilet to pass the time.

"Cargo-cults are bad. Anyways, here's my strong opinions about a bunch of stuff without any explanation." 😄

Precisely.

I don’t think that these bullets try to qualify anything, they are based probably by some experience that’s it 🙂

IME when you’re 6 years into development you go through the phase of denouncing some “accepted wisdom” that you consumed uncritically at the beginning. It helps 🙂

• “90% – maybe 93% – of project managers, could probably disappear tomorrow to either no effect or a net gain in efficiency.” Even if true, IMO it’s a challenge determining which are the 7-10% worth keeping around.

Hah, as someone with PM duties, I think that people really underestimate the role PMs play — or haven’t been in any reasonable large or long project with no PM, and haven’t seen what a shitshow it can be 🙂

in the take economy, detail and correctness are liabilities - being loudly wrong drives way more engagement than being boringly correct. that's why these lists are always at the top of HN: everyone feels the need to correct them

Using Clojure for a while, then looking in detail at how other languages implement these things, can certainly be a way to see why Clojure's implementation of equality semantics can be measurably slower than those others, I think.

Clojure's equality semantics is definitely doing some extra work for you, e.g. in Java land at least, to compare different storage sizes of integers as equal in integer numeric value to each other, not equal in (type, integer numeric value) combination as Java equals() method does.

@U0CMVHBL2 that is true. On the other hand (in my not so big experience with Clojure), it is so much more robust and predictable in the wayit works than what happens in other languages, it frees up a lot of mental resources and lines of code. Also, it seems to me, if it comes to a point where Clojure's equality semantics is a performance bottleneck, probably rethinking the whole idea of simply equating nested data structures for this specific use case would yield much better results than any possible optimizations.

I agree with you on the freeing up mental resources, no question. I have often looked into some of the implementation details and performance, which is the only reason I am aware that there is some performance cost for Clojure's = behavior. My current guess is that most of that performance cost is not due to the nested data structures reason (if you wanted that deep-equals behavior, you couldn't optimize it away in any way I know of, anyway), but the "making different JVM types be equal, despite their different types". Take that with a grain of salt, as I haven't done actual experiments/measurements myself to authoritatively say that X% of the cost of Clojure's = is due to reason X

I've heard the term "sharding" used in the context of databases, but I think that term is even more specific and has to do with how indexes are divided up. Or maybe I'm splitting hairs...

the most common version of this I've found is with CDN used for get of static assets - I don't know the technical term you want, but adding CDN to your search query might help

this is more specific than just segregating GET, it also looks at paths to determine which are static

sounds like a good case for hiring a contractor. kafka is a distributed immutable log, it can do some of the things that pulsar is used for, but isn't well suited for most pulsar uses IMHO having used both.

if what you want is persistent communication between services that spin up / change servers, pulsar is simpler than kakfa

if you want immutable data logs that can be used flexibly, kafka does the job (but is much more finicky to keep running)

pulsar is about messaging (which means it has logic about messages, senders, and recipients baked in), kafka is a distributed log, using it for messaging requires a lot of hacks and conventions and leads to weird corner cases in setup

Yeah hiring a contractor is definitely an option. Pulsar is serving us well for the most part, we’re not really looking to change to something else. But when there is a network outage or something in the data center, or on one of the nodes things go down due to out-of-memory, they take all of the other nodes down as well. That was not the point of having a four-node cluster I thought 🙂

in distsys even numbers tend to be a bad idea generally - you need majority so 4 nodes perform worse than 3 (or at least no better)

more relevant to your original question: confluent offers a docker compose config to spin up kafka locally, and you can purchase a kafka cluster as a managed service from aws

Confluent also has an open source operator for Kubernetes that seems to work fine. But like @U051SS2EU said, the typical use case for Pulsar is quite different from Kafka. The major drawback of Kafka is you can't commit a single message, and also can't easily read a single message. It's really made for streaming.

also IIRC pulsar has straightforward request - > response, which is something service clients tend to take for granted, making request->response on top of kafka gets super weird and messy

While you can't read or commit single messages in Kafka you do have transaction semantics which you can use to narrow your scope down to single messages, no?

@UK0810AQ2 not really. If you know the topic-partition and the offset you can just get one message, but it’s cumbersome. Or you can use ksqlDB, or similar things that need additional middleware to make it easier.