Clojurians Log v2

@stefan.van.den.oord I assume this is local? Because I don't see any PR builds from you on CircleCI. Generally, timeout tests are flakier than we would like. But the multipart tests shouldn't be, iiuc. That being said... I can't replicate your results. All those tests work for me locally. I tried bumping Netty up to 4.1.100.Final, and they still passed. If you think the branch is ready to merge, make a PR, and let's see what Circle says.

Ok will do!

By the way: I'm trying to make that pull request, but the tests are failing on master for me, even without changes:

Testing aleph.http-timeout-test

FAIL in (test-shutdown-timeout-2) (http_timeout_test.clj:70)
shutdown with a timeout of 0 second and no grace for in-flight requests

expected: (thrown-with-msg?
           Exception
           #"connection was closed after 0\."
           (clojure.core/deref resp))
  actual: #<clojure.lang.ExceptionInfo@60ea912b clojure.lang.ExceptionInfo: connection was closed after 0,054 seconds {:request {:path "/", :user-info nil, :pool #object[io.aleph.dirigiste.Pool 0x33945d80 "io.aleph.dirigiste.Pool@33945d80"], :throw-exceptions? false, :request-url "", :server-port 8082, :uri "/", :server-name "localhost", :query-string nil, :scheme :http, :request-method :get}}>

FAIL in (test-shutdown-timeout-4) (http_timeout_test.clj:94)
shutdown with a timeout of 1 seconds while waiting for body

expected: (thrown-with-msg?
           Exception
           #"connection was closed after 1\."
           (clojure.core/deref resp))
  actual: #<clojure.lang.ExceptionInfo@6657d966 clojure.lang.ExceptionInfo: connection was closed after 1,056 seconds {:request {:path "/", :user-info nil, :pool #object[io.aleph.dirigiste.Pool 0x33945d80 "io.aleph.dirigiste.Pool@33945d80"], :throw-exceptions? false, :request-url "", :server-port 8082, :uri "/", :server-name "localhost", :query-string nil, :s

Testing aleph.http.multipart-test

FAIL in (test-multipart-request-decode-with-ring-handler) (multipart_test.clj:216)
automatic cleanup with zero memory-limit

expected: (not (.exists file))
  actual: (not (not true))

FAIL in (test-multipart-request-decode-with-ring-handler) (multipart_test.clj:216)
automatic cleanup with zero memory-limit

expected: (not (.exists file))
  actual: (not (not true))

FAIL in (test-multipart-request-decode-with-ring-handler) (multipart_test.clj:216)
automatic cleanup with zero memory-limit

expected: (not (.exists file))
  actual: (not (not true))

I can give it a go later on, but like @dergutemoritz advised I'll stay on 0.6.x for production for now.

OK, you should be able to look at the 0.7.0-alpha2 code to see how we configure cert name validation, and apply it to your own setup

Right. My take-away from the earlier discussion is that I'm not affected because we're only using the server part of aleph.

> Can you update the transitive dependencies without breaking the direct dependencies? If everybody is precise with version numbers, it should be possible, but in practice... The major issue is Java's classpath is based on namespaces alone, without nesting/versioning. This results in things like Hickey arguing for namespace changes with breaking changes, but that's just a bandaid for Java, and is a very heavy "fix". It doesn't really solve the problem. Long-term, langs need to move to versioned/nested deps, so multiple versions of the same dep can coexist. This is how node_modules and Nix work. (Kind of weird, but this is one thing Js does right.)

If you're server-only, it should be fine. But if your server makes any network calls, remember that's the client, too

Yeah indeed that model in JS is better, but I wonder to what extent it causes people to be sloppy about versions and breaking things, because they feel it matters less?

Sloppy, sure, but shouldn't versioned deps break things less? (Assuming the versions are properly isolated and independent). This is where Nix is even better than node_modules (well, some node tools can dedupe the nesting), because it stores everything based on the hash of it and all its deps.

They should yes, sure 🙂

It guarantees that there is zero chance anything will use a version it wasn't built to

Nix is super-cumbersome to use right now, but I'm pretty sure all packaging/deps systems will use something like it in the future. The only downside is using extra disk space, but that hasn't been a real concern for years now.

I'm not familiar with Nix myself, but I'm hearing good things about it. And re disk space I completely agree. Apple has been bundling all libraries within application bundles for years now. Yes, there's duplication, but how nice is it to be able to just drag and drop an application to another computer and it just works?

Agreed. There's still the issue of communicating that you might have to do some work when breaking changes occur, but I don't think either semver or artifact name changes solve that problem (insufficient consistency in interpretation, and too heavyweight, respectively). We'll be reading changelogs for a long time, I think. 😛

Yup 🙂

Ah, yeah, I went by looking at the changelog for Netty 4.1.100.Final and it does indeed contain a fix for a different one, namely CVE-2023-44487.

See https://github.com/netty/netty/security/advisories/GHSA-xpw8-rcwv-8f8p

Hmm that's slightly concerning, that I'm not seeing that one now. Apparently the suppressions are less selective than I expected, having a tag <cve>...</cve> in them.

Other than that, I cannot recommend Nix enough, though as pointed out by @kingmob it does have some rough edges which make it a bit annoying to use. But it's slowly getting better and the more people help out, the faster it will :)

Also on Mac, or specifically for Linux?

@stefan.van.den.oord Looks like https://nvd.nist.gov/vuln/detail/CVE-2023-44487 doesn't yet have any relevant selectors

> This vulnerability is currently awaiting analysis.

so probably a matter of time till it shows up

ah ok

Looking forward to it 🤪

@stefan.van.den.oord You can also use it on Mac (Nix only then). That way you can get a taste. Then there's NixOS which is a whole Linux distro of its own which unlocks even more super powers. I've been using NixOS as my daily driver for about 10 years now and for a few years we also use it exclusively on all our servers at work. We even made https://github.com/bevuta/clojure-nix-locker to be able to fully nixify our Clojure application build 🙂

Oh that's interesting, that you're using it for your servers. Specifically for servers, what does it add over something like Debian in your opinion?

Oh that's a broad and deep topic 🙂 Let's hop over to #nix for that

Then other folks can chime in with their perspectives, too

From what I've seen this CVE only concerns Swift implementations though. Does it concern aleph?

@vale It affects Netty's HTTP/2 server implementation, too (see https://github.com/netty/netty/security/advisories/GHSA-xpw8-rcwv-8f8p) but since Aleph doesn't expose that, yet, it isn't affected.

@vale The rapid rst_stream CVE applies to all HTTP/2 servers. It's just in the nature of how HTTP/2 works. In a certain sense, there's nothing really new or exciting here. It's another twist on getting the server to waste CPU cycles, where there's an asymmetry in cost between the client and the server. Anyway, Aleph isn't currently affected, and I'll make sure the new H2 code uses the fixed Netty version

@kingmob Not all of them - haproxy was immune already 😄

heh

stops picking the nits

@stefan.van.den.oord I just got a notification about this comment: https://github.com/jeremylong/DependencyCheck/issues/5912#issuecomment-1761508540

IIUIC NVD accidentally considers 4.1.100 a fix version for that CVE, too? Maybe that's what threw you off

Hmm that might very well be, weird.

Note that the CVE only affects the HTTP/2 server implementation of Netty which is not yet exposed by Aleph

However, bumping the Netty version would still make sense, if only to silence automated vulnerability checkers 🙂

PR welcome!

Note that you can also override the Netty version on your end

so you could stick on 0.6.x for prod and update it to the latest Netty on your end

which is what I would recommend

Hi @stefan.van.den.oord, can you share a link to the CVE you're concerned about? We've tried to be very responsive when people bring them up in the past, and usually it's pretty easy to just upgrade Netty. But, as I've learned a little more about the CVE landscape, the picture looks a slightly less rosy: mischaracterized/overhyped severities, ignoring relevant authorities, etc. Because of that, I'd now like to review CVEs before making decisions. Even a simple Netty upgrade still requires checking the changelog (and Netty is a large project) to ensure nothing will break.

Sure, should have done that right away, sorry about that. I'm looking it up...

https://nvd.nist.gov/vuln/search/results?form_type=Advanced&results_type=overview&search_type=all&cpe_vendor=cpe%3A%2F%3Anetty&cpe_product=cpe%3A%2F%3Anetty%3Anetty&cpe_version=cpe%3A%2F%3Anetty%3Anetty%3A4.1.94

It appears to be that CVE-2023-4586 again. Maybe they did something in Netty to resolve it or something?

https://github.com/netty/netty/issues/8537

Hmmm, so a couple things. First, that link is talking about Hot Rod, not Netty. It doesn't even look like Hot Rod uses Netty afaict. It just seems to link to Netty at the bottom as an example of other software with the same problem. Though weirdly, if I look at https://ossindex.sonatype.org/vulnerability/CVE-2023-4586?component-type=maven&component-name=io.netty%2Fnetty-handler it refers to Netty. I...don't know why NIST and Sonatype aren't on the same page. Anyway, I think @dergutemoritz thought you were referring to a different CVE (this is why we need links), because your issue applies to the client, not the server, it applies to HTTP1 as well as HTTP2, and it was fixed in the latest 0.7.0-alpha2 a few weeks ago.

Netty itself is taking it's time, and if you look at the issue page, you can kind of follow why. As a lower-level library, it's a more defensible stance to say users need to configure their TLS setup properly. They say they'll make it the default in Netty 5, but they don't want to break backwards-compatibility in 4.x

I personally think they should do it, but I understand why they aren't

I guess I was confused because I suppressed the earlier issue, and now this popped up from dependency-check, but initially without the CVE number. I just noticed that now it does include it.

Yeah I also understand the breakage is not something to lightly do.

On one hand, it only breaks for people with misconfigured server certs. OTOH, Netty has a lot of users, so it'll definitely break for someone when deployed, and that's not great if their network is secure in other ways.

I think earlier it was only tagged for one of the netty dependencies, and now for pretty much all of them.

It's stuff like this that leads to people considering CVEs on a case-by-case basis.

Anyway, give 0.7.0-alpha2 a try. It has HTTP/2 in the client (but not server yet, that's still under review), and it's not enabled by default, so it should be a drop-in replacement

Yeah I find it sometimes very difficult to judge what to do about some of them. Can you update the transitive dependencies without breaking the direct dependencies? If everybody is precise with version numbers, it should be possible, but in practice... And if you suppress things, what is the best way to suppress them?