Clojurians Log v2

Bit late to the party, but if you're trying to understand how Netty handlers work, it helps to know that, at any time, you can write from the inbound handler chain to the outbound chain. This short-circuits invocations on all later handlers in the chain, and is usually a good idea for efficiency. Aleph wraps the user handler with its own netty-compatible handler, which takes your Ring response, adds the Server header (among others), and in 0.4.x, starts writing Http* objects. This calls the previous handler's .write method and works it way back in reverse. This is why .addBefore is what you want. My suggestion is to insert a netty/channel-outbound-handler and set the :write so that if it gets an HttpMessage, it alters the .headers, and passes it along. If it doesn't get an HttpMessage, it should just pass the object along (the general Netty strategy).

Modern Aleph will only set the server name if the header is missing, but that still doesn't help if you want to remove it entirely. Because of how the underlying Netty Channel multiplexes HTTP/2 connections (one shared connection-level channel, and then one channel for each HTTP/2 stream ID), the pipeline-transform keys are different, as you noticed. You'd want to .addBefore the handler again, but you want to use the http2-stream-pipeline-transform key this time

(NB: that's for HTTP/2 servers. If you have an HTTP/1 server, it's still pipeline-transform)

Oh, this is fantastic. Thank you. It all looked a bit interceptor-y.

Yeah, there's a lot of similarities, but fyi, that pattern for pipelines is older than the use of the term interceptor or the InterceptingFilter. Netty in particular predates Pedestal's popularization of the word. I dug a bit into the history of it all at one point. See my comment here: https://clojurians.slack.com/archives/CDG3YQV6Z/p1685076301109489

ok I'm so glad I pressed on past the 2-minute solution of (assoc-in request [:headers "Server"] "*REDACTED*")

If it's a pentest I'd personally add something like Server: Django/5.7 (Python/3.2) (look up the actual django string) and enjoy the onslaught

Looks like I get "Server:\n" from curl

I'm vaguely understanding I should proxy io.netty.Channel/ChannelOutboundHandlerAdapter because it has default methods, but I'm not at all convinced its .write method is the right one to override.

Try printlning the pipeline after you've added the scrubber. What does that show? Also, do your debug printlns get invoked?

Actually, not just printling the object, but (println (.names pipeline))

hm, no ... nothing prints. I ran the 2 forms in the comment, output on the right. Looking at the start-server docstring for 0.4.7 I seem to be lining up the :pipeline-transform keyword OK. Wonder if I'm not matching a method arity someplace.

oh, wait 🤦 ... output is going out to the REPL but my editor's not picking up output on some other thread

Yeah, just wanted to say that

Happens to the best of us 🙃

ok, so this is the pre- and post- (.names pipeline)

so the transform is running

Try putting the scrubber before http-server

(.addBefore pipeline "http-server" "scrubber" SCRUBBER)

huh, just after I pasted the last one, I got all this warning logging having done nothing else in the meantime

Yes, you got to instantiate that handler anew on each pipeline-transform

make it a defn instead of def

ohhhh, ok

getting closer ... the write method is actually running now

oh ... bad call to proxy-super

pretty close, the issue now is that the (when (instance? HttpResponse ,,,)) isn't the right thing to do

but now I can inspect, this is great

Println what's coming to you

cool ... I'll poke around a bit more with the netty class hierarchy but this is great progress, thank you

BTW, instead of proxy you can use this:

(aleph.netty/channel-outbound-handler
 :write ([_ ctx msg promise]
         (prn "running write method")
         (when (instance? HttpResponse msg)
           (prn "removing header")
           (.remove (.headers msg) "Server"))
         (.write ctx msg promise)))

Is this .addBefore now or .addLast?

before

Then it probably should be .addLast still

Because since you receive a byte buffer as a msg, it means your handler got invoked too late in the chain

hm, as .addLast it doesn't get invoked at all

(the scrubber proxy)

Between middleware, interceptors, and Netty handlers, I completely give up trying to understand the order of invocation in these chain of responsibility atrocities.

yeah, I usually just have to set up mw1 and mw2 and watch the logs

no hope of remembering

Hm, stupid idea: maybe try .addFirst?

yep, runs as .addFirst too

I'll mess with (.addBefore ,,,) and see what happens

presumably there's somewhere in the chain that makes sense

gotta step away from the keyboard for a while but thanks so much ... this is big progress

No problem

Try dancing around "request-handler", not "http-server", my initial suggestion was wrong

suppose I could just doseq on (.names pipeline) and stick it between all of them 🙃

ok now I'm late for my thing but TOTALLY WORTH IT ... (.addBefore pipeline "request-handler" "scrubber" (make-scrubber-2)) worked and makes the unit test pass 🎉

Yeah, this makes sense now... I confused http-server and request-handler. But think of the fascinating discoveries we made!

I somehow went 8+ years of clojure without even knowing proxy existed and now this is twice in two months

It really isn't needed that often, so no wonder. And it's quite clunky and buggy too, with its proxy-super stuff.

</me files proxy in the same mental drawer that already has cljs this-as>

have a great weekend and thanks again 👋

Cheers, likewise!

Ha. I did consider Server: DIAF/v1.0.0

We regularly observe super hacker bros scanning our servers from some German VPS for vulnerabilities. It's hilarious because our people have given plenty of talks about our stack and we even have a tech blog about it, but no they are brute forcing PHP and ASP known weak points and common misconfigurations...

ha, same ... I asked our pen tester about that since I was seeing a lot of Rails/Express/Django/ASP probing in our error logs, and he said it's just in the big automatic checklist their tools run through. In the debrief he was really enthusiastic about learning new stuff since he'd never seen transit on the wire before.

But now that I think about it it does seem inconsistent to both advise us to turn off the Server: header and to also completely ignore it in practice, because tooling makes ignoring it the path of least resistance.

having it off almost feels like it'll get us fingerprinted as being possibly worth extra attention (which I doubt we are, but still. Bit like the Do Not Track header being used to help fingerprint my browser).

Anyway, whatevs. Unlikely to be the most troubling not-entirely-thought-through thing I encounter today.

btw what happens if you set the Server header to an empty string explicitly in aleph? doesn't netty exclude it?