pedestal

rolt 2024-11-04T11:20:39.489859Z

it looks possible by using a custom configure, with a configurator that overrides modify-handshake maybe ? Or using a servlet filter ?

rolt 2024-11-04T15:37:04.871069Z

(def configurator
  (proxy [ServerEndpointConfig$Configurator] []
    (modify-handshake [^ServerEndpointConfig sec ^HandshakeRequest req ^jakarta.websocket.HandshakeResponse rep]
      (println (.getHeaders req))))

...
          :configure (fn [^ServerEndpointConfig$Builder sec-builder]
                       (.configurator sec-builder ^ServerEndpointConfig$Configurator configurator))
...
i tried the following without succes 😕 I may be missing somthing

maybenot 2024-11-04T15:37:50.642129Z

Thank you, have looked at configure didn’t found headers, found subprotocols (saw someone used them in the browser, where no headers on ws) but still would be nice to have a simpler way

rolt 2024-11-04T15:40:39.368359Z

how do subprotocols help ?

rolt 2024-11-04T15:41:18.559709Z

you do have access to the query string in an easy way if you want

👍 1
maybenot 2024-11-04T15:43:11.176009Z

You define custom sub protocol where token is part of the definition, then read it and authenticate connection But yeah, it would be much more convenient to just get headers in upgrade

maybenot 2024-11-04T15:41:08.739289Z

also another Q: in the docs related to http://pedestal.io/pedestal/0.6/guides/sse.html a saw core async blocking puts inside demo and also inside heartbeat in the lib. As far as I know pedestal doesn’t use virtual threads, so it’s kind of bad? Or am I missing something? Edit. Most importantly I was missing docs, which helpfully saying that you shouldn’t spin up a long process inside stream-ready callback

✅ 1
rolt 2024-11-04T16:46:28.022499Z

what should be avoided with core async is blocking operations inside a go block, this kind of patterns: (go (let [res (long-http-call)] (>! channel res)))

rolt 2024-11-04T16:49:54.776619Z

it looks like you're right, it is doing a blocking operation in the go block in the lib, I couldn't find the one in in the doc though

rolt 2024-11-04T16:52:20.832399Z

start-dispatch-loop has a go block that calls send-event, which is blocking

rolt 2024-11-04T16:56:11.127059Z

oh wait it's using put!, (and not >!! as I previously thought) so maybe it's ok ?

maybenot 2024-11-04T17:06:52.208839Z

I meant that sample is using >!! inside SSE fn (stream-ready), also most probably we’re inside some thread pool inside jetty. Could >!! saturate jetty thread pool? But maybe I’m completely wrong tho

rolt 2024-11-04T17:13:08.717599Z

no it's not jetty's thread pools. From what I'm reading start-event-stream calls your stream-ready fn in an async/thread block, so it's ok to do blocking operations here. It starts a regular thread from IIRC an unbounded thread pool.

1
rolt 2024-11-04T17:16:24.650179Z

the problem with go block is that the thread pool is really small so it's easy to exhaust it if many go block are running (and not waiting on <! or >! operations) at the same time.

maybenot 2024-11-04T17:16:48.692929Z

Oh, I see, but isn’t it still bad? Like if you stream to 100k clients

rolt 2024-11-04T17:18:41.561599Z

it would be like running 100k future

rolt 2024-11-04T17:19:00.546589Z

so probably too much with default settings

maybenot 2024-11-04T17:19:40.487719Z

I guess it would be better with virtual threads when they will be integrated to core async

rolt 2024-11-04T17:22:52.689289Z

probably because virtual thread should not consume resources when you're doing the Thread/sleep call that's in the doc

rolt 2024-11-04T17:23:48.369769Z

I have no experience with virtual threads

maybenot 2024-11-04T17:23:52.313409Z

Thank you @rolthiolliere ! I think I now kinda understand what’s happening here and why blocking operations are ok Also related to go blocks, if it’s mainly (non blocking) IO probably 8 threads should be enough? If you outsource compute to some other thread, etc?

rolt 2024-11-04T17:40:20.996919Z

I'm not sure how to answer that, there's several things... • go blocks have some overhead, binding variable, calling functions etc is a bit slower because of the macro is transforming the code a bit to capture variables and stuff. So (go (fn-that-hides-stuff) ...) is better that (go (complex inlined code) ...) • only 8 go block can run at the same time (but many many more can wait on an operation with <! , 1 million ?? 10 million ?? I never stress tested it). 8 is usually more that the number of core on the machine so normally it's plenty. • if you do computationally intensive operation, it's better to use threads yes. You would not run 10 million computationally intensive operations at the same time anyway. You probably want to bound yourself the executor thread pool for that and have a tight control on it

👍 1
maybenot 2024-11-04T17:53:23.053199Z

I just imagine a situation where start-event-stream starts a go block and you use <! inside stream-ready with a caution that you shouldn’t really block this fn, I assume it could scale better for multiple concurrent connections

rolt 2024-11-04T18:33:35.772919Z

I think so yes

hlship 2024-11-06T22:27:30.427099Z

I concur that guide is not so great; adding a thread or a properly-written go around the meat of that dotimes loop would still demonstrate the necessary concepts without conflicting with basic core.async guidelines.

🙌 1