Fork me on GitHub
#clojure-uk
<
2019-11-09
>
yogidevbear07:11:29

Video embedded in tweet

dominicm08:11:41

It would be interesting to have a library that read http requests from a stream or something. Rather than having a framework that calls you, we could stick to a composing library.

mccraigmccraig10:11:35

@dominicm that's what we do - streams of op-descriptions, which get mapped to streams of promise<result>, which get transformed. to streams of results.. and reduceed... although i don't think we have a generic data description of an http request, just of higher level ops which get implemented with http requests

dominicm10:11:38

I don't think I was clear. A stream of bytes. For parsing. I'm thinking about how you could unframeworkify a web server.

mccraigmccraig11:11:32

ah, see what you mean - so to impleemnt the kind of thing that #yada does with mime-multipart responses ?

dominicm11:11:55

Probably :) I've never tracked that closely.

mccraigmccraig11:11:08

umm. not multipart responses - multipart requests

dominicm11:11:46

But I was thinking also about managing the connection "myself". As in, I would get a connection, make a stream on incoming request, and hand it off to my parser.

mccraigmccraig11:11:41

iirc you can already get that from aleph, although it's a stream of netty ByteBuffers rather than a stream of bytes - and yada uses that to do the mime/multipart POST body parsing... although it then presents a callback-based api which wasn't so nice to work with

dominicm11:11:28

That's too late in the chain for what I'm thinking. I want to make the port binding myself, maybe even managing connections myself.

dominicm11:11:56

This isn't particularly in response to a pragmatic requirement, just a mental exercise at inverting the status quo.

mccraigmccraig11:11:07

you want to cut netty out and get straight down to the socket buffers ?

dominicm11:11:48

Thinking back to when I did networking in C I suppose. I didn't write functions that would magically be called later. I called listen(), accept(), then I did stuff with that socket.

mccraigmccraig11:11:21

but if you use the select api you write callbacks don't you ?

dominicm11:11:51

Then my code operates on the abstraction of socket, rather than "will magically be called later".

dominicm11:11:00

Nope. It's more like a case statement.

dominicm11:11:09

Think of alts in core.async

dominicm11:11:30

C doesn't really do callbacks in an api. All threading is manual.

mccraigmccraig11:11:40

huh, TIL, i'll have to go think about that... i've not done async socket programming in C

dominicm11:11:31

You have to do the async yourself. Delegating to a threadpool or whatever. In clojure you could use core async directly actually. Which is particularly nice as it has a thread pool and green threads, and it has no knowledge about networking at all. Perfect composition target in its design.

mccraigmccraig11:11:42

ah, ok - so select itself can block the calling thread... i was missing that

mccraigmccraig11:11:02

but it's pretty obvs when i think about it, otherwise the kennel would be having to get involved in user-space threads

dominicm11:11:24

Yeah. So you quickly as possible process what you got and then delegate the task. I have a suspicion you can have multiple things calling accept, but I don't know that. Then you could probably spend longer delegating.

dominicm11:11:57

Interestingly, this could lead to some optimizations. As you parse the request, you will find the :path long before the body, you could immediately acknowledge you don't recognize it and terminate further processing. Interesting idea.

rickmoynihan10:11:38

Years ago I played with a similar idea… It was back when Microsoft had just released RX, and I was curious about being able to model both the events from the parser and protocol layer. At the time I really liked marble diagrams for describing this stuff; but it was prior to core.async/manifold etc… so I ended up implementing my own poor-mans library. There was a similar effort at the time by Stuart Sierra: https://github.com/stuartsierra/cljque Which was far better… anyway, I didn’t get very far… and I quickly abandoned the whole thing largely because I got bored, and realised that it’d mean reinventing the whole HTTP stack.

mccraigmccraig11:11:16

a stream of byte buffers does seem like a good api for this

Laurence Chen16:11:27

Hello everyone, I am a Clojurian from Taiwan, and I will be at London around 12/1~12/5. I planned to attend ClojureX, but it was cancelled. I would be like to meet some London Clojurians around 12/1~12/5. If anyone have free time to meet, feel free to contact me. My blog: https://humorless.github.io/ My twitter: https://twitter.com/humorless

cddr16:11:03

Ah welcome to London Laurence. Are you aware of the #clojurex group which is attempting to resurrect the conference potentially on the same days?

Laurence Chen16:11:02

Yes, I just finished the revival survey form.

dharrigan17:11:58

I think we're very close to securing a venue, just needs confirming Monday/Tuesday

dharrigan16:11:27

There are moves afoot to have a 1 day informal "conference", like a phoniex, on the 3rd

dharrigan16:11:46

We're confident that something will happen, so you're welcome to come along!