Fork me on GitHub

Video embedded in tweet


It would be interesting to have a library that read http requests from a stream or something. Rather than having a framework that calls you, we could stick to a composing library.


@dominicm that's what we do - streams of op-descriptions, which get mapped to streams of promise<result>, which get transformed. to streams of results.. and reduceed... although i don't think we have a generic data description of an http request, just of higher level ops which get implemented with http requests


I don't think I was clear. A stream of bytes. For parsing. I'm thinking about how you could unframeworkify a web server.


ah, see what you mean - so to impleemnt the kind of thing that #yada does with mime-multipart responses ?


Probably :) I've never tracked that closely.


umm. not multipart responses - multipart requests


But I was thinking also about managing the connection "myself". As in, I would get a connection, make a stream on incoming request, and hand it off to my parser.


iirc you can already get that from aleph, although it's a stream of netty ByteBuffers rather than a stream of bytes - and yada uses that to do the mime/multipart POST body parsing... although it then presents a callback-based api which wasn't so nice to work with


That's too late in the chain for what I'm thinking. I want to make the port binding myself, maybe even managing connections myself.


This isn't particularly in response to a pragmatic requirement, just a mental exercise at inverting the status quo.


you want to cut netty out and get straight down to the socket buffers ?


Thinking back to when I did networking in C I suppose. I didn't write functions that would magically be called later. I called listen(), accept(), then I did stuff with that socket.


but if you use the select api you write callbacks don't you ?


Then my code operates on the abstraction of socket, rather than "will magically be called later".


Nope. It's more like a case statement.


Think of alts in core.async


C doesn't really do callbacks in an api. All threading is manual.


huh, TIL, i'll have to go think about that... i've not done async socket programming in C


You have to do the async yourself. Delegating to a threadpool or whatever. In clojure you could use core async directly actually. Which is particularly nice as it has a thread pool and green threads, and it has no knowledge about networking at all. Perfect composition target in its design.


ah, ok - so select itself can block the calling thread... i was missing that


but it's pretty obvs when i think about it, otherwise the kennel would be having to get involved in user-space threads


Yeah. So you quickly as possible process what you got and then delegate the task. I have a suspicion you can have multiple things calling accept, but I don't know that. Then you could probably spend longer delegating.


Interestingly, this could lead to some optimizations. As you parse the request, you will find the :path long before the body, you could immediately acknowledge you don't recognize it and terminate further processing. Interesting idea.


Years ago I played with a similar idea… It was back when Microsoft had just released RX, and I was curious about being able to model both the events from the parser and protocol layer. At the time I really liked marble diagrams for describing this stuff; but it was prior to core.async/manifold etc… so I ended up implementing my own poor-mans library. There was a similar effort at the time by Stuart Sierra: Which was far better… anyway, I didn’t get very far… and I quickly abandoned the whole thing largely because I got bored, and realised that it’d mean reinventing the whole HTTP stack.


a stream of byte buffers does seem like a good api for this

Laurence Chen16:11:27

Hello everyone, I am a Clojurian from Taiwan, and I will be at London around 12/1~12/5. I planned to attend ClojureX, but it was cancelled. I would be like to meet some London Clojurians around 12/1~12/5. If anyone have free time to meet, feel free to contact me. My blog: My twitter:


Ah welcome to London Laurence. Are you aware of the #clojurex group which is attempting to resurrect the conference potentially on the same days?

Laurence Chen16:11:02

Yes, I just finished the revival survey form.


I think we're very close to securing a venue, just needs confirming Monday/Tuesday


There are moves afoot to have a 1 day informal "conference", like a phoniex, on the 3rd


We're confident that something will happen, so you're welcome to come along!