@malcolmsparks: nice change for multipart form data

I have a question though: is it intentional that process-request-body only handles multipart/form-data and application/x-www-form-urlencoded?

no. it should be open

Where should you now coerce the schema for json, edn,...?

but I've only just reached the point where multipart works so I committed!

but haven't released yet

okay

i'm closing this pull request, ok? https://github.com/juxt/yada/pull/34

I had to remove the original body protocol

yes, the reason I couldn't merge that PR was that I needed to finish multi-part first, because body processing is now asynchronous

because yada must support large bodies, as well as small ones

the question is, how to support json/edn etc. again

and make things open

no, I totally understand

it should be a simple matter of extracting out the protocol again and making it open for extension

although obviously json and edn support should be built in

body coercion is still part of multipart/form-data processing

so it shouldn't be too much work

I'll work on this tonight and push a few more commits

i'll stay on my development branch until I can extend body coercion indeed

I'd like to test that it works with large streams, perhaps your video upload use-case

yes, forks are good

yes, i'm very interested in testing that!

I'll need to see first how you can asynchronously upload data into mongo's gridfs

(if it can be done with the java driver)

it doesn't make sense for yada to support all these http things, but then fall over at scale - and having bits of yada that require synchronous thread blocking is going to lead to a bottleneck

re. mongo gridfs - can you send me a link to the API

I think that wouldn't work well with aleph's model right?

i'm using monger http://clojuremongodb.info/

great thanks

it wraps the Java client

I didn't look into detail yet, but saw no async api though

there's an important protocol in multipart.clj called PartReceiver - the idea is you can hook into yada's body processing to send the individual parts (including partial pieces, if the part is large), to your own 'receiver', that could send these parts on to mongodb

so it's a pass through

I can see that we need a similar protocol for non-multipart bodies, where you want raw access to the incoming buffers

yes, in my use case there's no multipart, just raw data in the body

it looks like they're working on it right now https://jira.mongodb.org/browse/JAVA-1282

yes, async may be the future, but it's not always the present!

perhaps the (far) future is super lightweight threads on the jvm, then we'd be back to synchronous models - that might be a long way off though

https://clojurians.slack.com/archives/yada/p1447677064000063 I guess I'll need to re-submit those PRs then

@malcolmsparks: did you say there is no extension point now for that?

@imre: hi - yes, but I meant to say that this was a temporary state, and that a new extension point will be added - I've fetched your branches into my local repo and will attempt to bring your json/edn/transit work into the new model

@malcolmsparks: great stuff. If you do have trouble, leave them there and I'll update them once the new ext point is out