Fork me on GitHub
#yada
<
2015-07-01
>
stijn08:07:52

how do you guys usually deal with very large query parameters?

stijn08:07:19

i’m referring to the talk by David Nolen on sending the pull spec to the REST endpoint instead of having it configured statically in the server

stijn08:07:00

the Datomic REST peer deals with this by requiring a POST request for larger query expressions

stijn08:07:34

but HTTP caching becomes a lot harder then

ordnungswidrig08:07:12

I don’t it becomes harder.

ordnungswidrig08:07:31

First, the URL/URI/IRI in general have a (sane) limit of 64k but that’s up to the implementation

ordnungswidrig08:07:35

Responses to POST requests are perfectly cacheable if you set the respective headers. Additionally conditional requests are supported for POST, too.

stijn08:07:34

because last time I read up on the Apache docs, they would only cache GET requests

stijn08:07:08

section ‘What can be cached?'

stijn08:07:20

3. The request must be a HTTP GET request.

ordnungswidrig08:07:32

well, that’s what apache does.

ordnungswidrig08:07:15

If you encounter problems you can POST the pull spec to /db and be redirected to a resource based on a hash of the spec, e.g. /db/f8e3183d38e6c518

ordnungswidrig08:07:31

the server needs to maintain a short-lived cache for the mapping of the hash to the spec.

ordnungswidrig08:07:55

technically that’s bad conversational state but it gets it done in most cases

stijn08:07:21

seems reasonable

ordnungswidrig08:07:52

RFC7230 has a section on that

ordnungswidrig08:07:01

> Responses to POST requests are only cacheable when they include explicit freshness information (see Section 4.2.1 of [RFC7234]). > However, POST caching is not widely implemented. For cases where an origin server wishes the client to be able to cache the result of a POST in a way that can be reused by a later GET, the origin server MAY send a 200 (OK) response containing the result and a Content-Location header field that has the same value as the POST’s effective request URI (Section 3.1.4.2).

stijn08:07:04

I need to check how native mobile HTTP clients behave in this case, but for most of them you can configure the caching mechanism pretty well

ordnungswidrig08:07:41

besides that, are you investigating on that pull spec over http topic now? I had a intense discussion with David on the usefulness of HTTP2 in general and especially server push for serving responses to pull specs.

ordnungswidrig08:07:52

I want to prove him wrong simple_smile

stijn08:07:16

ha, interesting to hear that simple_smile

stijn08:07:10

well the thing is, it would be nice for the client to be able to specify which data exactly it would receive on a certain screen

ordnungswidrig08:07:45

sure, I guess that http2’s support for fine grained resources would be improve the caching situation

stijn08:07:47

now we’re using some ad-hoc stuff like ?detail=full or whatever, but you can’t cater for every situation obviously

stijn08:07:21

I guess I need to read up on http2 simple_smile

ordnungswidrig08:07:37

http2 allows the server to push responses to the client on it’s own.

stijn08:07:49

another thing we’re doing frequently with datomic is ‘change detection’: client opens a long running connection on a certain resource endpoint, specifies which data it is interested in and only receives updates when changes are made in the database and are in scope of the client’s request

ordnungswidrig08:07:38

yes, david favors a polling loop. the reasons have been left unclear to me unless positive experience of him with that at NYT

ordnungswidrig08:07:49

with server pus, if you send a pull spec, you could get back a bunch of responses, one for the „result list“ and one for each result. Every response has it’s own cache lifetime and modification date.

ordnungswidrig08:07:16

if another search gives a similar result list all the common results do not need to be sent again.

stijn08:07:41

interesting

ordnungswidrig08:07:11

but, YMMV and it needs to be tested.

stijn08:07:40

and how far is HTTP2 from becoming a reality? simple_smile

ordnungswidrig08:07:19

in chrome, netty, jetty, firefox, whatever. simple_smile

ordnungswidrig08:07:31

RFC2616 is obsolete

stijn08:07:43

I’ll need to investigate if our stack is ready (nginx as SSL proxy, retrofit on Android, AFNetworking on iOS)

ordnungswidrig08:07:07

Sounds like quite some work…