Fork me on GitHub
#yada
<
2016-10-08
>
ericnormand02:10:22

hey @malcolmsparks if you're still there

ericnormand02:10:53

I'm wondering about security and how you would protect :get access to a resource for certain users

ericnormand02:10:01

say, only the owner of that resource

ericnormand02:10:45

you've authenticated a request (let's say with a cookie) but then you have to check whether that particular user has access to that particular resource

ericnormand02:10:34

does that go in :allow-methods or :roles/methods?

malcolmsparks09:10:57

hi @ericnormand - I'm in GMT+0100 and was sleeping, been a hectic week with our #XT16 conf!

malcolmsparks09:10:52

yada, like many other web libraries/frameworks, separates authentication and authorization into separate steps. Each step is dealt with by a separate interceptor (yada's approach to modularity), and these are in yada.security.

malcolmsparks09:10:37

Unlike many other web libraries/frameworks, authentication only determines the veracity/credability of the claims made in the request - for instance, with Basic auth, is it the correct password? for JWT token auth, is the signature cryptographically valid?

malcolmsparks09:10:24

Unlike Liberator (and others) the authentication step does not typically return an error like 401 if there is no password/token. It can however, invite the user-agent to provide authentication credentials in the WWW-Authenticate response header.

malcolmsparks09:10:09

So the authentication step is only about authentication. Liberator's authorized? decision point also did authentication, and it was rather confusing to explain to people what was going on, and that a 401 Not Authorized actually meant Not Authenticated.

malcolmsparks09:10:26

yada is based on the substantially cleaned up and disambiguated HTTP RFCs starting with 7230 onwards, where authentication (checking the veracity of the claims) and authorization (determining whether the user can access the resource) and entirely separate.

malcolmsparks09:10:27

In yada, it is the authorization step that produces 200, 401 and 403, depending on what's in the request. You can use yada's only built-in role-based authorizer, or plugin your own. Authorization is done via the yada.authorization/validate multi-method which dispatches on the value of the authorization :scheme.

malcolmsparks09:10:20

Critically, authentication is done before a resources properties (meta-data) are loaded (perhaps involving a database trip). If you're only concerned about checking the veracity/truthfulness of a request, you don't need to know anything about the resource involved.

malcolmsparks09:10:35

Then the resource's properties are loaded (which could involve loading certain attributes about the resource, enabling attribute-based access control ABAC schemes) - https://en.wikipedia.org/wiki/Attribute-Based_Access_Control -

malcolmsparks09:10:15

Then the authorization step is called, which makes the final determination whether a user is allowed to access the resource, based on the established reliable claims in the request, and possibly the properties of the resource.

malcolmsparks10:10:30

Finally, to clear up confusion about when yada emits 200, 401 or 403, my preferred analogy is the passport. A countries passport office does the authentication step (you provide it with evidence of your identity, and it creates the passport). When you arrive at the border between 2 countries there are various outcomes. You might not need a passport to cross the border (200), you might have brought your passport with you anyway but not needed it (200), you might have needed it but forgot to bring it with you (401), you might have brought it with you but it doesn't have the required visa stamp (403). The 403 is useful because it says 'there's no point in coming back here with the same passport, you're still not entering this country', whereas the 401 means 'try bringing your passport with you next time?'

malcolmsparks10:10:21

The upside of this interpretation is that it entirely aligns with the HTTP specs from a terminology perspective. Authentication means authentication, Authorization means authorization.

malcolmsparks10:10:03

However, be warned, there are subtle differences between yada's approach and others' e.g. https://leastprivilege.com/2014/10/02/401-vs-403/

malcolmsparks10:10:00

(Of course, I think I'm correct and that others are wrong, wrt. what the HTTP authors intended, but that's a matter of opinion!)

malcolmsparks10:10:38

Oh, and don't be confused about :allow-methods, that is a CORS thing.

malcolmsparks10:10:14

Oh and also, yada's built-in authorization scheme does allow for method-level role-based authorization, if that's what you're looking for.

malcolmsparks10:10:40

Sorry for these lengthy explanations in slack, I will collate and produce a blog article from them

ericnormand17:10:11

thanks for the clear and lengthy explanation

ericnormand17:10:19

I'm totally on board

ericnormand17:10:34

what I'm confused about is what keys to use and where to put them

ericnormand17:10:01

I've got a person authenticated already

ericnormand17:10:27

but to check if user abc is authorized to GET document at /doc/1234, I need to look up in the database in the access control table if abc owns 1234.

ericnormand17:10:34

where does this logic go?

ericnormand17:10:39

is that a "property"?

malcolmsparks17:10:57

yes, one of the properties would be the 'owner' in this case. You add a :properties entry with a value that is a function that takes the request context as the parameter. The docs are a bit too brief but they're here: https://juxt.pro/yada/manual/index.html#properties

malcolmsparks17:10:17

your properties function is called after authentication but before authorization

malcolmsparks17:10:23

you can do anything you like in a the properties call back, but the idea is that you might visit a database or something that established the properties of the resource. Depending on what you're doing, you might elect to load the whole resource in this step so you can provide the response without revisiting the database

malcolmsparks17:10:09

also, like any other callback, you can return a future/promise/channel from this function.

ericnormand17:10:26

and then you write a custom authorization scheme to check the owner?

malcolmsparks17:10:58

in this case, the owner is a property of the resource, so you need ABAC rather than RBAC here.

malcolmsparks17:10:28

yada can't really know what you mean by 'owner', so of course you have to provide your own authorization function

malcolmsparks17:10:59

you can find many of the properties in the docs, in the source (especially yada.schema), but a ref guide is on its way

malcolmsparks17:10:55

warning: most yada users end up cloning the yada repo and getting a basic familiarity with the code-base - that helps because the code-base makes a good reference as to what interceptors use what properties.

ericnormand17:10:21

I've already dived in

ericnormand17:10:54

I was trying to figure out how to respond with a Location: and provide a body

malcolmsparks17:10:58

that might sound a very unsatisfactory excuse for poor documentation, but in practice the code for each interceptor is pretty small (around a dozen lines or so) and yada.resource shows which the list of the interceptors that make up the processing chain

ericnormand17:10:07

which led me to find that infinite loop last night

malcolmsparks17:10:29

for POST you can return a java.net.URI and yada will figure out that's the Location and set the status to 201

ericnormand17:10:31

well, if you'd like help documenting, I don't mind adding to it as I learn

malcolmsparks17:10:07

well, great - the docs are in /docs and in asciidoc (which I feel gives more features for rich tech docs)

malcolmsparks17:10:16

would be very welcome!

ericnormand17:10:16

they're very nice

ericnormand17:10:33

not too long, good explanation of motivations

ericnormand17:10:50

I was reading this:

ericnormand17:10:57

> 10.2.2 201 Created > The request has been fulfilled and resulted in a new resource being created. The newly created resource can be referenced by the URI(s) returned in the entity of the response, with the most specific URI for the resource given by a Location header field. The response SHOULD include an entity containing a list of resource characteristics and location(s) from which the user or user agent can choose the one most appropriate. The entity format is specified by the media type given in the Content-Type header field. The origin server MUST create the resource before returning the 201 status code. If the action cannot be carried out immediately, the server SHOULD respond with 202 (Accepted) response instead.

malcolmsparks17:10:00

you can also have more control over the response by returning (:response ctx), perhaps modified. yada knows what (:response ctx) is, and knows the record so doesn't confuse it with a map.

malcolmsparks17:10:16

if you return a map from a yada response function, mostly yada thinks you're trying to return pure data, not a Ring response

ericnormand17:10:26

I was having trouble with that

ericnormand17:10:38

I was trying to figure out how to construct one

ericnormand17:10:45

but I didn't think of grabbing the response out of it

malcolmsparks17:10:07

The general idea is that yada follows the spec. If there's any debate between us about what yada should do in a given situation, the spec. always wins!

malcolmsparks17:10:18

well, that's the idea.

ericnormand17:10:19

but it seemed like adding a clojure map as the :body was not converting correctly to json

malcolmsparks17:10:40

are you telling yada you're producing json?

ericnormand17:10:06

let me try again with this new technique of using the context's :response

malcolmsparks17:10:14

oh - well, the reason is that if you are returning (:response ctx) yada gets out of your way and you're in full control

malcolmsparks17:10:20

so you won't get any coercions

malcolmsparks17:10:42

but if you just return a map, and the content negotiation has establised, say, application/json, then yada will kick in and convert the map to json for you

malcolmsparks17:10:31

I see, if you want to return a body from the POST response (as well as your Location header), you have to do that yourself

malcolmsparks17:10:41

yada's semantics for POST assumes you won't return a body

ericnormand17:10:59

right, but the spec says you SHOULD

malcolmsparks17:10:44

ah yes, you're right, but yada should create that for you.

ericnormand17:10:15

hmm, json coercion seems to be working now

ericnormand17:10:24

must have been late night hacking

ericnormand17:10:32

some weird state I got my code into

malcolmsparks17:10:37

Quite high up on the todo list is reactive negotiation, which is where you get 300 response codes and lists of representations

malcolmsparks17:10:00

when I've added that to yada, it might be a good idea to allow you to return a list of URIs from a POST and yada can then generate the body

ericnormand17:10:51

I'm not familiar with reactive negotiation

malcolmsparks17:10:52

my understanding from the spec. snippet you quoted is that the body of a POST is arbitary (set by the user), it is rather a 'list of resource characteristics and locations'

malcolmsparks17:10:11

so perhaps yada should allow a POST response function to return a list of maps, or something.

ericnormand17:10:13

yeah, totally arbitrary

ericnormand17:10:54

I'm returning a JSON with the id

ericnormand17:10:00

nothing special

ericnormand17:10:07

I should probably put the uri in there as well

malcolmsparks17:10:11

yes, it's funny, few people are familiar with it, there's lots of things in HTTP that aren't used. My design position is that it's all going to get implemented eventually, regardless of who uses it. Maybe folks don't use a feature because it isn't widely implemented rather than it not being useful, I don't know

ericnormand17:10:30

yada is awesome so far, btw

malcolmsparks17:10:37

thanks for that

malcolmsparks17:10:15

I think thought that you're not really meant to return a body from POST (or rather, the actual body is prescribed by the spec.)

ericnormand17:10:17

I made about an hour of lessons for http://purelyfunctional.tv using yada

malcolmsparks17:10:47

normally you POST and return a Location header and a 201 and then the user-agent can follow the Location header to show the resource.

malcolmsparks17:10:49

I usually elect to return 201 because single page apps can follow links like this - if you want old-school behaviour where you generate redirects, that's possible too -you might only do so if you know you returning text/html. If you're returning application/json I don't think the redirect thing is useful

malcolmsparks17:10:28

the ultimate goal behind yada is to make it really simple and easy for beginners to write web APIs that actually work, are standards compliant, and don't require all this manual soldering that people do all the time.

malcolmsparks17:10:46

so educational courses are great -

ericnormand17:10:48

replacing an existing small ring application with yada was pretty smooth

ericnormand17:10:53

took me a few hours

malcolmsparks17:10:28

there's a bit of a learning curve with yada too because it's quite different conceptually from the usual 'write a function that takes a request and returns a response' nature of other web libs

malcolmsparks17:10:15

btw. I've been watching Zach's Strange Loop talk today: https://www.youtube.com/watch?v=_1rh_s1WmRA

malcolmsparks17:10:42

many of the concepts match yada ones, but he's writing in Java 🙂

malcolmsparks17:10:35

@ericnormand thanks for that bidi PR last night btw., I merged it to master and will release in due course

ericnormand17:10:44

one part of the requirements for the app I'm writing is generating documentation

ericnormand17:10:54

so I'm excited to include swagger

malcolmsparks17:10:05

yes, the swagger part is made a lot easier once you have your services full specified in data, because it's really just a data->data transformation

ericnormand17:10:09

I think yada will be a big win in general

lmergen17:10:51

@malcolmsparks i saw that talk shared in #aleph, it’s interesting because i’m working on something similar with yada (i personally call ‘subservices’)

malcolmsparks17:10:14

that's intriguing!

lmergen17:10:29

basically the client part of yada, with monitoring, but also circuit breakers, bulkheads, etc

lmergen17:10:37

and tagging of log messages

lmergen17:10:41

passing along authorization headers

malcolmsparks17:10:07

wow, sounds really cool.

malcolmsparks17:10:37

I need to finish watching the talk, I haven't got to the bulkhead bit yet

lmergen17:10:49

not sure whether that’s actually in the talk, though

lmergen17:10:04

but yeah, it seems to be a missing component for microservices in the clojure ecosystem

malcolmsparks17:10:29

what's a bulkhead then?

malcolmsparks17:10:36

i'm still on the pumpkin bit!

lmergen17:10:53

basically ensuring thread starvation never happens by ensuring there’s exactly 1 thread per component

malcolmsparks17:10:07

it's a lovely talk but Zach needs to slow down, I can't keep up with him 🙂

lmergen17:10:27

it’s a bit technical, so instead of explaining it myself, i’ll refer you to: https://johnragan.wordpress.com/2009/12/08/release-it-stability-patterns-and-best-practices/

malcolmsparks17:10:36

his talks are always so dense with information and wisdom

lmergen17:10:52

you should buy the book

lmergen17:10:40

tl;dr: bulkheads prevent deadlocks in distributed systems where service A is waiting for service B that is waiting for service A again

malcolmsparks17:10:07

this is very close to what I've been thinking about regarding yada supporting 503s and 408s, throttling (which manifold already has a function for), rate-limiting (controllable programmatically), etc.

malcolmsparks17:10:53

> I am 110 pages into Michael Nygard’s “Release It!”. I ignored this book earlier because I pictured some namby, pamby book on release best practices, but this book is absolutely fantastic and fascinating to read!

lmergen17:10:15

yeah, the name is really the worst name ever

malcolmsparks17:10:17

haha, that was my first thought too...

malcolmsparks17:10:38

I know Mike, I'll annoy him about it next time we talk...

lmergen17:10:06

well, either way, i’ll probably release it as opensource in the near future, as a kind of middleware between yada and aleph’s http client

lmergen17:10:19

i’ll keep you informed

malcolmsparks17:10:22

ok, book bought!

malcolmsparks17:10:28

getting it on monday

malcolmsparks17:10:40

I should get an amazon echo, that would have saved me 30 seconds 😉

malcolmsparks17:10:53

(and probably ordered me the wrong book)

lmergen17:10:09

(and shipped it to australia instead)

malcolmsparks17:10:42

thanks - Zach's talk has already reminded me I need to add support for timeouts and 408s. I'm really keen on rate-limiting because something I read on HN the other day about an Apple hack where they didn't rate-limit the 'find my phone' feature

malcolmsparks18:10:02

but reading a book first is probably the best thing to do

lmergen18:10:28

i don’t think yada is the right place to implement rate limiting

malcolmsparks18:10:33

I really love hammock-driven development now, I don't feel guilty spending some time thinking about a design

lmergen18:10:35

it doesn’t really have the appropriate context

malcolmsparks18:10:52

yes, that might be true

malcolmsparks18:10:04

can you expand though?

lmergen18:10:21

well, yada’s context is only the service it rusn

lmergen18:10:35

while in reality, people will likely run a pool of yada servers

malcolmsparks18:10:36

ah, you mean yada's really stateless - like http

malcolmsparks18:10:45

it only has the context of the current request, by design

malcolmsparks18:10:56

ok, got it, nice point

lmergen18:10:10

i think yada should have a facility to send a ‘back off, load balancer!’ response

malcolmsparks18:10:12

but it would be nice to integrate it with something that could do rate-limiting

lmergen18:10:16

that would be the appropriate way

malcolmsparks18:10:41

and I'd love to put something into yada that collected metrics about how long each interceptor was taking to give a comprehensive view of the status of services

malcolmsparks18:10:50

I appreciate it's a hard problem

malcolmsparks18:10:46

I think what I mean is that, as a yada user, I'd like to be able to say (in my resource declaration) that the resource should only allow so many requests per second -I'd like to declare that on a per-resource basis.

lmergen18:10:50

yes, but it’s mostly about finding the correct interface

lmergen18:10:02

all these things can easily be implemented as interceptors

malcolmsparks18:10:23

So if I was returning a static file, I don't really care, but if I'm looking up something in a database, or doing some authorization requiring a network call, I'd like to signal a limit

lmergen18:10:30

ahh right, i guess that’s valid — you should implement it using manifold’s stream rate limiting, probably

malcolmsparks18:10:06

but it would also be nice to do this dynamically, e.g. backpressure if I'm detecting that a call to a remote service is taking longer than usual

lmergen18:10:18

backpressure is more focused towards distributed systems

lmergen18:10:27

(which i think is yada’s sweet spot)

malcolmsparks18:10:43

well, the web is kind of a distributed system 🙂

malcolmsparks18:10:59

and I think the web is good enough an architecture for companies to use internally

malcolmsparks18:10:18

that's kind of been my playbook in 'enterprise' consulting