Fork me on GitHub
#ring
<
2016-08-31
>
zane21:08:43

@weavejester: You frown on attaching, for example, datomic db instances to the request, right?

weavejester21:08:03

@zane Right, I think it’s better to use a closure

zane21:08:19

Closing around the connection?

weavejester21:08:50

Around the handler, so you have a function like...

weavejester21:08:52

(defn endpoint [db]
  (fn [request]
    …))

zane21:08:57

Right, totally.

zane21:08:08

But middleware gets assembled at application startup…

zane21:08:22

I suppose that doesn't have to be the case.

zane21:08:09

Using your example, are you suggesting that we build a new endpoint instance on each request?

zane21:08:30

(Because each request wants to have an up-to-date db.)

weavejester21:08:57

No, you run the endpoint function when your system starts up, and that gives you your handler that you can feed into your adapter.

weavejester21:08:18

Presumably the database is something that you know at startup.

zane21:08:40

Not with datomic.

zane21:08:49

Since "databases" values are essentially snapshots.

weavejester21:08:11

In the case of datomic, substitute “database” for “connection"

zane21:08:13

We could have endpoint take the database connection instead.

weavejester21:08:35

The database should be derived from the connection on each request.

weavejester21:08:52

But you don’t need to attach that to the request...

zane21:08:58

We're trying to see if we can reduce the number of functions that take connections because testing against database instances is easier than testing against connections.

zane21:08:26

So I'm wondering if it would work to have a middleware that basically does this:

weavejester21:08:39

I see where you’re going, I think.

zane21:08:46

(defn endpoint [conn]
  (fn [request]
    (let [db (d/db conn)
          rest-of-handlers (-> (some-handler db)
                               (some-middleware db)
                               (some-other-middleware db))]
      (rest-of-handlers request))))

zane21:08:53

^ Is that bananas?

zane21:08:30

Also, thank you for being so accessible!

zane21:08:56

@annataberski and I were just talking about how amazing your response time was to our question. 😄

weavejester21:08:17

You were probably just lucky! I’m not available all the time 🙂

zane21:08:02

That hasn't been our experience. 😉

weavejester21:08:25

I see the problem you have. With other databases I’d be inclined to use a protocol as a form of encapsulation.

weavejester21:08:23

I think the same idea could be used here. So something like...

weavejester21:08:12

(defprotocol FooDatabase
  (get-foo [abstract-db id]))

(extend-type datomic.Connection   ;; or a component record
  (get-foo [conn id]
    (get-datomic-foo (db conn) id)))

(defn endpoint [datomic]
  (GET “/foo/:id” [id]
    (get-foo datomic id)))

weavejester21:08:01

Datomic’s immutable DBs are really useful, but since you usually need to perform writes as well, they’re not a complete abstraction.

weavejester21:08:42

Whereas if you wrap the datomic connection, or a containing component, in a protocol, then you have a line of separation.

weavejester21:08:57

So you can test the HTTP functions with a mock of the protocol

weavejester21:08:14

And you can test functions like get-datomic-foo individually

weavejester21:08:34

And you can test the datomic protocol implementation as well.

weavejester21:08:38

My general feeling is that it’s a good idea to wrap I/O-like operations in a protocol, both as a way of improving testing, and as a formal border between your functional internal code and the not-so-functional outside.

zane21:08:24

I see, I see.

zane21:08:34

So you're saying that we should coerce the connections into databases dynamically if necessary?

zane21:08:55

The problem with that is that each middleware will get a different database instance, right?

weavejester21:08:19

You’re doing something with the database inside the middleware itself?

zane21:08:45

Validation middleware, for example, needs the database to know how to validate.

weavejester21:08:44

In which case you might need to add the database to the request. to be clear, I’m not against adding additional information to the request map, but it does represent additional complexity.

zane21:08:14

I didn't like adding it to the request map because it doesn't feel like it's part of the request.

zane21:08:22

This feels worse to you?

weavejester21:08:26

Right, that’s my feeling as well 🙂

weavejester21:08:50

My inclination wouldn’t be to put validation in middleware

weavejester21:08:14

Since I tend to approach web apps with the idea of “How to I throw away information as quickly as possible”.

weavejester21:08:25

So a request map comes in, but we typically don’t need all that info

weavejester21:08:34

Or it can be reduced down

weavejester21:08:54

The sooner we can do that, the less data we have to deal with, and the simpler our software can be.

zane21:08:16

Oh, interesting.

zane21:08:43

So you're saying we should put validation in a simple function (rather than middleware) and flatten our middleware down?

zane21:08:42

Hmm. Does that mean that the request map should be paired down as you go? Or does it even mean that the value flowing through the middleware will cease to be a request map after a certain point?

zane21:08:48

Just want to make sure I'm understanding.

weavejester21:08:55

My feeling is that it’s useful for the request map to be a request map, in that all the information inside tends to be connected. Having a lobotomised request map seems like you could be running into type problems; something that looks like a request but is missing data.

weavejester21:08:20

But once you no longer need the request map, I think it makes sense to throw away all the information you don’t need.

zane21:08:21

Super helpful.

zane21:08:50

So we could have another chain of handlers inside the bottom-most handler that operates on something other than the request map (and closes over the database).

weavejester21:08:51

I think I’d need to think about this, and it really depends on what you’re doing! Have you taken a look at Duct?

zane21:08:03

Only a little.

zane21:08:25

> really depends on what you’re doing Happy to add detail wherever it would be helpful!

weavejester21:08:45

So… lets say you’re getting some data in. You need to validate it, and if it’s okay you put it in the database, and return some result. To me that feels like a polymorphic function. Or something that calls polymorphic functions.

zane21:08:09

Polymorphic on what?

zane21:08:17

I guess I need more detail.

weavejester21:08:00

On the type of the data you’re receiving. Or… since you’re using Datomic, maybe you just treat each attribute as individually validated.

weavejester21:08:30

So… let’s say you’re handling user creation

weavejester21:08:09

You might have three values: :user/email, :user/password, :user/retype-password

weavejester21:08:16

Maybe you have a validation function like (validate map key value)

weavejester21:08:28

Maybe a multimethod

weavejester21:08:18

(defmethod validate :user/password [m k v]
  (and (string? v) (= v (:user/retype-password m))))

weavejester21:08:35

Or something like clojure.spec, schema, or whatever you happen to use

weavejester21:08:56

So that’s polymorphic over the attribute

zane21:08:10

In our particular case the validation is dependent on the state of the database.

weavejester21:08:42

Yeah, so add db as an extra argument in the validation function.

zane21:08:47

For example, required attributes for each "model" are expressed via a datomic attribute attached to the entity for the attribute that is required. (Hope that makes sense.)

zane21:08:49

Makes sense.

zane21:08:01

So what you're saying is, request middleware isn't appropriate here.

zane21:08:03

Just use functions.

weavejester21:08:05

Then once you have your piece, you can validate the whole data

weavejester21:08:24

I think so… I tend to look at functions and ask “What does this function need to know?”

zane21:08:33

That's a good principle. 😄

zane21:08:51

I think in our early design of this system we overused request middleware. We're coming around, but it's taking a while.

weavejester21:08:30

I think in Clojure it’s unfortunately easy to make things complex.

zane21:08:45

That's ironic given Rich's goals for the language. 😄

weavejester21:08:51

Clojure has a lot of tools for dealing with complexity and encouraging simplicity.

weavejester21:08:03

But it doesn’t mandate these tools.

weavejester21:08:35

There’s no strict type system, framework, or whatever to enforce the idea of keeping things simple.

weavejester21:08:11

And because Clojure programming is different from OOP, it’s easy to layer in complexity without realizing it.

weavejester21:08:41

Since OOP takes a defensive approach to complexity. You have classes and objects that setup walls to contain complexity and mutable state.

weavejester21:08:18

With Clojure, at least to me, it feels like you have to go on the offensive. Be very aggressive and eliminate instead of contain.

weavejester21:08:46

So OOP says “let’s contain mutable state in objects”, and Clojure says, “let’s eliminate mutable state where possible”, as one example.

weavejester21:08:43

So I think my approach is to ask what a function needs, and whether its needs can be simplified. For instance, an immutable database is simpler than a raw connection.

weavejester21:08:26

So I’m in favour of your approach of trying to make as many things use databases over connections. But ideally you’d want to do it without drawing in the request, if possible.

weavejester21:08:28

Not sure if that’s any help 🙂