Fork me on GitHub
#clojure-europe
<
2020-10-25
>
dominicm06:10:07

@raymcdermott instead of DI you just def things, you mean?

dominicm06:10:50

How do you stop them, to reclaim the port?

genRaiy09:10:04

I don’t care about that. Why should I

genRaiy09:10:57

My other library - going the opposite direction - is satisfactory

genRaiy09:10:19

It uses SAT algorithms and constraints to produce maps that have been made with the assurance that the system is feasible

genRaiy09:10:45

My serious point is that DI often introduces time into a system through the back door

genRaiy09:10:48

Our programs don’t control the world and we just need some maps to understand how to talk to the outside world

otfrom09:10:46

How do you create an environment to not care about it then?

genRaiy09:10:36

Terraform

genRaiy09:10:05

DI mixes things up in a confusing manner imho

genRaiy09:10:16

Test fixtures can still have start / stop semantics of course using some data in a map

dominicm11:10:47

@raymcdermott RE port reclaim, I'm thinking if you do this:

(def db (make-connection))
(def srv (jetty/run-jetty (make-handler db) {:port 3000}))
and then you change make-handler , how do you solve that situation?

dominicm11:10:55

(same sort of thing applies to tests though)

dominicm11:10:23

None of the Clojure DI libraries really do DI anyway, they just do two things: 1) Toposort a graph 2) Run that serial set of actions, allowing for partial collected results to be collected for, e.g. shutdown on failure. I don't think that does anything like: > My serious point is that DI often introduces time into a system through the back door

borkdude11:10:24

use #'make-handler. also you should probably defonce these things in order to not lose the references.

dominicm11:10:35

@borkdude That doesn't work here :)

dominicm11:10:10

That's fine for:

(def srv (jetty/run-jetty #'make-handler {:port 3000}))
But not for a function which returns a handler (e.g. it uses a closure for the db)

borkdude11:10:16

yeah you're right. it needs start/stop and I like component for this

borkdude11:10:02

we do a lot of useful stuff in our start/stop, like flushing redis caches etc

borkdude11:10:42

but we also have a separate function for this: (clear-cache [system ...])

dominicm11:10:52

Your def solution solves the serial set of actions with partial results. And you have a human toposort rather than a computer (which I don't have a problem with, think that's great!).

borkdude11:10:19

The def solution is basically just the mount solution without any other benefits

👍 6
genRaiy11:10:36

I don’t need libs with benefits ;-)

genRaiy11:10:37

some of the other problems with Clojure DI libs esp component is that it’s viral

genRaiy11:10:20

I know that they’re not all like that but that part is unacceptable to me

borkdude11:10:26

@raymcdermott In our system, a lot of functions at the edges (those hooked to routes) usually have the signature: (fn [system user ...]). system is just a map we can use to pluck out the components we need for whatever parts. None of those functions know about component.

dominicm11:10:04

@raymcdermott can you explain what you mean by viral?

genRaiy11:10:40

It’s a whole system rather than something at the edge

genRaiy11:10:29

So I’m with @borkdude on the general approach to limit exposure

dominicm11:10:05

@raymcdermott oh, you're referring to how with component records tend to propagate throughout your code?

dominicm11:10:03

Integrant and clip both avoid that problem by using values instead of objects with methods. Methods are called on the returned value.

dominicm11:10:29

Oops, I missed that.

genRaiy11:10:40

Why do you need to understand the topology?

genRaiy11:10:43

For me, that’s not interesting if you accept that one or more services might not be available.

dominicm11:10:54

You mean the dependency ordering? Well, if you use a graph approach, you don't. If you use a def approach, so that you can def them in the right order.

dominicm11:10:46

Hmm, system frameworks and service availability doesn't seem related?

genRaiy11:10:14

Question is must DB start before web server...

genRaiy11:10:03

If that’s a criterion they are related and they shouldn’t be imho

dominicm11:10:16

A reference to the db must exist before the web server can be created, doesn't necessarily need to be up.

borkdude11:10:28

@dominicm Why? One can also just use a global connection pool + global db config? Devil's advocate here

dominicm11:10:36

@borkdude those are both references. The web server needs a handler to run on request. The handler needs some way to look up the database when it's called (it's behavior on failure is up to it).

dominicm11:10:58

You could do a "lazy" component which returned things on deref if you wanted

borkdude12:10:10

@dominicm > The handler needs some way to look up the database when it's called This isn't an argument against the global def approach though

dominicm12:10:19

@borkdude nope. But the question is why do you care about ordering. We don't, except that we need an order to our references.

dominicm12:10:31

(def db {})
(def handler (mk-handler db))
You can't reorder statements.

borkdude12:10:51

namespaces already take care of that and then you end up with something like mount. which has pros and cons (I'm not a fan). yeah, I guess it basically comes down to taste

dominicm12:10:35

Right. But I think that's a separate discussion to whether you need a topology or not.

genRaiy12:10:05

As long as there’s no expectation that things are available, it works for me

dominicm12:10:27

Admittedly I can't think of a time for the db case where that's true, but I assume I could configure hikari to behave that way.

borkdude12:10:54

Maybe everything. The ultimate Clojure way of life.

3
dominicm12:10:02

Maybe not 😛

borkdude12:10:14

maybe everything is the same as maybe not right?

borkdude12:10:52

Eh, no database? Sure, we'll write to a .csv today

dominicm12:10:41

namespace vs map reification has these points: 1. Namespaces are already a tool you know how to use to incrementally build up resumable conditions that might fail. It's easy to eval something when you're part of the way through: 2. Copying, it's very hard to copy a namespace. Especially as you can't do things like merge into it, or update it, etc. So very hard to create deviations for things like testing or production. 3. Some tools discourage bad practices, some rely on discipline. Mount falls into the discipline category, with the whole "globally available, but pls don't use" vs component's "you have to be explicitly given things, and you have to ask for those things". The latter makes it trickier to do something "incorrect" by reaching globally for a db. This matters so you can "swap out" the database, e.g. during development, or when testing against prod from a local database, etc.

borkdude12:10:33

Maybe swapping out things in tests is also an anti-pattern. I usually don't go there and write tests against a real running environment

borkdude12:10:14

And if you need to swap out: make things arguments and pass different arguments

3
borkdude12:10:39

opinions! :P

dominicm12:10:19

Sure, it seems integration tests are variable in utility. Different projects, different needs.

dominicm12:10:21

I definitely load extra components in dev, for example components for compiling sass or cljs (but that's an anti pattern too).

borkdude12:10:15

We do this with babashka

borkdude12:10:56

script/dev.clj:

#!/usr/bin/env bb

(ns dev
  (:require [babashka.process :refer [$ destroy-tree *defaults*]]))

(alter-var-root #'*defaults* assoc
                :out :inherit
                :err :inherit
                :shutdown destroy-tree)

(defn cljs []
  ($ "./clojure" "-A:frontend:cljs/dev"))

(defn less []
  ($ "./clojure" "-A:frontend:less/dev"))

(defn clojure []
  ^{:inherit true} ($ "./boot" "dev"))

(cljs)
(less)
(-> @(clojure) :exit (System/exit))
This very much resembles our old boot tasks, but now they are just factored out into a script

dominicm12:10:10

I guess the real question is whether you should use your system library or not for tests. Whether you should is really dependent on whether your test is of a function or handler (unit, but with state, whatever that's called?). If you're testing that "on login, counter is updated", but the counter update might be async based on a job queue. So the job queue is part of the system, but you might not need to know how it's implemented as part of the system. You might want to allow a function's list of arguments to grow, e.g. To later add a job queue without having to rewrite all your tests.

borkdude12:10:55

Yeah, at my current job we avoid testing at this implementation-specific level

dominicm12:10:15

Don't take that as a real stance, but I can see how someone might want to do those things.

borkdude12:10:41

We either test pure functions or systems as a whole

dominicm12:10:05

Sometimes people use mocks, and they test that jobs end up on queues, or calls are made to services, etc. I don't personal see the value in testing those particular things, but I think it's great that you can. Why not give someone the choice?

dominicm12:10:27

Maybe I should try and take these and turn them into a ramble of a blog. I have loads of design notes from clip which might have something to add.

borkdude12:10:40

I would read that

dominicm13:10:59

The risk would be making a new scar in this space in the clojure community. We had this discussion, it was respectful, a bit confusing, and tiring. Although perhaps we're at the point where clarity is useful.

genRaiy13:10:12

I think the community can take your brave words :)

otfrom13:10:09

I think the trade offs a team is willing to make will depend on their circumstances

otfrom13:10:57

I don't think any of these options are a clear winner to use all the time

dominicm13:10:45

As long as you mean "in a server application" by "all the time", then for me the "component derivatives" are the clear winners :)

dominicm13:10:55

some other shit

dominicm13:10:13

(defonce srv (make-srv db))
(defonce db {})
Does work if you run it multiple times :p

otfrom14:10:06

I suppose as I'm running a batch job it is less of an issue for me

otfrom14:10:49

And my io is all file based

otfrom14:10:14

So a map in a namespace probably works for me

otfrom14:10:30

Ala chapter 6 in Clojure Applied

otfrom14:10:53

But a bit simpler as I don't have quite the same complexity of problems

otfrom14:10:25

I just need to create and wire together my core.async things before pushing the data through

otfrom14:10:35

So having functions I pass in and out channels to should work for me and I'll create all the channels in an assembling namespace