Fork me on GitHub
#clojure-spec
<
2019-09-19
>
dominicm16:09:45

Has anyone experimented with making specs for stateful things, with the intention of using mocks for those values?

dominicm16:09:57

Thinking in the context of dependency injection somewhat.

Joe Lane16:09:40

@dominicm Do you mean something like

(stest/instrument `invoke-service {:stub #{`invoke-service}})
found at https://clojure.org/guides/spec#_combining_check_and_instrument

dominicm16:09:05

oh wow, that's fantastic!

dominicm16:09:26

oh, wait. That's not quite what I was looking for. But still very interesting in what I'm thinking of :thinking_face:

dominicm16:09:14

that's really neat. I didn't know you could do that. That's actually really handy for what I'm working on.

dominicm16:09:58

I'm actually thinking of the case where you have a function like invoke-service, and you want a dumb wrapper which can figure out what to call it with (from it's registry of "stuff") based on the spec. I'm a bit frustrated with the pattern of passing around a grab-bag of state which inevitably grows into an unrepl-able mess.

djtango09:09:01

instrument actually lets you go even further and completely replace the function

dominicm09:09:33

Yeah, that's great at the repl. I don't want to do that in production though 🙂

djtango09:09:02

ah - sorry, I am still catching up with the discussion though have found the intercepting properties useful at test-time

dominicm09:09:03

I'm a little unsure on that. Parallelizing tests is valuable for speed (until Rich invents his tool for running only the required subset of the tests). I don't know how much I love that idea really.

dominicm09:09:42

But yeah, if you're willing to trade those two properties, it's all good.

djtango09:09:44

mm yeah fair enough. So is your use-case this: use fdef to document what deps your function needs have some kind of wrapper that then knows how to extract and inject only those deps into a calling function?

dominicm09:09:18

Yeah. I'm coming round to the idea that I might be barking up the wrong tree though 🙂 I'm thinking a little more time in the hammock should help me puzzle out the use cases.

dominicm09:09:44

I'm trying to exploit maximum leverage without creating unnecessary verbosity. It's a hard balance 🙂

djtango09:09:21

I guess I can in theory see how you could fetch the fspec then use a generator, to satisfy the s/keys :req portion but feels brittle, if only as I'm not convinced it's an appropriate usage of the tools

dominicm09:09:20

Me neither 🙂

djtango09:09:49

sounds tricky - would be interested to know how you get on

dominicm09:09:58

I'm hoping to publish my results somewhere, I think I'm onto something, I've reduced the initial problem space into something more palatable. But I still have to handle the stateful dependencies part.

Joe Lane16:09:37

I'm not sure I understand the usecase for the dumb wrapper. What is registry of "stuff"?

dominicm16:09:17

I suppose something to the effect of, I have a "system" like: {:db db-conn :some-stateful-service statey} and I want to direct things directly to a function defined like:

(defn add-user
  [db add-user-data] …)
So the "registry" is that first map.

Joe Lane16:09:43

So, in component terms, a "system"

Joe Lane16:09:57

(Or a pedestal context map)

dominicm16:09:35

Yeah. Exactly. But passing around systems is an anti-pattern. And I also can't figure out what's in my pedestal context map half the time, because there's lots of interceptors messing with it, and it's unclear why routeA gets :mongodb and routeB doesn't. (So I'm trying to come at this with a new angle of positional parameters)

Joe Lane16:09:37

So, is the goal here to assert a property about the dumb wrapper but you're trying to determine the best way to inject actually stateful mocks?

dominicm16:09:59

I guess I could write a spec parser for s/cat? And that would let me figure out the arguments in order. Well, actually I was thinking that it would be handy to just start by being able to call add-user by using it's fdef to figure out that it needs a database, and getting one from the system. (and somehow connecting ::db/db as a spec to that)

Joe Lane16:09:12

(= dumb-wrapper add-user)

dominicm16:09:27

yeah, right 🙂

Joe Lane16:09:54

I might be wrong, but it sounds like you want an integration between your DI tool (are you using component?) and the s/select facilities in spec2. If you're using component though, shouldn't that fdef to figure out.... step be happening at system/lifecycle start time?

dominicm16:09:57

But I'd be using this in production, not just for testing. So I would know that this function only takes 1 non-stateful argument (the "event", or maybe "req" for http). So I just need to get the rest of the arguments.

dominicm16:09:08

I'm not using component. The reason being that a system probably ends up with ~50 or so handlers, and the boilerplate involved is quickly tedious. Although maybe I should just use a macro for that 😛

dominicm16:09:24

The way I see it, I have actual stateful things (e.g. db conn) and things that want to use those things.

Joe Lane16:09:04

This is starting to feel more and more like spring style annotation DI vs data oriented component DI. And further away from a problem related to spec.

dominicm16:09:04

A little, yeah. And that's somewhat intentional. I see one of two patterns in this space: 1. create a map with everything in, call it "deps" and hope the function you're passing it to understands it (and there's a bunch of bad patterns which fall out of this). 2. roll entire namespaces into being partial applied with their state available, and passed in as arguments.

dominicm16:09:02

Tbh, I expect there's not a good spec function for figuring this out 🙂 As I have a lot of contextual awareness that spec doesn't have. (e.g. I know it's always a list, and I will always know everything except 1 argument).

Joe Lane16:09:59

I think the 3rd pattern is component/system/mount oriented, where functions are passed positional arguments (not a deps/req/context map) which were determined by the DI tool. I don't think you can reduce that kind of complexity, only choose where to solve it, 1 large place (component) or in every dumb-wrapper by declaring the deps at the callsite. I definitely don't think spec is going to be helpful here.

dominicm16:09:08

I haven't seen 3? what does that look like

dominicm16:09:01

oh, maybe I do know what you mean.

dominicm16:09:17

I guess pattern 3 here is implemented in terms of my listed patterns 1 & 2.

jaihindhreddy19:09:13

I just started solving some basic programming puzzles with Clojure, with the constraint that I want to spec the solutions. I hit a snag right on the first one. Here's the fn I want to spec:

(defn two-sum
  "Returns indices of two elems in ints that add up to sum"
  {::url ""}
  [ints sum]
  (loop [l 0
         r (dec (count ints))]
    (let [s (+ (nth ints l) (nth ints r))]
      (cond (= l r) nil
            (= s sum) [l r]
            (< s sum) (recur (inc l) r)
            :else (recur l (dec r))))))
And here's what I came up with:
(s/fdef two-sum
  :args (s/cat :ints (s/coll-of int? :min-count 2 :kind vector?) :sum int?)
  :ret (s/and (s/tuple (s/and int? #(>= % 0)) pos-int?) (fn [[l r]] (< l r)))
  :fn (s/and
        #(< (-> % :ret second) (-> % :args :ints count))
        #(= (-> % :args :sum)
            (+ (nth (-> % :args :ints) (-> % :ret first))
               (nth (-> % :args :ints) (-> % :ret second))))))

jaihindhreddy19:09:03

The puzzle is to find two (different) indices s.t. the elements in ints in those indices add up to sum. The catch is: there is always exactly one solution. How do I spec my args to get gen right?

jaihindhreddy19:09:11

Unless I use the fn itself in it's spec (or trusted equivalent), I can't seem to get gen. to work.

seancorfield19:09:06

@jaihindhreddy That sounds like an external constraint on the data? An arbitrary vector of 2+ ints and an arbitrary sum aren't going to satisfy that condition so the behavior of two-sum on such data will be...? Undefined? nil?

seancorfield19:09:39

Your :ret spec doesn't account for the function returning nil -- which it clearly can -- so your spec isn't right as it stands.

seancorfield19:09:02

user=> (two-sum (into [] (range 10 20)) 9)
nil
So your :ret spec needs to be s/nilable or use s/or and then your :fn spec needs to accept that (:ret %) can be nil (or should satisfy that complex predicate).

jaihindhreddy19:09:24

It is an external constraint on the data. I thought of using s/nilable there but actually the fn is not supposed to be called with such args and its UB, and I'm trying to get the generation to work in such a way that there exists exactly one answer.

jaihindhreddy19:09:45

I'll probably look into using fmap to generate the sum from the ints. Thanks for your help!

seancorfield20:09:52

And it's also an ascending sequence of ints, yes? (based on the < logic in there)

seancorfield20:09:47

I think what you're attempting is a bit self-defeating: the generator and the :fn spec together are pretty much going to be a re-implementation of the function logic at this point...

seancorfield20:09:14

(i.e., you're over-spec'ing things, IMO)

💯 4
jaihindhreddy20:09:06

Kinda reckoned that myself.

jaihindhreddy20:09:18

The fn returns a pair of indexes into the ints arg, so I'm checking that the second one in the return value is less than the count of ints.

vlaaad20:09:22

noob question: why spec forms sometimes have qualified, and sometimes unqualified symbols?

vlaaad21:09:14

minimal example:

(s/def ::vec vector?)

(:pred (first (::s/problems (s/explain-data ::vec {:a 1}))))
; => clojure.core/vector?

(:pred (first (::s/problems (s/explain-data (s/coll-of vector?) [{:a 1}]))))
; => vector?

Alex Miller (Clojure team)21:09:40

Should always be qualified

Alex Miller (Clojure team)21:09:00

There are some pending patches for this stuff

vlaaad21:09:39

I'm fascinated by spec, sprinkling it a bit here and there to document/enforce contracts where it makes sense... do you think it's the future of strong gradually static type systems?

vlaaad21:09:32

like being able to say something along the lines of

let n: int? = (get-some-int) 
let x: (and int? even?) = n ;; checks for `even?` during casting

Joe Lane21:09:37

I think it will push significantly more boundaries than just "type systems"

Joe Lane21:09:53

I also think the traditional notion of static type systems look very different if you can work in a dynamic system.

Joe Lane21:09:10

But what do I know :man-shrugging:

seancorfield22:09:56

I don't consider it "like" a type system (and I think it's a bit misleading to think of it in those terms).

👍 4
seancorfield22:09:38

We've been using spec in production code very heavily since it first appeared. We don't use it much on functions (which is the only "like types" part of it -- and even that isn't much like a "type system").

seancorfield22:09:12

We mostly use data specs and validation/conformance. We use it to generate (random, conforming) data for example-based tests. We use it for generative testing of some things. We use it via instrument a little bit during dev/test.