Fork me on GitHub
#clojure-spec
<
2016-12-29
>
seako07:12:53

@sophiago carin meier wrote a blog post about using spec to generate code that you might find interesting http://gigasquidsoftware.com/blog/2016/07/18/genetic-programming-with-clojure-dot-spec/

seako07:12:05

also, i would like chime in that i also think it would be very interesting to read about generator combinators and category theory

stephenmhopper14:12:00

I have a spec that I've written with clojure.spec and I can use it to generate valid Clojure data structures. Can I use it to generate invalid structures?

gfredericks14:12:02

generating things in a negative way is hard to do in general

gfredericks14:12:38

I expect clojure.spec makes no attempt to support that, since it's not obvious what strategy it would use

gfredericks14:12:24

you can use unit testing for that, or custom generators to cover more specific cases

gfredericks14:12:51

for a lot of cases you might do fine by using gen/any and filtering out things that match the spec

stephenmhopper14:12:35

I did not know about gen/any, but that should do the trick. I was considering just creating a version of my schema where all required keys are instead marked as optional while also changing the data type / generators for some of the keys, but gen/any is simpler

naomarik15:12:44

are you guys converting database records field names to namespaced keywords?

naomarik15:12:05

or just specing database stuff with :req-un

gfredericks15:12:46

I think converting would be cooler, but it takes more work and I haven't worked on something like that yet anyhow

gfredericks15:12:08

e.g., a namespace per table

naomarik15:12:15

yes this is what i’d like

naomarik15:12:42

i spent some time specing all my stuff like that and while i can just use req-un i would rather see it qualified

naomarik15:12:55

would be great if i didn’t have to rename joined tables either, ie multiple id fields

naomarik16:12:00

there’s anyway to conform a spec given keys with :req-un into the matched qualified keywords?

joshjones16:12:34

I do not think so; do you have a particular use case for this?

seancorfield16:12:57

FWIW clojure.java.jdbc supports producing qualified names via the :qualifier option. Doesn't handle joins tho'.

joshjones16:12:40

You could use s/describe on the spec to get the vector of :req-uns, and then pull the key with the name of the key you want to qualify, grab its namespace, and assoc a “newly created” fully qualified key onto the conformed data

joshjones17:12:05

If you really wanted to. 🙂

naomarik17:12:38

@seancorfield ya saw that, will probably do that for non joined data and use an identifier that will handle select AS statements

joshjones17:12:47

just as an exercise @naomarik :

(defn qualify-keys
  [spec conformed]
  (let [unq-to-qual (as-> (s/describe spec) $
                          (second (drop-while (partial not= :req-un) $))
                          (zipmap (map name $) $)
                          (reduce-kv #(assoc %1 (keyword %2) %3) {} $))]
    (reduce-kv #(assoc (dissoc %1 %2) %3 (get %1 %2))
               conformed unq-to-qual)))

(s/def :ns.one/requnkey string?)
(s/def :ns.two/reqkey string?)
(s/def ::mymap (s/keys :req-un [:ns.one/requnkey] :req [:ns.two/reqkey]))

(s/conform ::mymap {:ns.two/reqkey "required!" :requnkey "not required!"})
=> {:ns.two/reqkey "required!", :requnkey "not required!"}
(qualify-keys ::mymap *1)
=> {:ns.two/reqkey "required!", :ns.one/requnkey "not required!"}

joshjones18:12:39

@seancorfield you just mentioned in the #clojure channel about using with-redefs to stub functions for testing; what’s your take on taking a spec’d function and now being able to do this, versus the with-redefs approach?:

(stest/instrument `my-func {:stub #{`my-func}})

seancorfield18:12:58

@joshjones I haven’t experimented with stubs in clojure.spec yet so I don’t have an opinion.

seancorfield18:12:26

Mostly we’re spec’ing data structures and using s/conform etc — we’re not spec’ing functions much.

joshjones18:12:55

ok — I have not tried it for production-level testing either, but it looks useful:

(defn get-data-from-db [] nil)

(s/fdef get-data-from-db
        :args empty?
        :ret (s/int-in 1 10))

(stest/instrument `get-data-from-db {:stub #{`get-data-from-db}})

(get-data-from-db)
=> 6
(get-data-from-db)
=> 7

seancorfield18:12:31

So it stubs the named function to return generated data per spec rather than actually calling it? Interesting. Wouldn’t work for anything that actually required mock state, rather than just stubbed data, but I can see that being useful sometimes.

seancorfield18:12:29

I haven’t generally found stubs to be as useful as mocks so I don’t know how much I’d use it.

joshjones18:12:35

right, that’s a good point

seancorfield18:12:21

I can see it being useful as you’re developing top-down where you stub functions as placeholders until you actually write them.

seancorfield18:12:59

Right now, I’d actually write the physical stub… with the spec approach you could just get instrument to “write the stub” for you, but then you’d have to re-instrument or un-instrument as you actually wrote each function.

joshjones18:12:07

is there something you typically use for mocking state? or it just depends on your particular application?

roelof18:12:32

How can i check in a spec for a key in a nested map ? so this is valid (s/explain ::objectNumber-list [{:body{:artObjects{:objectNumber "sk-c-5"}}}])

seancorfield18:12:37

Since mocks tend to be pretty tied to your implementation, I generally just hand-roll them.

seancorfield18:12:03

@roelof Please keep working in the #beginners channel on that.

seancorfield18:12:18

You need to think carefully about the elements of that list, is all I’ll say.

seancorfield18:12:15

@joshjones When we mock things, we tend to need to coordinate data across a series of calls — or even just record data across a series of calls — so custom functions that use atoms or refs to track that is our “go to” approach.

seancorfield18:12:04

For example, we mock a payment provider to track calls and amounts, and pass or fail the transaction based on state.

joshjones18:12:54

I guess the amount of data you mock is pretty small though right? i.e., the purpose of the mocking at this stage is not for load-testing, which is done in a completely different way.

joshjones18:12:22

for example, we load test our elasticsearch code using dedicated aws servers, so it’s purely a load test — for testing correctness, etc., small amounts of mock data is used locally to be sure everything works as expected.

seancorfield18:12:26

Mocking is to verify that a function calls some other function(s) in a particular way and/or that a series of calls to the mocked functions returns data in a particular way.

henriklundahl20:12:09

@roelof, you spec each value which is associated with a specific key and then specify which keys should be included in each map. Start with the innermost map(s) and work your way out.