Fork me on GitHub
#clojure-spec
<
2018-09-26
>
steveb8n07:09:27

question : I’ve heard about context specific specs quite a few times i.e. same entity but different uses e.g. reading vs inserting a map from/to db. another might be reading entities in a list vs individually. I can guess how these specs might look but was hoping I could learn from looking at others. does anyone know of public codebases with examples of this technique?

jaihindhreddy-duplicate12:09:29

@steveb8n I would be interested to see this too. But what you describe feels like something spec is not meant to do. Spec specs data, not entities or identities. ::entity-while-reading, ::entity-while-inserting might be better as two different specs. Maybe metosin/spec-tools can do this.

steveb8n12:09:25

agreed. I think each context is probably it’s own spec.

favila12:09:19

Yeh spec doesn’t support this, not sure where you heard about that. If you find out I’m interested

favila13:09:23

There are two cases that hurt where I want this. One is dealing with datomic data. Because it’s a graph each ref key could be (in the widest possible case) an entity id, a string, a lookup ref, or a nested vector or map, again with unknown keys ( depends on what was pulled)

favila13:09:00

At certain spots in the program I want to allow only a few of those possibilities. Eg On read I often want to guarantee that a certain set of sub-keys was pulled

favila13:09:53

The other case is dealing with data where the same key is used but it may have a narrower spec depending on the “kind” of map it is in

favila13:09:33

Eg a key could have values 1 2 or 3, but that key must have value 1 when some other key has a certain value

favila13:09:14

Multi-spec and predicates can do this but it’s awkward and verbose and doesn’t gen well

pvillegas1216:09:58

Interested about this, I would like to be able to express that a restriction on a composed spec. If spec1 has value :a within a set of alternatives, spec2 cannot have that value (both of spec1 and spec2 are keys of another spec)

pvillegas1220:09:43

To do this, one can s/and an arbitrary function at the compound spec level to make this happen defined as the second operand (for gen support to be there)

favila21:09:18

s/and of s/keys works very badly

favila21:09:38

it's unlikely a useable generator will result

favila21:09:50

s/and maps to gen/such-that

favila21:09:12

this is why s/merge exists

pvillegas1221:09:28

In my case I don’t want to merge two sets of specs. I have a spec that is the composition of 5 specs (with s/keys) and I wanted two ensure that two of those did not have the same value

pvillegas1221:09:03

This works with gen because the set of possibilities for those specs is 2 😛 (a really small set)

favila21:09:15

5 s/keys specs?

pvillegas1221:09:49

(s/def ::integration (s/and 
                      (s/keys :req [:m/name
                                    :m/doc-type
                                    :m/origin
                                    :m/target
                                    :m/rules])
                      distinct-origin-target))

favila21:09:23

oh that's not at all what I run in to

pvillegas1221:09:32

what do you run into?

pvillegas1221:09:56

(I can see how distinct-origin-target can bump gen if the resulting restriction is too big a set)

favila21:09:38

something like if :m/doc-type is "x", :m/origin must be some narrower range of values that is normally allowed

favila21:09:53

vs if :m/doc-type were "y" or "z"

pvillegas1221:09:24

Right, if m/origin for example is restricted to a given string matching a regex it will destroy gen’s ability to generate example data

pvillegas1221:09:20

not sure how with-gen would work with that compound spec though

pvillegas1221:09:26

(that’s how I make sure gen continues to work)

favila21:09:24

s/keys+ works like s/keys, except you can add a :conf map from spec->override

favila21:09:52

the spec is checked all the time

favila21:09:12

but the override is also checked, and is used for gen

favila21:09:09

this is really just contravariance/covariance at the end of the day. The trouble is spec doesn't have a way to structurally use the same map key and spec that key two different ways

favila13:09:39

I wrote a hacky keys+ macro to try and do this. It allows an inline redef of a key’s spec+generator, but it will still conform both the “natural” and the redef-ed spec of the key

justinlee17:09:32

In my reagent app on cljs, I use a spec validator on my app’s state atom. This works well, but is unfortunately a bit slow in some cases. Is there anything clever one can do to do validations in this way but skip parts of the state tree that haven’t changed? The answer might be just to use more than one atom.

noisesmith17:09:00

the validation function receives the two states as args, so at least hypothetically you could check only changed subtrees using cheap identity checks

noisesmith17:09:04

I'd imagine a classic recursive tree walk, either short-circuiting if identity is true, or checking if identity was false, for each branch new/old

noisesmith17:09:20

then again you'd have to also walk subtrees of the spec to really make it work...

noisesmith17:09:26

sounds messy

dadair17:09:55

Is the state atom flat or deep? If its flat you could have a spec for each of the main subtrees, and only validate the main subtrees that have changed?

noisesmith17:09:10

that is probably the best plan

justinlee17:09:49

yea that’s probably the right approach. actually i already have specs for each of the main subtrees

noisesmith17:09:06

and I bet using identical? instead of = performs better for checking each subtree for changes, but that should be straightforward to swap out and test

justinlee17:09:00

that works beautifully. the obvious thing i didn’t realize is that the validator of course is called before the atom is changed so you can just compare it against the old atom

pvillegas1217:09:04

I would like to be able to express that a restriction on a composed spec. If spec1 has value :a within a set of alternatives, spec2 cannot have that value (both of spec1 and spec2 are keys of another spec’d map)

pvillegas1217:09:09

Is there some way to achieve this?

noisesmith17:09:58

@lee.justin.m one of the args to the validator function is the old state, one is the new state

misha17:09:09

assuming the state tree is a map, and if there are no cross-keys validations (if k1 is this, k2 should be that), you can top level diff new and old values, and just test it against empty (s/keys) @lee.justin.m

misha17:09:33

eg: old {:a 1 :b 2}, new {:a 1 :b 3 :c 4}, diff {:b 3 :c 4}, (s/valid? (s/keys) diff)

misha17:09:07

might save you some milliseconds, if state is huge, but diff is just few keys at a time

justinlee17:09:01

@noisesmith I was using https://clojuredocs.org/clojure.core/set-validator! which says `validator-fn must be nil or a side-effect-free fn of one argument, which will be passed the intended new state on any state change`

justinlee17:09:09

is there something else I could be using?

justinlee17:09:57

@misha yea there aren’t that many top level keys and most of them won’t change, so I think I can just test them with identical?

misha17:09:47

that's the validator, yes. I don't know about other ones either. I think @noisesmith meant usual ref-callback with [key ref old new] signature

dadair17:09:23

that’s using add-watch

noisesmith17:09:25

@dadair yeah that's the one I was thinking of

dadair17:09:40

it mostly depends on how you want to handle the spec failure

noisesmith17:09:43

I got the two functionalities confused

justinlee17:09:22

the only thing I’d really like to do is have the validator throw an error synchronously so that i get a stack trace from the offending function, though this is straying from the channel topic

justinlee19:09:46

are specs queryable once they’ve been made? it’d be cool to ask for the spec corresponding to a key in a map dynamically

Chris22:09:32

Is there a known issue with s/or and not working with s/keys?

Chris22:09:01

For example, this works fine:

(s/conform (s/or :success (s/keys :req [::email/email-address])
                 :not-found nil?) {::email/email-address ""})
but if I have additional keys in the value, i get invalid:
(s/conform (s/or :success (s/keys :req [::email/email-address])
                 :not-found nil?) {::email/email-address ""
                                              ::email/creation-timestamp "2018-09-26T21:47:57.304Z"})

seancorfield23:09:11

@chris547 If you call s/explain (instead of s/conform) in that second case, what does it say? I suspect you have a spec for ::email/creation-timestamp and your string is not valid for that.

Chris23:09:40

It tells me that creation-timestamp doesn't match the :not-found predicate

Chris23:09:39

In: [:io.cloudrepo.signup.email/creation-timestamp] val: "2018-09-26T21:47:57.304Z" fails spec: :io.cloudrepo.signup.email/creation-timestamp at: [:success :io.cloudrepo.signup.email/creation-timestamp] predicate: :clojure.spec.alpha/unknown
val: #:io.cloudrepo.signup.email{:email-address "", :creation-timestamp "2018-09-26T21:47:57.304Z"} fails at: [:not-found] predicate: nil?

Chris23:09:01

It's super weird

Chris23:09:41

because all it should care about is if ::email/email-address conforms, since that's the only key mentioned in the :success condition of the s/or