Fork me on GitHub
#clojure-spec
<
2019-02-11
>
joshkh01:02:59

how might i go about creating a generator for BigDecimal values? is this a job for fmap?

(defn bigdec? [n] (instance? BigDecimal n))
(s/def :bank/balance bigdec?)
(s/gen :bank/balance)
ExceptionInfo Unable to construct gen at: [] for: :bank/balance  clojure.core/ex-info (core.clj:4739)

misha09:02:14

@joshkh

;; Clojure 1.10.0
(s/exercise decimal?)
=>
([0.5M 0.5M]
 [1.0M 1.0M]
 [2.0M 2.0M]
...

👍 5
misha09:02:38

(defn decimal?
  "Returns true if n is a BigDecimal"
  {:added "1.0"
   :static true}
  [n] (instance? BigDecimal n))

borkdude14:02:17

maybe stupid question, but what’s the benefit of s/defop over a macro? https://github.com/borkdude/speculative/issues/124#issuecomment-462352972

mpenet14:02:39

it's basically a parameterized spec, not 2 different specs

Alex Miller (Clojure team)14:02:35

the benefit is that it forms back to the op

mpenet14:02:59

I didn't check the source but reading the comments about it on the dev notes that what it felt like

Alex Miller (Clojure team)14:02:08

that is, you have added a new symbol to the spec op language

Alex Miller (Clojure team)14:02:16

and yes, it’s a parameterized fixed spec op

Alex Miller (Clojure team)14:02:30

not a combinator of arbitrary other spec ops

Alex Miller (Clojure team)14:02:00

it’s not intended to cover every possible case, just address one common need

borkdude14:02:21

what’s the benefit of “forms back to the op”?

Alex Miller (Clojure team)14:02:02

as in the example at https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha, you have created something with your own semantics

mpenet14:02:09

s/form return value is your op

Alex Miller (Clojure team)14:02:24

user=> (s/form ::zip)
(user/bounded-string 5 9)

Alex Miller (Clojure team)14:02:53

so you retain your semantics

borkdude14:02:55

ah. I was just trying this out. I imagined you would get a specific error message for bounded-string:

user=> (s/explain (bounded-string 0 2) "foooo")
"foooo" - failed: (<= 0 (count %) 2)

Alex Miller (Clojure team)14:02:24

you’re getting the more specific error messages from the definition of bounded-string

mpenet14:02:33

you can also map the op to something useful in your context for other means I guess

mpenet14:02:04

user/json-object

Alex Miller (Clojure team)14:02:07

many of the spec ops have internal constraints that will generate explain problems

Alex Miller (Clojure team)14:02:18

so this is the same as other spec ops

Alex Miller (Clojure team)14:02:43

like regexes check whether the input is nil? or sequential?

Alex Miller (Clojure team)15:02:59

it may be that some further adjustments should be made in the explain generator (just deferring to the definition spec right now)

Alex Miller (Clojure team)15:02:28

I didn’t highlight it, but inst-in, int-in, and double-in are all derived specs and I reimplemented all of those in terms of defop

Alex Miller (Clojure team)15:02:47

they are effectively all parameterized compound specs

borkdude15:02:50

This gives more or less the same result when it comes to error messages:

user=> (defmacro bounded-string2 [min max] `(s/and string? #(<= ~min (count %) ~max)))
#'user/bounded-string2
user=> (s/explain (bounded-string2 0 2) "foooo")
"foooo" - failed: (<= 0 (count %) 2)
but s/form will give the expanded spec form, so that’s indeed different

Alex Miller (Clojure team)15:02:18

the other big thing is that with defop, we actually def a macro

Alex Miller (Clojure team)15:02:41

well I guess you’ll get the same effect if you’re using defmacro here

Alex Miller (Clojure team)15:02:11

(vs just implementing the lower-level create-spec)

borkdude15:02:39

@mpenet > it’s basically a parameterized spec, not 2 different specs Not sure what you mean by this. If you call (bounded-string 1 10) and (bounded-string 1 20) you will get two different spec objects

mpenet15:02:41

yeah but the form share quite a bit, I guess the facts it's 2 distinct spec objects is an impl detail

mpenet15:02:10

as I said I don't know more than what I read on the blog post + intuition of what's the final intent

mpenet15:02:17

for lib authors it's nicer to read (s/sorted-coll-of x comparator) than (s/and (s/coll-of x) #(..))

borkdude15:02:28

when would a lib author read this? not trying to argue for the sake of arguing, just want to get it clear for myself 🙂

borkdude15:02:28

I haven’t used s/form and friends myself yet, so I haven’t needed this feature much (probably out of ignorance)

mpenet15:02:30

ex: spec-tools when trying to understand specs to build json schemas

mpenet15:02:17

here we have specs that are used to validate json params also, one of which is some kind of derivative of coll-of, it makes dealing with it much nicer too

borkdude15:02:52

but compared to a macro you don’t see a benefit there yet? I mean when reading and writing the literal code?

mpenet15:02:05

not sure what you mean

mpenet15:02:40

more readable s/form is a plus for me, ability to build your own s/coll-of like ops is good too

borkdude15:02:05

I mean, when you define a macro you get to write same code:

user=> (defmacro bounded-string2 [min max] `(s/and string? #(<= ~min (count %) ~max)))
#'user/bounded-string2
user=> (bounded-string2 1 100)
#object[clojure.spec_alpha2.impl$and_spec_impl$reify__1407 0x64b3b1ce "clojure.spec_alpha2.impl$and_spec_impl$reify__1407@64b3b1ce"]
user=> (bounded-string 1 100)
#object[clojure.spec_alpha2$op_spec$reify__1022 0x1a0b5323 "clojure.spec_alpha2$op_spec$reify__1022@1a0b5323"]

borkdude15:02:17

when are you using s/form?

borkdude15:02:29

maybe this is used by explain logic?

mpenet15:02:03

anytime you want to understand what's behind a spec

mpenet15:02:05

programatically

borkdude15:02:12

> ability to build your own s/coll-of like ops is good too this is limited with s/defop since you cannot pass in any spec argument. e.g. (my-coll-of (s/nilable ::my-spec)) won’t work?

mpenet15:02:38

as I said I read only the blog post

mpenet15:02:54

and what's here

borkdude15:02:55

that’s what left me wondering about defmacro vs s/defop

borkdude15:02:51

maybe not a huge limitation since you can always def the “anonymous” spec first

Alex Miller (Clojure team)15:02:24

int-in, inst-in, double-in are all good examples where this is useful too

Alex Miller (Clojure team)15:02:38

they are all compound parameterized specs

borkdude15:02:40

> where this is useful What exactly are you referring to?

borkdude15:02:07

the result of s/form?

Alex Miller (Clojure team)15:02:11

I just mean general cases where s/defop is better

Alex Miller (Clojure team)15:02:53

symbolic specs form a language, defined by the ops

Alex Miller (Clojure team)15:02:31

using a macro that expands to a compound spec is fine - you’re using an initial step to produce something in the language

Alex Miller (Clojure team)15:02:21

using defop lets you create new ops in the language for the special case where the op can be defined in terms of other spec ops (and is not parameterized by another spec)

Alex Miller (Clojure team)15:02:39

and if you need that, you can drop down another level and do the same thing the provided ops are doing - implement the create-spec multimethod to return a Spec protocol instance

Alex Miller (Clojure team)15:02:54

(but a lot more code is required)

borkdude15:02:01

is nilable an op? then why can (s/nilable ::foo) not be passed as an argument to defop, but ::foo can? sorry, bit confused about “symbolic specs”

Alex Miller (Clojure team)15:02:59

as I said above, defop is not designed to be parameterized by other symbolic specs

borkdude15:02:14

so passing ::foo just accidentally works?

borkdude15:02:49

(seqable-of ::foo) vs. (seqable-of (s/nilable ::foo))

Alex Miller (Clojure team)15:02:07

it really goes to the evaluation model. spec names (fq keywords) are a little special in that they eval to themselves and also they are the only thing other than a spec object explicitly handled in the spec api functions.

Alex Miller (Clojure team)15:02:03

in this case, ::foo evaluates to a valid symbolic spec (where another spec form evaluates to a spec object, which is not a symbolic spec)

Alex Miller (Clojure team)15:02:22

so it will work, but you’ve created a narrow constraint on how it can be used

borkdude15:02:54

right, so it’s something that happens to work, but not really the common use case for defop

Alex Miller (Clojure team)15:02:02

yeah, I hadn’t really thought about that

Alex Miller (Clojure team)15:02:47

you’re also not going to get proper explain path-ing with it as a spec created by defop is considered to be a single op

Alex Miller (Clojure team)15:02:04

so if the sub-spec fails, you won’t get the parent spec in the path

borkdude16:02:39

ok. to conclude: defop is not designed to support spec arguments. If you want that, either write a macro and accept less helpful error messages and s/form output, or “drop down another level and do the same thing the provided ops are doing - implement the create-spec multimethod to return a Spec protocol instance” which requires more code

Alex Miller (Clojure team)16:02:33

although I think in the macro case, the errors are almost exactly the same

Alex Miller (Clojure team)16:02:47

so I would maybe quibble with that part

borkdude16:02:35

not going to publish this anywhere, so I think for now it’s clear 😉

borkdude16:02:03

I notice that regular pre-defined predicates are also not supported in defop:

user=> (s/defop first-pred [pred] (s/and (pred (first %))))
#'user/first-pred
user=> (s/valid? (first-pred number?) [1 "f"])
Maybe a bad example

Alex Miller (Clojure team)16:02:23

the definition in defop is not going to evaluated - it should be a valid symbolic spec but where the parameterized values are substituted (defop is literally going through and replacing the args with their values)

borkdude16:02:20

Made a typo. This works:

user=>  (s/defop first-pred [pred] (s/and #(pred (first %))))
user=> (s/valid? (first-pred number?) [1 "f"])
true

seancorfield16:02:22

For specs parameterized by other specs, you can do something like

(defn domain-keywords
  "Given a spec, return a new spec that can conform a keyword or string to
  a keyword that is constrained by the given spec."
  [spec]
  (s/spec* `(s/with-gen (s/and ::keyword ~spec)
              (fn [] (g/fmap name (s/gen ~spec))))))
(that and bounded-string above come from the World Singles Networks' codebase)

borkdude16:02:49

@seancorfield what benefit does that have over writing domain-keywords as a macro?

Alex Miller (Clojure team)16:02:15

well, it’s a function so you can apply it, so usual benefits of function over macro

Alex Miller (Clojure team)16:02:23

(with the caveat that the spec arg needs to be a form, not an object, here)

seancorfield16:02:44

It was a function in the spec1 world (without (s/spec* ..) and the quote/unquote, so we made it a function in the spec2 world. Minimal change (and it still composes and applies etc).

borkdude16:02:38

@seancorfield I had a similar thing with seqable-of. Function in spec1, but when I turned it into a function in spec2 using s/spec*, I could not provide specs like (s/nilable ::foo) because I got an error, so then I made it a macro.

seancorfield16:02:39

I plan on writing up a (probably long) blog post on our conversion from spec1 to spec2 when Alex tells me spec2 is stable enough for that to be widely valuable 🙂

Alex Miller (Clojure team)16:02:48

you can’t use this with in s/def though (like (s/def ::x (domain-keywords ...)))

seancorfield16:02:05

Right. And we use s/defop for those sorts of things.

Alex Miller (Clojure team)16:02:06

but you could with the functional entry point s/register which takes an object (which is what domain-keywords returns)

Alex Miller (Clojure team)17:02:11

@seancorfield btw, I spent some time working on the regex thing and I need to undo the changes I made to support forward references in regexes

Alex Miller (Clojure team)17:02:25

which will fix the nesting issue, but re-break forward references

seancorfield17:02:42

@borkdude Mostly, we've found changing our defn spec builders over to s/defop has been all we've needed in the most common cases. A few have needed s/spec* instead.

borkdude17:02:46

> Right. And we use s/defop for those sorts of things. Sorry, referring to what?

seancorfield17:02:30

@alexmiller That's fine -- the forward reference issue only affected one spec in our entire code base so I can just move it below the sub-specs 🙂

Alex Miller (Clojure team)17:02:31

Forward refs in regex is solvable via different means but I need to talk to Rich before I commit to a path on that and he’s out today

seancorfield17:02:55

@borkdude "those sorts of things" = "use this with in s/def"

borkdude17:02:21

I can’t write a very long blogpost about transitioning to spec2. All I had to do is report bugs, which were all fixed, wrap a bunch of predicates in s/spec, refactor one predicate to #(not (sequential %)) instead of (complement sequential?) and turn a private function into a macro.

borkdude17:02:40

I think I might write a tweet about it instead.

Alex Miller (Clojure team)17:02:30

@seancorfield reverted the fwd reference fix, which should fix the nesting issue (but break that fwd reference)

borkdude17:02:51

did forward referencing ever work? didn’t know

Alex Miller (Clojure team)17:02:10

in general, specs delay lookup of named specs until use

Alex Miller (Clojure team)17:02:23

I fixed several places where that wasn’t being done

Alex Miller (Clojure team)17:02:05

but changes in how regexes are implemented mean that we effectively broke it for them

Alex Miller (Clojure team)17:02:54

regex impls used to not be specs but would get spec-ized when needed. in spec 2, regexes actually are spec instances (thanks to metadata protocol implementation!) which simplifies the code in many places, but removed the point where this delay naturally happened before

Alex Miller (Clojure team)17:02:18

fixing it is … tedious

borkdude17:02:38

(s/declare ::foo) 😉

seancorfield18:02:35

@alexmiller Good to know. I'll run a full test suite with the latest spec2 shortly.

seancorfield19:02:46

@alexmiller Confirming that our full test suite runs on the latest spec2, with that one forward reference regex spec moved after the specs that it refers to.

♥️ 10
borkdude19:02:40

speculative still works as well

♥️ 5