Fork me on GitHub
#clojure-spec
<
2018-03-30
>
robert-stuttaford06:03:32

likely a common question, and also likely not something spec is suited for, but i’m curious even so - i have a map with start and end dates. is there a spec pattern for declaring a relationship between those two values, i.e. one must be larger than the other? one structure that occurs is spec/fdef’s :ret, but i’m wondering if perhaps the spec api has something else like this?

borkdude09:03:08

@robert-stuttaford you can use s/and + a conformer function

borkdude09:03:56

Something like

(s/def ::dates
  (s/and
   (s/keys :req-un [::start ::end])
   (fn [{:keys [start end]}]
     (< start end))))
where < is the comparison function of your choice.

robert-stuttaford10:03:00

wonderful, thank you! that seems ridiculously simple, in hindsight. like, ‘how did i not see this’ simple

robert-stuttaford10:03:56

@borkdude i guess i’d have to write my own generator too then

borkdude10:03:54

Don’t know if the default generator generates enough samples where the conformer can strip away the invalid ones. Not as efficient as writing your own, but it could work

borkdude10:03:11

it’s about 50% chance for each sample

borkdude10:03:41

when you write an fdef in another namespace than the function, require the namespace where the function lives? or don’t and just fully qualify the symbol in fdef? hmm

borkdude10:03:25

Right now I have an init namespace that just requires them all, so no cyclic dependencies and I can just fully qualify in fdef

robert-stuttaford10:03:03

what’s causing you to want to keep the fdef separate, @borkdude?

borkdude10:03:21

because it’s more than 100 lines

dominicm10:03:16

I feel like I remember there being some kind of way to add conformers to specs "locally"? Is that a correct memory?

robert-stuttaford10:03:32

ah 🙂 Qualifies fn-sym with resolve, or using *ns* if no resolution found. seems to suggest that fully-qualified is fine, and probably better for discoverability

borkdude10:03:27

I have an interesting problem:

(s/def ::selection
  (s/nilable
   (s/and
    (s/keys :req-un [:selection/options
                     :selection/id]
            :opt-un [:selection/default-option
                     :selection/type])
    ;; default option must be one of the option ids
    (fn [dropdown]
      (if-let [opt (:default-option dropdown)]
        (contains?
         (set (map :id (:options dropdown)))
         opt)
        true)))))

(s/def ::dropdown ::selection)
I want the id in dropdown to be optional… Maybe I should make an extra spec without the required id and then make the id in selection required with s/and?

borkdude10:03:11

Like this:

(s/def ::selection
  (s/nilable
   (s/and ::selection*
          (s/keys :req-un [:selection/id]))))

(s/def ::selection*
  (s/nilable
   (s/and
    (s/keys :req-un [:selection/options]
            :opt-un [:selection/default-option
                     :selection/type])
    ;; default option must be one of the option ids
    (fn [dropdown]
      (if-let [opt (:default-option dropdown)]
        (contains?
         (set (map :id (:options dropdown)))
         opt)
        true)))))

(s/def ::dropdown ::selection*)

borkdude10:03:51

Seems to work

robert-stuttaford11:03:45

what’s the blessed method for checking whether a spec is registered? s/spec? seems to be for something else

borkdude11:03:08

@robert-stuttaford brute force method: use a println in a conformer 😛

borkdude11:03:25

or just make an obvious mistake and if no exception, then no 😉

robert-stuttaford11:03:11

-grin- its for datomic attrs. the code using the spec can’t assume a spec is registered; it has to check first before it attempts to use it to validate.

robert-stuttaford11:03:30

grr, i’m having to wrap my defmulti with a normal defn so that i can instrument it. otherwise defmethods defined after instrumentation fail

borkdude11:03:52

Funny that s/spec? doesn’t return a boolean

borkdude12:03:36

don’t all new fdefs after instrumentation fail?

borkdude12:03:48

in the sense that they aren’t instrumented yet

Alex Miller (Clojure team)13:03:31

Yes, although I wouldn’t call that a fail

borkdude13:03:34

more appropriate: not yet in effect

gfredericks14:03:19

argument validation in libraries: going forward, should it be done entirely with s/fdef, meaning no validation happens unless the user instruments the functions?

borkdude14:03:23

s/assert is also an option I guess which can be compiled away

borkdude14:03:35

but for arguments s/fdef is nicer

gfredericks14:03:19

the downside is that things are a lot more GIGO for users who don't think to instrument e.g., as a user, can I easily instrument all the spec'd functions in all my libraries without having to know which ones have specs? is that too much? wouldn't it be more efficient to instrument only the functions I'm calling directly?

borkdude14:03:00

you can instrument all fdef-ed functions with stest/instrument?

borkdude14:03:31

but it has to be after you load their namespaces

borkdude14:03:53

maybe adding it to reloaded.repl/reset will be a common thing

borkdude14:03:45

but I get what you mean now. so you want to go only one level deep

borkdude14:03:57

no transitive fdef checking

gfredericks14:03:21

that'd be nice, since you probably have a large tree of libraries and don't want to slow down your dev by testing all the interactions between them

borkdude14:03:11

What overhead are we talking about? I don’t mind a couple of milliseconds more during dev

gfredericks14:03:44

totally depends on the libraries and what they're doing

gfredericks14:03:36

in the extreme case, if specs get added to most of the clojure.core functions, instrumenting those will result in milliyears instead of milliseconds

borkdude14:03:50

good question

hlship20:03:44

Here's a question. I have a value that I want to ensure is a keyword, string, or symbol AND that it's string conforms to a particular regexp. Example here: https://github.com/walmartlabs/lacinia/blob/05940c7f7819fd88bc4e50c860b8d9854c3fa0b2/src/com/walmartlabs/lacinia/schema.clj#L306

Alex Miller (Clojure team)20:03:54

that’s not a question :)

hlship20:03:07

I'm working on it ...

hlship20:03:58

I've been down this path before, and what I've found is that the next term in the s/and gets the conformed value from the s/or, a tuple of (say), [:keyword :frob].

hlship20:03:08

That's been fine so far EXCEPT as I'm switching to using Expound, the use of a conformer here is a problem:

(s/explain ::schema/enum-value "this-and-that")
             clojure.lang.ExceptionInfo: Cannot convert path. This can be caused by using conformers to transform values, which is not supported in Expound
clojure.lang.Compiler$CompilerException: clojure.lang.ExceptionInfo: Cannot convert path. This can be caused by using conformers to transform values, which is not supported in Expound {:form "this-and-that", :val [:string "this-and-that"], :in [], :in' []}, compiling:(/Users/hlship/workspaces/github/lacinia/src/com/walmartlabs/lacinia/expound.clj:50:3)

Alex Miller (Clojure team)20:03:50

this issue is actually discussed in the backchat

hlship20:03:19

Recently? Got a link?

Alex Miller (Clojure team)20:03:38

2 days ago in this room - just scroll up till you see the expound stuff

hlship20:03:09

That discussion wasn't helpful, if its the right one. I think they're hitting the same problem and want Expound to print it differently. I want to modify my spec to not trip over this scenario. s/nonconforming may work!

hlship20:03:06

So my question is, how can I achieve the kind of spec I want in a way that avoids the use of a conformer in the middle.

Alex Miller (Clojure team)20:03:55

another option is to wrap s/nonconforming around s/or

Alex Miller (Clojure team)20:03:06

then you get just the value without the tag

Alex Miller (Clojure team)20:03:12

when you conform that is

Alex Miller (Clojure team)20:03:40

currently that’s an undocumented function but I think it’s likely we will either keep it or add a nonconforming variant of s/or

seancorfield20:03:25

And in the backchat, one suggestion was to look at pinpointer instead of Expound.

bbrinck20:03:50

@hlship I will likely be adding a “fallback” feature to expound soonish where you will see the vanilla spec error in this case

bbrinck20:03:18

Note that will help if you only occasionally use conformers, but not if you have lots of them. Definitely check out pinpointer 🙂

hlship20:03:39

Again, I'm quite willing to modify my spec to bypass this problem.

bbrinck20:03:58

Ah, sorry, I missed that in the thread. Yes, that’d be the best approach! 🙂

bbrinck20:03:03

I’ve seen people run into other issues with conformers so it’s probably best to avoid e.g. https://groups.google.com/forum/#!searchin/clojure/conformer%7Csort:date/clojure/Tdb3ksDeVnU/uU0NT4x6AwAJ

bbrinck20:03:21

FWIW, I tried to support conformers in expound but it’s really tricky if you’re just looking at the explain-data

hlship20:03:09

s/nonconforming looks to be just what I want:

(s/explain ::schema/enum-value "this-and-that")
-- Spec failed --------------------

  "this-and-that"

must be a valid GraphQL identifier: contain only letters, numbers, and underscores
BTW why are explicit messages not indented by Expound? Should I file an issue?

bbrinck20:03:23

Can you modify that to show what you’d prefer?

bbrinck20:03:00

(I’m always happy to get bug reports too 🙂 if that’s easier to discuss the options)

bbrinck20:03:47

I’ll say this - off the top of my head, I think it’s working as I intended, but I’m always interested in improving the layout of error messages if it’s not clear

hlship20:03:47

Here's a better example:

(s/explain ::schema/resolve {})
-- Spec failed --------------------

  {}

should satisfy

  fn?

or

implement the com.walmartlabs.lacina.resolve/FieldResolver protocol
The final line should be indented the same as the fn? line, don't you think?
;; is passed and should return.
(s/def ::resolve (s/or :function ::resolver-fn
                       :protocol ::resolver-type))
(s/def ::resolver-fn fn?)
(s/def ::resolver-type #(satisfies? resolve/FieldResolver %))

bbrinck20:03:24

Yeah, when it’s part of the “or” it looks weird. I guess I was thinking that since “should” starts on left, custom messages would be the same.

bbrinck21:03:22

The reason it’s on the left is that it should be in the same spot as “should” like so:

(s/def :example/temp #{:hot :cold})  
(expound/expound :example/temp 1)
;;-- Spec failed --------------------
;;
;;  1
;;
;;should be one of: :cold, :hot

(expound/def :example/name string? "should be a string")
(expound/expound :example/name 1)
;;-- Spec failed --------------------
;;
;;  1
;;
;;should be a string

bbrinck21:03:54

but I agree that when it’s part of a long “satisfy..or” block, it looks bad. I’ll make a bug

hlship21:03:03

Another expound question (time for its own channel?) :

user=> (require [clojure.spec.alpha :as s])
nil
user=> (s/explain keyword? 3)
val: 3 fails predicate: :clojure.spec.alpha/unknown
nil
user=> (require '[expound.alpha :as expound])
nil
user=> (alter-var-root #'s/*explain-out* (constantly expound/printer))
#object[expound.alpha$printer 0x3c64e2a2 "expound.alpha$printer@3c64e2a2"]
user=> (s/explain keyword? 1)
val: 1 fails predicate: :clojure.spec.alpha/unknown
nil
this works fine when I use set!, but not when I use alter-var-root!. Any ideas?

bbrinck21:03:00

I’ve always used set! myself, so I’m not sure. Can you talk a little bit more about your use case and neither set! nor binding are a good fit here?

hlship21:03:55

Well, at application startup, might want to alter-var-root, so that any later threads will use the Expound printer, not the default one.

bbrinck21:03:25

Thanks for that. The short answer is I don’t know unfortunately - this seems to affect any printer, not just expound. Try: (alter-var-root #'s/*explain-out* (constantly (fn [ed] "Hello!"))). It may be a result of the way the REPL is set up - does it reproduce if you put the alter-var-root in the main of an application (as opposed to doing it in the REPL context)?

hlship22:03:54

I suspect it's because clojure.spec is AOTed. I hit something similar with redefining stuff in clojure.test and the hack was to do a RT/loadResourceScript: https://github.com/AvisoNovate/pretty/blob/db4e7677f74d8efb149db3e9ba5974fa9c84b6a0/src/io/aviso/repl.clj#L72

bbrinck22:03:44

Ah, gotcha. I certainly understand your use case and I suspect others may run into the same thing. If you figure it out, let me know and I’ll update the docs. I have a note about using alter-var-root in non-REPL context, which IIRC, works fine, but if your scenario is at the REPL, then no dice

hlship22:03:18

binding should work, as that will carry over into most started threads, including core.async threads.

Alex Miller (Clojure team)23:03:55

I think the difference is that the repl binds explain-out so alter-var-root changes the root but the repl binding is the one being seen

Alex Miller (Clojure team)23:03:30

So you have to set! at the repl