Can anyone explain this? This works:

((fn []
   (if-let [k nil]
     k
     :other-keyword)))
=> :other-keyword

((fn []
  (if-let [k nil]
    k
    :clojure.spec/invalid)))
CompilerException clojure.lang.ExceptionInfo: Call to clojure.core/if-let did not conform to spec:
In: [2] val: :clojure.spec/invalid fails at: [:args :else] predicate: any?
...

:clojure.spec/invalid is a special value — you have to be really careful about code that produces it

I think you can do (keyword “clojure.spec” “invalid”) as one way to get around that (i.e., generate it at runtime).

(I suspect there’s a cleaner way)

seancorfield, I agree that it's special, I am using it in a conformer

this is a stripped down example

It’s special to spec itself. And in 1.9, spec is used to syntax-check various macros.

in your opinion, is this expected behavior?

Another option is

(def csi :clojure.spec/invalid)
…
((fn [] (if-let [k nil] k csi)))

It is certainly a known issue but I don’t know whether there are plans to change the behavior.

it seems weird that there are special keywords i can't return from an if-let

(I was actually a bit surprised that I could just def a global alias for it like that)

why not? it's just a keyword

http://dev.clojure.org/jira/browse/CLJ-1966

oh ok great, someone filed it

Known since last June and no indication that it really is a problem that needs to be fixed (and it has an easy workaround — that I’d simply forgotten!)

(ironic that I was the one that provided the easy workaround in that ticket but I’d forgotten)

so (if-let [k nil] k ':clojure.spec/invalid)

that's a cool workaround 🙂

Since macros are just functions on code-as-data, spec can’t tell the difference between a conforming spec that produces :clojure.spec/invalid — the special value — and a literal :clojure.spec/invalid in the actual code.

i hope it get's fixed

https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/conformer

the docs should probably change then

I suspect the answer from the Clojure/core folks will be: if you’re writing a spec or a conformer that needs to produce the special invalid value, you must quote it.

it's not at all obvious that you're supposed to quote it

You can return :clojure.spec/invalid in all sorts of constructs just fine. You only need to quote it in some situations.

We have all sorts of specs and conformers that produce :clojure.spec/invalid as a literal value — we just don’t have code around it that is a macro that is checked by spec in the compiler.

(if (some-check v) v :clojure.spec/invalid) works fine, as does (try (some-coercion v) (catch Exception _ :clojure.spec/invalid))

i wonder what will happen once cond is spec'ed too

you can return it now

but not later?

coming back to our discussion the crux of the issue is the following

(s/valid? (s/* any?) [::s/invalid])
=> false
(s/explain (s/* any?) [::s/invalid])
In: [0] val: :clojure.spec/invalid fails predicate: any?
=> nil
(any? ::s/invalid)
=> true

that's a contradiction

Sorry for breaking into the discussion. Does anyone know it's a known limitation or an unintended behavior that multi-spec in general cannot be unformed?

Is there some kind of core.spec function for "generate a view of this data structure that conforms to this spec"? for instance

(s/def ::just-one-key (s/keys :req-un [::apples]))
(*mystery-function* ::just-one-key {:apples "apples" :bananas "bananas"})
;-> {:apples "apples"}

conform is supposedly for destructuring, but perhaps my understanding of destructuring is wrong, because intuitively I feel like "select this known data pattern out of this unknown data" is what destructuring is for

Basically, the use case here is "web request should conform to this data, but if you pass in other data, that's perfectly fine. But when we store the data off in the database, we should only select the part that we understand and care about."

so, the spec's don't disallow keys, but we select the parts we're interested in for passing along

Recently I sometimes see error messages like these: adzerk.boot_cljs.util.proxy$clojure.lang.ExceptionInfo$ff19274a: ERROR: Attempting to call unbound fn: #'clojure.spec/macroexpand-check at file /Users/Borkdude/.boot/cache/tmp/…/app/1x42/fg7eat/public/js/app.out/ajax/core.cljc, line 591, column 1 I think it was after upgrading clojure.future.spec to alpha15

Not sure where it comes from

Now I get: Attempting to call unbound fn: #'clojure.spec/macroexpand-check at line 2164, column 1 in file /Users/Borkdude/.boot/cache/tmp/.../app/1xx5/-x902hh/public/js/app.out/cljs/pprint.cljs

Hmm, sorry, this was with alpha-14. Now I upgraded to clojure.future.spec alpha 15.

this is crazy, it doesn't even make any sense to me. the guide here https://clojure.org/guides/spec seems to imply that conform is used for destructuring. Here's the example it gives:

(defn configure [input]
  (let [parsed (s/conform ::config input)]
    (if (= parsed ::s/invalid)
      (throw (ex-info "Invalid input" (s/explain-data ::config input)))
      (for [{prop :prop [_ val] :val} parsed]
        (set-config (subs prop 1) val)))))

why not just do (if (s/valid? ::config input)?

Destructuring plucks out the parts you care about — but leaves all the other parts in place (which you can see with :as).

Clojure takes an inherently “open for extension” stance on that sort of thing: it’s considered good practice to be able to accept (and ignore) additions to data structures that still conform.

In order to save stuff to a database, you need to explicitly restrict the keys in a map — either with select-keys (ignoring the extra keys) or something that checks you’ve only get the expected keys and no others.

In general, when saving to a database (with java.jdbc), we tend to build the map we need from the conformed input map, so we explicitly select the columns we need and ignore the rest. ^ @tjtolton

yeah, that's a shame @seancorfield I'm totally down with the "open for extension" concept, but I'm also interested in reducing the cost of detection with spec problems.

Basically, permissive specs are great, but they let you do stupid things, like mispelling an optional key

if you mispell an optional key, you have no way of knowing about it. Nothing will ever fail it

I would expect code that called a function incorrectly like that to be caught by tests on the calling function.

i.e., if func-a calls func-b with misspelled arguments, tests for func-a should catch that.

so, its interesting, because I basically just want a select-keys, but one that is recursive and spec aware

It's still very strange to me that it isn't available. Even in an "open for extension" system, this seems like an important utility.

Especially for web programming. I only want to display these values on screen, I only want to return these values in the api call, I only want to save these values to the kv store

all things that spec is great at

@tjtolton in spec-tools there is a conformer that drops extra keys out of keys-specs. Will blog about those soon.

I should be able to reuse that functionality

@ikitommi spec tools?

metosin?

is there a way to enforce that a pair of keys exists in a map, or else neither? That is, if one of them exists the other is required, however they are both optional otherwise.

ddellacosta something like

(fn [m]
  (or (and (:k1 m) (:k2 m)) 
      (or (not (:k1 m)) (not (:k2 m)))))

ie. just use a regular function

ugly but I guess it works. I’ll have to compose it with s/keys so I can enforce the presence of other keys

hmm actually I’m not exactly sure how I would

(s/and (s/keys …) key-check-fn)

I think the second term should be (not (or (:k1 m)) (or (:k2 m)))

@ddellacosta (s/keys :req [(and ::foo ::bar)])

huh, that works? neat

Yep, check the doc string

oh but wait, that means they are both required always, doesn’t it?

Ah yeah, maybe you have to put it in :opt

oh but if I can do that then still, problem solved

lessee

ah no, it's only supported in :req

well, that is lame

but, great idea dergutemoritz , thanks for trying

Too bad 🙂

yeah

Well, good to know in other situations anyway!

yeah!

in the end this worked:

26│  (s/def ::thing
 27│    (s/and
 28│     (s/keys :req [::required1]
 29│             :opt [::optional1 ::optional2 ::start-date ::end-date])
 30│     (fn [m]
 31│       (or (and (::start-date m) (::end-date m))
 32│           (not (or (::start-date m) (::end-date m)))))))

hopefully including them in the :opt enforces value checking on those fields, going to check now

Yeah, s/keys checks all keys, even those not specified

Heh ok that's a bit ambiguous

yeah, I guess I should have assumed that

Even those not passed to s/keys if they have a spec

but yeah, that works—thanks @schmee and @dergutemoritz for the help

You're welcome!

oh yeah? didn’t realize that, that’s handy

Yeah, it's handy but has been the source of surprise in this channel more than once 🙂

I can imagine

I actually wonder why and and or are not supported in :opt now ...

Your case makes it seem like a reasonable thing to expect

Note that or in :req isn’t mutually exclusive: (s/conform (s/keys :req [(or ::a ::b)]) {::a 1 ::b 2}) succeeds and produces {::b 2 ::a 1}

It just means "at least one of these must be present”. Both can be present in the normal “open for extension” model that spec has.

^ @dergutemoritz @ddellacosta

@seancorfield yeah, in this case the problem is that I want these two to either be both present, or not at all

but yeah, good to note

Since :opt just means “and here are some more keys you may encounter that have associated specs” — because additional keys are always accepted, they just won’t be assumed to have associated specs to conform to.

right, that makes sense now that I’ve worked through this

@ddellacosta Yeah, the best you can do here is specify them both as :opt if you want them conformed, and then s/and a key check predicate around s/keys. We’ve run into this a lot with mutually exclusive keys — we have to have a separate predicate that checks at most one of them is present. Not sure whether I feel this should be built-in or not...

yeah, that’s exactly what I ended up doing. Not the most elegant thing but it works well.

I mean, I feel like @dergutemoritz ‘s suggestion would have done that very elegantly

but maybe that’s challenging to implement, I dunno

haven’t looked at the actual clojure.spec code so dunno

I'm using spec/conform to destructure & validate a largish data structure from a client, and at some point it's using coll-of. But in the process, conform seems to be reversing the contents of the list its given. Is this expected/normal?