This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2017-06-28
Channels
- # beginners (33)
- # boot (58)
- # cider (21)
- # cljs-dev (197)
- # cljsrn (112)
- # clojure (136)
- # clojure-belgium (5)
- # clojure-dev (57)
- # clojure-greece (1)
- # clojure-italy (3)
- # clojure-russia (6)
- # clojure-spec (148)
- # clojure-uk (54)
- # clojurescript (29)
- # cursive (24)
- # datomic (36)
- # devops (4)
- # emacs (11)
- # figwheel (1)
- # graphql (18)
- # hoplon (6)
- # leiningen (2)
- # luminus (4)
- # off-topic (7)
- # om (4)
- # onyx (27)
- # precept (1)
- # protorepl (12)
- # quil (1)
- # re-frame (28)
- # reagent (10)
- # ring (9)
- # robots (1)
- # rum (2)
- # slack-help (5)
- # spacemacs (16)
- # sql (16)
- # untangled (16)
- # vim (3)
- # yada (2)
whats the proper way to define a spec referring to another namespace?
(s/def ::foo :bar.alice/bob)
(s/def ::my-spec (s/keys :req-un [::foo]))
for some reason i get complaints about unable to resolve spec :bar.alice/bob
using the above method
might i be required to be explicit in :require [bar.alice :as bar.alice]
so load-order is taken care of properly?
The ns containing that spec (`:bar.alice/bob`) must be loaded before you can use it in another spec. Otherwise the s/def
will not have been executed.
is there a way to override the default generator of s/keys
but only for a single key? (without renaming the key)
We have some JSON data coming in over the wire that uses strings as keys e.g. {"city" "Denver" "state" "CO"}
.
We’ve been converting all strings keys to keywords in order to spec them with s/keys
, but of course, doing the mapping takes a call to clojure.walk/keywordize-keys
and then, if there is a problem that we want to report to the dev, we might need to convert back to strings to make a sensible error about the data.
In this case, do people do the conversion like this? It would seem like being about to have string keys would be useful e.g.
(s/keys :req-str [:location/city :location/state])
where :req-str
is like :req
and :req-str-un
would be like :req-un
, but for string keys
bbrinck: would be reasonable to file a jira enhancement for htis
@U064X3EF3 Can do!
I had some slow checks that I was trying to performance tune. One approach I wanted to try was swapping out the fdef spec during the stest/check
run, similar to how you can swap out generators using the :gen
option. I tried instrumenting the function and using the :spec
option prior to the stest/check
run, but that didn't have an effect. Does this seem like a reasonable thing to do? Is there a way to do it?
grzm: check takes a generator override map - did you try that?
It's not clear to me how providing a generator would replace the function spec: I'm not trying to change the values that are supplied to the function: I'm trying to change the spec that's applied to the function as a whole for the scope of the check. Or am I misunderstanding?
oh, then instrument with a replacement spec should be what you want
however, calling instrument correctly is sometimes hard
in particular, anything that is being changed (including the thing whose spec is changing) needs to be in the list of instrumented vars. You should see that var in the return value from instrument as well - if you don’t, it wasn’t changed.
here’s a spec replacement example:
(defn a [x] (if (zero? x) "a" (inc x))) ;; special behavior on 0
(defn b [y] (if (zero? y) 0 (+ (a y) 10))) ;; but b guards this
;; so use simpler spec when testing b
(stest/instrument `a
{:spec {`a (s/fspec :args (s/cat :a int?) :ret int?)}})
(stest/check `b)
(defn a [x])
(s/fdef a
:args (s/cat :x int?)
:fn (fn [_] true))
(s/fdef b
:args (s/cat :x int?)
:fn (fn [_] false))
;; should pass
(stest/check `a)
(stest/instrument `a {:spec {`a `b}})
;; should fail
(stest/check `a)
What I've been trying is more direct. In your example, you're checking b
which calls a
. I'd like to check a
directly.
I’m not sure the val of the :spec map will actually resolve that via the registry - did you try passing the actual spec there?
(stest/instrument `a {:spec {`a (s/get-spec `b)}})
doesn’t seem like that works either
I'm considering taking your spec course at the Conj. How far into the weeds are you going to get? Do you think you'll have a syllabus available?
the example I gave you above is from the spec course
so about that far :)
sorry, I was looking at the code. I understand why it doesn’t work, still thinking about what that means though.
the spec override is used when building the wrapped var in instrument
but check still uses the version in the spec for checking ret and fn, not the overridden spec
and indeed check can’t even see that - it’s just part of the check-fn on the instrumented var
seems like it would be totally reasonable to want to do something like this in the context of check though
instrumented vars will only catch invalid invocations, not invalid results.
so I think a ticket to add the ability for check to take spec overrides etc would be reasonable. we have talked about making things like generator overrides, spec overrides etc a consistent api used across exercise, exercise-fn, instrument, check, etc which would be in this area as well
Okay. I'll open one. (No need to apologize, by the way. I suspected as much, and even if you were doing something else, I'm grateful for the time and attention you're providing.)
I was similarly surprised about generator overrides with instrument and check. I expected generator overrides to apply globally. From what I can tell, they only apply to the checked function, not the functions called by the checked function.
it’s a little more subtle than that
they do apply globally, but there are a couple cases where they don’t get picked up
there are some tickets on this
https://dev.clojure.org/jira/browse/CLJ-2095 is another, although I’m not sure that this is something we actually can or will change
Here's the ticket I opened. https://dev.clojure.org/jira/browse/CLJ-2193
Are you guys happy with Jira? Is it worthwhile for someone to make a Clojure syntax highlighter plugin?
Here's a gist showing the instrument
generator override behavior I mentioned above. https://gist.github.com/grzm/0ada176caee5bcdab39d820577ed3823 I'll bring this up in #clojure-spec as well and continue discussion there unless you bring in back here. Thanks again for your help today.
Performance and the fact the validated data doesn't match the original, so error messages either need to be converted back or are somewhat confusing.
as always with performance i would test and verify that it’s going to be an issue before worrying about doing 2x conversions
@bbrinck with respect to the validated data doesn't match the original, you're already doing a conversion (I suspect) from JSON objects to Clojure maps. The similarity between the two (including syntax) obscures this a bit.
@lwhorton. Fair point about perf. My greater concern is complexity of implementation when converting to validate and then back to report for what seems like a common case of string keys (when working with JSON).
Frankly it's not a huge cost. Just wondering if this workaround is how others are tackling this case.
maybe im looking at it wrong, but i dont see complexity around (keywordize-keys json)
and (string-keys edn)
if you convert to edn, run a spec/validate and it fails, then convert it back to json and make a POST… that all seems fairly straightforward?
I'd ideally like to give error messages that are relevant to the original data. Imagine an API that accepts JSON. If I want to give an errr message about non conformance, I want this to be about strings not keywords.
I suppose I can explain-data and then walk each problem and convert back to strings. The n format the data?
But that's just the whole thing right? That's not the specific section that has the problem
i suppose if you wanted to get into the exact location where a conform failed, yes you might have to do explain-data
You'd need to walk each problem and reconvert, then reformat. But you are correct, it's not a huge amount of work
i would also look around for some libs that might already exist for this sort of thing
Right. Thanks for the ideas! Good to know I'm not missing an obviously simpler way :)
with plumatic.schema there was coersion, but i’m not sure if spec went down that path as well
AIUI, spec is avoiding coercion but I could be mistaken. I might just end up implanting a version of 's/keys' that accepts strings (and produces a generator that uses string keys). Could be useful for JSON validation
Note I agree the specs themselves should be keywords. I'm just thinking that they could be included in maps as strings.
I’ve got a question for you 🙂 … have you ever used branching in a spec to define a spec? Say you have a key :foo and :bar, but :bar can pull from set A or B of valid values. Is there a way to make :bar read :foo and decide which is a valid spec?
spec newbie here, please excuse my ignorance. I have a data structure which I've defined a spec for. I've just needed to add a channel as one of the items. How do I update my spec for this type of a field?
Pretty sure there’s no way to spec a channel (same as there’s no way to spec atom/delay/etc) so you can just use any?
(or not provide an actual spec for that key).
Spec’ing something to be a channel wouldn’t be very useful: you couldn’t generate values for it (since it has type channel).
oh, ok, there's a lot of spec I'm still pretty confused about. So, spec'ing data structures is primarily for building up things from basic types (ints doubles bools and functions) as opposed to interfaces and complex objects?
Hmm, not really…
You can spec domain-level entities — so you can give things meaning — but you need to think about what you’re trying to achieve with spec. Why are you spec’ing a particular structure? What are you going to use the spec for?
There’s not much point in spec’ing everything and I think it’s a common misunderstanding to try to treat spec like a type system and “spec everything”.
You also need to consider whether you want specs to be “generatable” (you probably mostly do) so you need to avoid specs with types that can’t be generated (since they’re not very useful, in general).
yea, that makes sense. I haven't gotten into that part yet. Still only using it as a way of just making sure my datastructure remains in the format I expect/need.
Are you using conform
or valid?
or some other way of checking against the spec?
this is all part of my first GUI app using re-frame so its validating the main app-db as the UI changes
Since spec checks are all runtime, if you really need to “assert” your data structure has a specific key, you can always spec it as any?
and if you use something as a channel and it isn’t, it’s going to blow up at runtime anyway.
I would be a bit suspicious of data including a channel tho’…
after all, that’s not data you can serialize and pass around so it’s not really data…
(I may be taking a bit of a purist view here — not sure how others feel?)
can someone point out my (probable) misuse of multi-spec? what i’m trying to go for, in plain english, is to specify a map where some key’s spec depends on the value of another key. If key a is one of “cat” or “dog”, key b should be spec’d differently in either case.
(defmulti b-depends-on-a :a)
(defmethod b-depends-on-a "cat" [_] (s/or :cat-only-thing ::cat-thing
:cat-or-dog-thing ::shared-things))
(defmethod b-depends-on-a "dog" [_] (s/or :dog-only-thing ::dog-thing
:cat-or-dog-thing ::shared-things))
(s/def ::a #{"cat" "dog"})
(s/def ::b (s/multi-spec b-depends-on-a :a))
(s/def ::foo (s/keys :req-un [::a
::b]))
I would expect to be able to run something like (s/conform :: foo {:a "cat" :b ...})
or (s/conform :: foo {:a "dog" :b ...})
and have the response [:cat-only-thing …]
or [:dog-only-thing …]
or [:cat-or-dog-thing ...]
depending on the value of b.
I think your multi-spec needs to be a hash map that has a key :a
So ::b
would be spec a hash map — do ::cat-thing
etc spec hash maps?
i’ve been banging my head on that page for quite a while now .. i dont’ really understand the ‘tag’ argument to be honest
This https://clojure.org/guides/spec#_multi_spec also shows the multi-specs as being hash maps.
the call to s/multi-spec event-type
in that example is calling on a defmulti event-type
, unless I’m misunderstanding
The :event/type
key is common to all the (hash maps) that are valid for that set of specs.
In your example, you should have a hash map that has a key :a
that is the “tag” and it can have the value "cat"
or "dog"
and the defmethod
s should returns specs for the cat, or dog, hash map that includes the tag :a
.
oh you mean the object on which i’m invoking the spec? i.e. s/conform ::foo {:a "cat" (or "dog") :b something-else}
?
Right… so your defmethod
s should be (s/keys :req-un [::a :cat/b])
and (s/keys :req-un [::a :dog/b])
then the multi-spec
selects which spec to use based on the tag :a
Depends what ::cat-thing
, ::dog-thing
, and ::shared-things
are…
which is something like “if the multimethod matches on ‘cat’, return a spec which is an (s/or …)
so you are saying ::cat-thing, ::dog-thing, ::shared-things would have to be maps containing the key :a, correct?
What are you trying to do?
If you explain how you want :b
to vary, I can probably show you…
but depending on the value of :operation
, the spec for status should be “limited”. that is, given the idle
operation, the available statuses should only be one of #{:a :b :c}
, and given the downloading
operation, statuses should only be one of #{:c :d :e}
(note the overlap)
the (s/or ..)
comes into play because there is some overlap between :downloading / :patching / :idle, but only 1-2 states
Sure… so you need to spec the different types of status values, e.g., (s/def :downloading/status #{:c :d :e})
and (s/def :idle/status #{:a :b :c})
and then your defmethod
would return (s/keys :req-un [::operation :downloading/status])
for the downloading branch and (s/keys :req-un [::operation :idle/status])
for the idle branch
oof, so all this pain and confusion because i was trying to do an s/or
(to union) where I should have just defined the full sets outright, regardless of overlaps?
ill give that a whirl and get back to you.. regardless of the outcome thanks for your help
(s/def ::operation #{:idle :downloading :patching})
(s/def :idle/status #{:a :b :c})
(s/def :downloading/status #{:c :d :e})
(s/def :patching/status #{:a :c :f})
(defmulti operation-spec :operation)
(defmethod operation-spec :idle
[_] (s/keys :req-un [::operation :idle/status]))
(defmethod operation-spec :downloading
[_] (s/keys :req-un [::operation :downloading/status]))
(defmethod operation-spec :patching
[_] (s/keys :req-un [::operation :patching/status]))
(s/def ::machine (s/multi-spec operation-spec :operation))
(s/conform ::machine {:operation :idle :status :a})
succeeds
(s/conform ::machine {:operation :idle :status :d})
fails
(s/conform ::machine {:operation :downloading :status :a})
succeeds
well it’s good to know now that defmethod has to return “the original match” + the spec, not some hob-goblin pseudo spec
Thanks for the question — I hadn’t played with multi-spec
before and this gave me a good opportunity to try it out.
The different branches could also have different keys etc.
And, indeed, doesn’t have to be a hash map but then the tag function would need to be something other than a simple keyword.
so does the tag arg on (multi-spec spec tag)
have to be identical to the fn/kwd in defmulti name fn/kwd
?
not really sure why the tag is there.. unless its a way to generate fields under a different key from the one of the multi-match?
random thought: i kinda wish there was a way to alias keys, just as there is a way to alias attributes in datomic - would help with refactoring and “subtyping”
@lwhorton I believe the tag is needed in multi-spec
so that the generator can figure out what key to work with.
(since defmulti
could take a function etc)
i see .. if you still have that ^above code in a register, what happens when you try to generate some samples?
If defmulti
was some complex selector function, the argument passed to multi-spec
would have to kind of reverse that I think…
([{:operation :idle, :status :c} {:operation :idle, :status :c}]
[{:operation :patching, :status :c}
{:operation :patching, :status :c}]
[{:operation :downloading, :status :c}
{:operation :downloading, :status :c}]
[{:operation :idle, :status :b} {:operation :idle, :status :b}]
[{:operation :downloading, :status :e}
{:operation :downloading, :status :e}]
[{:operation :patching, :status :f}
{:operation :patching, :status :f}]
[{:operation :idle, :status :c} {:operation :idle, :status :c}]
[{:operation :idle, :status :b} {:operation :idle, :status :b}]
[{:operation :patching, :status :c}
{:operation :patching, :status :c}]
[{:operation :idle, :status :b} {:operation :idle, :status :b}])
from s/exercise
Yup, I pretty much always try to s/exercise
my specs as a sanity check, and also to see the sort of crazy stuff they can produce. We use regex fairly heavily so we rely on @gfredericks test.chuck
which has a generator for regex string specs — awesome stuff!