Fork me on GitHub
#clojure-spec
<
2017-06-22
>
madstap01:06:24

In Rich's speculation keynote he mentioned that there are ways to check if backwards compatibility is broken in both set theory and regex theory. The set one would presumably be something like:

(defn breaking-change? [old-required-keys new-required-keys]
  (not (set/subset? new-required-keys old-required-keys)))
Are there any links to info about the regex part? Or am I misunderstanding something?

Alex Miller (Clojure team)02:06:41

madstap: the regex part is based on Matt Might’s papers with regex derivatives and I believe Rich has spent enough time with it to believe that the math there makes this computable. I’m having a strong sense of deja vu - have you asked this before?

madstap02:06:18

I'm pretty sure I haven't. Thanks for the pointers 🙂

Alex Miller (Clojure team)03:06:54

Must have been someone else, or my addled brain

sooheon01:06:17

Hey is there function to coerce all keys in a map to namespace-qualified keys, other than the #::{} syntax? I’m receiving an unqualified map from an API call, and would like to spec the result.

jjttjj01:06:10

@sooheon if you just need to spec un-namespaced keys you can just do (s/keys :req-un [::my-key1 ::my-key2]) to require those keys un-namespaced, ie {:my-key1 "x" :my-key2 "y"}

sooheon01:06:15

Thanks, I did see that in the docstring, and it did work

sooheon01:06:40

But now when I pass around this map and assoc my own keys to it, it will have a mix of qualified and un- keywords…

jjttjj01:06:11

s/keys can take vectors for both :req-un and :req if you have both namespace qualified and unqualified keys

sooheon01:06:34

Yeah, it’s just a bit of overhead for me

sooheon01:06:47

would have preferred to just go all namespaced, you know?

sooheon01:06:54

Thanks anyways :)

seancorfield02:06:26

@sooheon You could just use reduce-kv and produce a new map from the old one with namespace-qualified versions of the keys (assuming you want the same qualifier on all of them).

seancorfield02:06:44

Something like (reduce-kv (fn [m k v] (assoc m (keyword "qualifier" (name k)) v)) {} api-result)

sooheon02:06:04

ah cool! hadn’t thought of that. Was just hoping something like (into #::{}) would magically work

sooheon02:06:51

is this something that is recommended, btw? to use qualified keywords for everything?

seancorfield02:06:31

I think it’s a reasonable approach for handling domain data within your system, yes.

seancorfield02:06:14

Bear in mind tho’ that you probably want to keep those API result keys separate from your actual domain keys in order to avoid conflicts.

seancorfield02:06:59

So you might have :external.api/name and :my.domain/name with different specs.

seancorfield02:06:01

This is one of the reasons that the latest java.jdbc lets you specify :qualifier in the options on almost every call so you can get namespace-qualified keys in your SQL result hash maps.

sooheon02:06:45

I see. I think it’ll just take some experience (just like knowing how to split up and organize ns’es in the first place)

seancorfield02:06:40

Bear in mind that the namespace-qualifiers on keywords do not need to correspond to actual code namespaces so you have a lot of flexibility.

seancorfield02:06:50

For example, we use a qualifier of wsbilling for all entities that relate to our (World Singles) billing system but we don’t have a namespace called wsbilling.

seancorfield02:06:27

When we accept JSON/Clojure data from outside (APIs, databases), we can normalize it to whatever qualifier we want to distinguish it, within our system, from our billing data.

seancorfield02:06:58

(if this was a public Clojure API we’d use a reverse-domain-name prefix, like com.worldsingles.billing, I expect)

seancorfield02:06:22

You can choose how to keep unique groups of entity names separate from each other.

joshjones02:06:31

@sooheon @seancorfield I made a function a while back to namespace-qualify a map. here's what i have, if it's useful to anyone

(defn map->nsmap
  "Creates a namespaced-map from a standard one and a namespace (obj or string)"
  [m n]
  (reduce-kv (fn [acc k v]
               (let [new-kw (if (and (keyword? k)
                                     (not (qualified-keyword? k)))
                              (keyword (str n) (name k))
                              k) ]
                 (assoc acc new-kw v)))
             {} m))

sooheon02:06:58

Ah great, that’s very useful

sooheon02:06:11

@seancorfield I’ve still got to experience first hand how ns qualifying keys works with stuff like destructuring, and making sense of how I want to organize things, but thanks for the pointers

seancorfield02:06:19

That’s good for preserving existing name-qualification. Sometimes you need to override that. And of course my simplistic example only works for keys that you can call name on (and assumes you want all-keyword keys back out)!

seancorfield02:06:46

@sooheon Yeah, it takes some getting used to, after so many years of unqualified keywords!!

seancorfield02:06:30

In terms of destructuring:

user=> (let [{:foo/keys [a b c]} {:a 1 :foo/b 2 :c 3 :foo/c 4}] (println a b c))
nil 2 4

joshjones02:06:40

yes, i wrote it to behave the way the #: reader macro behaves, namely, it does not disturb non-qualified keys

joshjones02:06:42

#:foo{:a 1 :bar/b 1}
=> {:foo/a 1, :bar/b 1}

sooheon02:06:58

you mean already qualified ;)

joshjones02:06:23

and non-keyword keys

joshjones02:06:25

#:foo{:a 1 :bar/b 1 42 100}
=> {:foo/a 1, :bar/b 1, 42 100}

seancorfield02:06:56

also

user=> (let [{:keys [a b foo/c]} {:a 1 :foo/b 2 :c 3 :foo/c 4}] (println a b c))
1 nil 4
So you can put qualifiers in the key vector — and you get unqualified symbols back out!

sooheon02:06:32

how would you use the :: sugar w/in destructuring

seancorfield02:06:51

Like this:

user=> (let [{::keys [a b c]} {:a 1 :user/b 2 :c 3 :user/c 4}] (println a b c))
nil 2 4
Note that we’re in the user namespace so ::keys means :user/keys

seancorfield02:06:41

You can also do this (which I’m surprised is allowed):

user=> (let [{:keys [a ::b ::c]} {:a 1 :user/b 2 :c 3 :user/c 4}] (println a b c))
1 2 4

joshjones02:06:03

actually it seems to work by putting the :: in the vector of keys too, though i'm not sure it's a good idea

joshjones02:06:04

(let [{:keys [a b c]} {:a 1 :foo/b 2 :c 3 ::c 4}] (println a b c))
1 nil 3
=> nil
(let [{:keys [a b ::c]} {:a 1 :foo/b 2 :c 3 ::c 4}] (println a b c))
1 nil 4

sooheon02:06:20

so the last example would be the way to mix and match ::b :foo.bar/c and :d

seancorfield02:06:31

I expected that to be an error but apparently you can use keywords in the key vector? @alexmiller Is that supposed to be valid?

sooheon02:06:41

You can’t have multiple :keys ::keys and :foo/keys deconstructions separately at once, right? You’d do {:keys [a ::b foo/c]}

seancorfield02:06:25

That’s my understanding, yes.

Alex Miller (Clojure team)03:06:55

@seancorfield yes, we support keywords there so that you can use autoresolved keywords

seancorfield03:06:31

Is that documented? (I guess the docs could have been updated since I last looked at them… 🙂 )

Alex Miller (Clojure team)03:06:34

And you can have multiple keys destructurings at the same time for different namespaces

Alex Miller (Clojure team)03:06:47

It was in the changelog when it went in but that was a while ago

seancorfield03:06:55

Yup, the destructuring guide contains examples.

seancorfield03:06:12

Not of ::k directly but of ::p/name

Alex Miller (Clojure team)03:06:27

Rich and I worked on some updated docs for destructuring but I can't remember if those actually got all the way done

seancorfield03:06:00

It’s fairly comprehensive, TBH. Thank you. And Rich.

hlship15:06:09

@alexmiller From my issue in the tweet, here's the tail end of a very long stack trace:

clojure.spec.alpha/regex-spec-impl/reify/gen*                 alpha.clj: 1672
                        clojure.spec.alpha/re-gen                 alpha.clj: 1601
                              clojure.core/every?                  core.clj: 2572
                                clojure.core/next                  core.clj:   64
                                              ...                                
                              clojure.core/map/fn                  core.clj: 2657
              clojure.spec.alpha/re-gen/ggens/gen                 alpha.clj: 1582
                        clojure.spec.alpha/re-gen                 alpha.clj: 1597
                        clojure.spec.alpha/gensub                 alpha.clj:  275
                             clojure.core/ex-info                  core.clj: 4617
clojure.lang.ExceptionInfo: Unable to construct gen at: [:exception] for: (instance? java.lang.Throwable %)
    clojure.spec.alpha/failure: :no-gen
       clojure.spec.alpha/form: (clojure.core/fn [%] (clojure.core/instance? java.lang.Throwable %))
       clojure.spec.alpha/path: [:exception]

hlship15:06:15

What I don't get is why it is trying to create a generator. It just supposed to be validating inputs into our schema/compile function.

hlship15:06:53

Basically, the function I'm passing in doesn't have a fspec, so I'm going to add one and see if that helps.

hlship15:06:29

Nope. Bad guess. Providing my own fspec didn't work.

Alex Miller (Clojure team)16:06:29

Right, so that was my guess

Alex Miller (Clojure team)16:06:58

the way that fspec args are validated in instrumentation is by using the fspec’s generator to generate values and invoke the function

Alex Miller (Clojure team)16:06:52

and check the ret and fn spec to verify the function with random inputs is valid according to the fspec

Alex Miller (Clojure team)16:06:02

thus all fspec’s should have an args spec that generates

Alex Miller (Clojure team)16:06:31

the surprising bit here is that they need that not just for check, but also for instrument

Alex Miller (Clojure team)16:06:01

same is also true of fdef for check, but not for instrument

Alex Miller (Clojure team)16:06:51

(because checking an fdef involves generating using its args generator)

hlship16:06:40

(in a meeting, will digest and get back to you)

mpenet16:06:25

Just wondering, why not just wrapping args/ret of fspec fns with s/valid? when instrumented

mpenet16:06:00

It would fail "later" (invoke time) but might be more intuitive (and faster)

mpenet16:06:45

Could be an option

Alex Miller (Clojure team)17:06:16

not sure if Rich considered that or not, haven’t talked to him about it

mpenet12:06:42

alexmiller: just wondering: wouldn't it be better to keep https://dev.clojure.org/jira/browse/CLJ-1936 open then?

mpenet17:06:51

Not the first time this comes up: clj-1936

hlship17:06:05

So I'm a bit lost .... is there no way I can simply declare what my callback should look like without coming up with a generator for the callback?

Alex Miller (Clojure team)19:06:47

hlship: in short no, if you want to also validate the fdef. other options are to override the fdef args generator or to use a generator override on instrument

mpenet18:06:25

You can use fn? (meh)

hlship19:06:30

mpenet: Yes, that's where I'm headed.

Alex Miller (Clojure team)19:06:19

actually, ifn? is better if you go that route

hlship18:06:34

I almost don't care about validation as much as documentation of my Argos and result.

arohner19:06:25

(gen/sample (s/gen (s/spec ::clojure.core.specs.alpha/map-bindings)))
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.  clojure.core/ex-info (core.clj:4725)

arohner19:06:14

@gfredericks in core.specs.alpha

arohner19:06:26

the clojure.core specs that got split out into a separate lib when spec got split out

arohner19:06:34

@alexmiller I’m confused by the definition of ::core.specs/map-bindings. What does it mean for (s/every ... :into {}) I don’t see a guarantee that the every coll has an even number of elements, nor a requirement that the input coll is a map

arohner19:06:52

(this is coming up because I’m trying to run spectrum inference on clojure.core)

gfredericks19:06:58

@arohner every branch of map-bindings seems to be a pair, does that explain your confusion?

arohner19:06:28

(s/conform ::clojure.core.specs.alpha/map-bindings '[[foo bar]]) => [[foo bar]]

arohner19:06:54

it seems very weird to me that :into is used for generating, but the resulting conform doesn’t change

arohner19:06:37

the real source of my trouble is just below, when I have to handle (s/def ::map-binding-form (s/merge ::map-bindings ::map-special-binding))

arohner19:06:48

where s/merge takes maps

Alex Miller (Clojure team)19:06:19

sorry, just reading this now. where is your question at now?

Alex Miller (Clojure team)19:06:06

note that s/merge doesn’t flow conformed values so only the last map spec in the merge will matter for the conformed value

arohner19:06:11

::map-binding-forms takes two maps, because of s/merge, but map-bindings is not guaranteed to return maps, because it uses :into and not :kind

Alex Miller (Clojure team)19:06:25

I don’t agree with the last clause there?

Alex Miller (Clojure team)19:06:07

both :into and :kind affect s/every gen

arohner19:06:15

(s/conform ::clojure.core.specs.alpha/map-bindings '[[foo bar]]) is that expected?

Alex Miller (Clojure team)19:06:18

I’d say that’s not a supported map binding form

Alex Miller (Clojure team)19:06:31

I guess we could use :kind to narrow that

arohner19:06:51

right, spectrum is currently picky about that, and I’m trying to understand it / make it work

arohner19:06:05

spectrum says map-binding isn’t necessarily a map, so the merge isn’t guaranteed to work in all cases

Alex Miller (Clojure team)19:06:51

yeah, that seems good. if you want to file a jira, we can try to get that in next release

arohner20:06:01

sure. in CLJ, or does spec have it’s own project now?

Alex Miller (Clojure team)20:06:56

if you want to make a patch, that will be faster than me making it (as I can screen it)

Alex Miller (Clojure team)20:06:12

patch against core.specs.alpha of course

peeja21:06:27

What's the best way to define a function :args spec for a function which takes a single argument?

joshjones21:06:00

@peeja - same way as you do for any :args spec

(s/fdef your-func
        :args (s/cat :single-arg some-pred?)
...)

peeja21:06:15

So, still use s/cat then?

joshjones21:06:24

yes, as it's still a vector of one element

Alex Miller (Clojure team)22:06:54

agreed and to take that a step further, it’s useful to use a regex op (s/cat) here because the conformed value will be a map with the arg name as the key, and that’s useful in the :fn spec or in explanations