Fork me on GitHub
#clojure-spec
<
2016-07-08
>
donaldball00:07:06

I started a library with some ostensibly generally useful specs, e.g.: https://github.com/SparkFund/useful-specs/blob/master/src/specs/internet.clj

donaldball00:07:31

Would very much appreciate feedback on design, etc. It’s my second day using spec in anger, as it were.

Oliver George01:07:37

it'd be nice if there was a way to modify how instrument reports. the default spec-checking-fn can be super verbose when turning values to str for the ex-info message.

Oliver George01:07:11

Here's my quick workaround. Not perfect but gives me something nicer to investigate: https://gist.github.com/olivergeorge/c4a26886b7bbd05f2e9018614a2213ff

Oliver George01:07:49

I rely on cljs-devtools to make the console.debug output to be expadable

Alex Miller (Clojure team)02:07:41

@glv I have much to report re the “hybrid map” stuff, but not quite ready yet to do so

Alex Miller (Clojure team)02:07:08

have been working on it much of yesterday and today

Alex Miller (Clojure team)02:07:09

@olivergeorge: you have explain-data ex-data coming back in the exception - why not write a function that customizes output from that, rather than hacking spec-checking-fn?

glv02:07:28

That tells me that it's a legit problem that y'all are taking seriously and working on, so I'm cool with that. :-)

bbloom02:07:57

@glv: There some considerable advantages to just using another map. I’m curious what @alexmiller is working on, but still: you should just add a :grid key or similar

bbloom02:07:21

what if you want to be able to represent a board without a starting coordinate?

bbloom02:07:40

you can easily take a grid and merge in some extra options, but how easy is it to dissoc them?

bbloom02:07:05

what if you want to change the representation? now you’ve coupled your param spec with your grid spec

bbloom02:07:51

what advantage is there to abusing the fact that keywords and grid coordinates are disjoint? slightly shorter syntax? seems not worth the complexity

glv02:07:56

This isn't the representation of the grid; it's the result of a particular kind of analysis of the grid, which always, inherently has a starting coord.

bbloom02:07:38

composite structures come in two flavors: homogenous and heterogenous

bbloom02:07:52

of course if you have disjoint key sets, you can combine them

bbloom02:07:56

but it’s just clearer to keep them separate

bbloom02:07:38

also when combined, you can’t enumerate keys without filtering extra keys

glv02:07:40

I'm certainly willing to entertain arguments that I should push the coord->distance mapping down a level.

bbloom02:07:59

i guess i’m making such arguments 😛

bbloom02:07:13

but more importantly: what advantage is there to flattening it?

glv02:07:21

So far, I don't find them convincing. :-)

glv02:07:49

I see three advantages.

Alex Miller (Clojure team)02:07:58

@glv totally legit - I have a couple different solutions with existing tools and something new for just this problem that may help if it comes together

bbloom02:07:52

i’m not saying hybrid maps are never useful

bbloom02:07:09

i’m saying that it’s pretty rare and barring additional info, i don’t think you’ve found a justified use case 🙂

glv02:07:01

1: the coord->distance mapping is the primary result of the algorithm. The ::start-coord, ::max-coord, and ::max-distance entries are annotations that could be reconstructed (with some cost) if they weren't present. But the only way to make a non-hybrid map would be to leave the secondary annotations at the top level and push the primary info down into a nested map. That doesn't seem right to me.

bbloom02:07:08

you can return those keys + a ::grid key and then create a helper function which simply extracts the grid key — then the caller can decide what is “primary” or not

glv02:07:50

2: the code that's involved—both the code that builds the map and that which uses it—would get more complex and clumsy (and slower) that way

bbloom02:07:37

really? how?

glv02:07:13

3: That's the way it is now, and it has never caused a hint of a problem; the only reason to change would be to accommodate clojure.spec.

Alex Miller (Clojure team)02:07:41

I don’t think it’s that rare/weird

Alex Miller (Clojure team)02:07:59

others (including me) have run into same problem

glv02:07:41

How? Isn't that obvious? The vast majority of the updates and accesses will be to the coord->distance mapping, so pushing that a level deeper means extra lookups in the vast majority of cases.

bbloom02:07:04

alexmiller: any examples you can share? i don’t think i’ve ever encountered such a hybrid map that didn’t (surprisingly quickly) need a (->> hybrid keys (filter #(… that could just be replaced by (-> hybrid :foo keys)

bbloom02:07:22

@glv: just build the grid and then assoc the extra keys in at the end

Alex Miller (Clojure team)02:07:47

sure, I ran into it with map destructuring (so, a syntax example) - the base map is symbol->any but also special options like :keys, :syms, :or, :as etc

bbloom02:07:17

alexmiller: heh, yeeaaaah i was wondering if you were going to mention macros. destructuring is like the ultimate core macro 😉

bbloom02:07:47

@alexmiller it’s funny b/c syntax is the one place where “just a little bit shorter syntax” is totally worth it for a common operation

glv02:07:57

@bbloom: even if that were sensible (it's not) it would only help on the creation side of the process, not the use side.

bbloom02:07:17

(:grid (let [start 123, m …build up m here….] {:start start :m m}))

glv02:07:25

It's not sensible because keeping track of those values through the process (so that you could assoc them in at the end) would be clumsy compared to just tracking them in the map as I go.

bbloom02:07:26

seems pretty sensible to me 🙂

bbloom02:07:02

er i meant: :grid m

glv02:07:05

::start-coord is predefined; the other two emerge out of the process itself.

bbloom02:07:19

i’d use mutation 😉

bbloom02:07:04

@alexmiller: one thing i’ve discovered tinkering on language design: it turns out that language designers have to do the things that they tell other people not to do, heh

glv02:07:24

cf. Stu's recent tweet about why clojure.core can't be idiomatic.

bbloom02:07:08

anyway, @glv definitely wait to see what alex et al come up with. the emitting things during the middle is IMO almost a reason to abuse the key space 🙂

glv02:07:55

"abuse" :-)

bbloom02:07:38

i’m not above loaded language 😛

bbloom02:07:00

thanks for the discussion! gotta run

Oliver George06:07:10

@alexmiller thanks for the suggestion. I would love to do that but don't know how. I presume you are suggesting a big try/catch around my code... my code is typically om/re-frame UI stuff which complicates that.

Oliver George06:07:28

Perhaps I'm missing something obvious

donaldball14:07:13

Just to confirm, we don’t yet have a way to unite the specs for e.g. homogenous (`map-of`) and heterogenous (`keys`) maps?

glv14:07:55

Which isn't bad at all, but could be made more expressive.

Alex Miller (Clojure team)14:07:45

b/c it turns it from a declaration of truth into a process

Alex Miller (Clojure team)14:07:41

there are a couple of ways to do it with existing tools

Alex Miller (Clojure team)14:07:30

one other way is to treat the map as a coll-of s/or of s/tuples (where each tuple describes different kinds of map entries). For cases where the “hybrid”-ness does not include registered attributes, this is probably a good choice. So something like a map that was either string->string or number->number.

Alex Miller (Clojure team)14:07:17

you can handle registered attributes in the same way, but then you lose the goodness of s/keys and the attribute semantics

Alex Miller (Clojure team)14:07:03

so in that case, you want to s/merge an s/keys (for the registered attributes) with something that more accurately states the truth of the map definition

Alex Miller (Clojure team)14:07:30

sorry for not being detailed here - I hope to write a blog with the detail

Alex Miller (Clojure team)14:07:14

and then I’m working on something new that will make this a little more palatable, but it’s not quite done yet

glv14:07:53

Well, compared to my attempt at a solution, it’s not bad. 🙂

eggsyntax17:07:07

In 1.9, is there a way to disable namespaced-map output (eg for data that's going to be parsed by a process that's unaware of the idiom)? ie to get output like {:foo/bar 1} rather than #:foo{:bar 1}? [Edit:simplify example]

eggsyntax17:07:35

Answer, if anyone's curious: CLJ-1967 will address this. For now ya gotta hack it if ya need it. http://dev.clojure.org/jira/browse/CLJ-1967

donaldball18:07:00

Are double-in and friends macros so that clojure core doesn’t require test check or as a performance optimization?

donaldball19:07:47

exercise-fn is pretty rad, but correct me if I’m wrong, it looks like if the fn in question has an options map specified with s/keys using :opt or :opt-un, the chances of getting a map with those keys is… small

donaldball19:07:21

Should the generator for s/keys bias towards optional keys’ presence?

jr20:07:25

I think they are macros because they build on other macros (like spec)

stuarthalloway20:07:35

@donaldball: double-in is a macro to capture the form for error reporting, IIRC

Alex Miller (Clojure team)20:07:35

in general, all of the spec creators are macros to capture forms

donaldball20:07:29

I’m afraid the implication is not clear to me 😕

donaldball20:07:01

What am I missing by (lazily) writing it as a fn instead of a macro?

Alex Miller (Clojure team)20:07:02

(s/explain (s/double-in :min 0.0 :max 5.0) 20.0)
val: 20.0 fails predicate: (<= % 5.0)
=> nil
(s/explain (decimal-in :min 0.0 :max 5.0) 20.0)
nil nil 0.0 5.0
val: 20.0 fails predicate: pred

Alex Miller (Clojure team)20:07:48

by using let from test.check, you’ve also made test.check a production time dependency for you

Alex Miller (Clojure team)20:07:00

(which is a bummer given how nice let is - I keep wanting it too)

Alex Miller (Clojure team)20:07:24

but you can easily rewrite that with gen/fmap

donaldball20:07:33

Yeah, I plan to replace it with fmap … jinx

Alex Miller (Clojure team)20:07:30

double-in as a macro (and spec under it) allows the form of the predicate to be captured as a form as well as evaluated as code

donaldball20:07:39

Thanks, that makes good sense

donaldball20:07:59

Btw any opinion on my optional keys generator observation?

Alex Miller (Clojure team)20:07:17

did you maybe not generate enough to tell?

Alex Miller (Clojure team)20:07:35

(gen/sample (s/gen (s/keys :req [::a ::b] :opt [::c ::d])))
=>
(#:spec.examples.guide
 {:a 0, :b 0}
 #:spec.examples.guide
 {:a 0, :b 0}
 #:spec.examples.guide
 {:a 0, :b -2, :d -1, :c -1}
 #:spec.examples.guide
 {:a 0, :b -2, :c -2}
 #:spec.examples.guide
 {:a 3, :b -1}
 #:spec.examples.guide
 {:a 0, :b 3, :c 0}
 #:spec.examples.guide
 {:a 13, :b -29}
 #:spec.examples.guide
 {:a -17, :b -1}
 #:spec.examples.guide
 {:a -12, :b 4, :c 0}
 #:spec.examples.guide
 {:a 5, :b 5, :c -8, :d -9})

Alex Miller (Clojure team)20:07:45

I see optionals showing up pretty regularly there

donaldball20:07:46

Hmm, maybe I’m doing something else wrong. Thanks.

Alex Miller (Clojure team)20:07:22

I’m not discounting the possibility of something wrong either :)

Alex Miller (Clojure team)20:07:37

just that it’s not obvious to me

donaldball20:07:54

Hmm, looks like keys* and keys have very different behavior

donaldball20:07:15

(gen/sample (s/gen (s/keys* :opt-un [::foo])))
(() () (:_.c/*! [[{{} (Kk.B/?Q)}]]) (:AC!e._*0_/V ((({:*g #uuid "99a8b1d6-d3c3-40f7-a7c9-0fd7beaf14e8"})))) (:q?a.*kS.K*q?/?*k ()) (:x8x6?A.!/qU_m? {#uuid "5f81c97f-0e94-4a95-bdae-ff2adaba5378" 5} :T9y17p.!7p/!5a8 () :?x.d95?.*.OVnzN_/?3?J* [] :_-+U.z.F2JXV6.U070/+9a2? [()] :xQb ["<[Ê©" .EM_N 1]) (:FtROj-1 [(19 :q0P:x5F:*T*41_G:H5_G:N-xN)] :?6IS.a+.AE4/vq2* [] :G.G+.d7Z!.w_+.a.C23/? () :?P*M/tj_Zg_ ()) (:XX [()] :F_Rv+.T+*n.Vp+2VR.w/+1tJQTg () :+R+9.r.per24*!9._NNt4*+.+.d5*T!.!9c/I {} :b.mw*bbbXT.*T+*-?g._h!!3_tg.Jvx.I.*++g14/+-S?Znvq () :K.H220.A9?g?h.s71-_+42.!9.-.z9iq4D*/jr83+S {} :x0w5+v.W!l6!s+/*5a? {0 -7}) (:Y!!D9w/J- [] :+!HS_l.XG+._XK*Bw_.UNW?.FPV20-3+_.*MYrk.Z1.pk-_!_/X*v {g._Y?!.QN+9g!/q.zC00Wj4 \space, Bj_t9- J?E**, true s!E-fSL4.R3Oj-A4._!vJ1!!.Vs5iz*n9a.-dM?5!9c.U.?7.*y/?-p+a!Lc, 1.0 :kP+b2Y:D*AA:74:U:-6yKx:phX!9??:I*Is, #uuid "f34904dd-a05a-4247-ad6d-1972baa0f58f" true, false -4.0, -5 false, \þ 5} :y*!G [:*_p?d?:-!:9f9:l0-3-Cw_5:2uT7f:46-_8P!h:C49*!*?gb:QJw:x8D \Ÿ 0.5 1/6] :mH.FH_PUY/Ik!3+z? ([{([]) [[*QN.]]}]) :ez0_.qeTiE_?.?-.d11.DrQDGZXu7.j9+7ou/N3M*T {} :!vdNK.-!!cb/Gv {ZS.K_d.fWR2/f+6.e?S. y4, v.w "|u\b"}) (:j65f.Z3r*.e8V7+6Y.L5_d.l+R8.dTp.L.l0n*9Z.AmW*_W8/c___ ({{} [()]})))
user> (gen/sample (s/gen (s/keys :opt-un [::foo])))
({} {} {} {:foo 1} {:foo 0.75} {:foo 0} {} {:foo -2} {:foo -8} {})

Alex Miller (Clojure team)20:07:16

oh, there was a bug in keys* gen that Rich fixed in master today

donaldball20:07:01

Sorry, I just assumed keys* was a pure wrapper macro

Alex Miller (Clojure team)20:07:04

it’s a composition of &, a conformer, and keys

ajss23:07:10

not sure if others are interested, but I've figured out the nice way to do dependent types for spec/fdef arguments : use with-gen on the :args parameter

(defn sum [a b] (+ a b)) 
(spec/fdef sum {:args (spec/with-gen (spec/tuple int? int?) (constantly (gen/fmap (fn [x] [(* x 2) (* x 3)]) gen/int)))})