Fork me on GitHub
#clojure-spec
<
2019-02-07
>
borkdude12:02:23

I thought I had to make upgrading existing spec libraries less painful is to have something like a reader conditional that can dispatch on the version of spec. E.g.:

#?(:spec2 (:require [clojure.spec-alpha2] :as s) :default (:require [clojure.spec.alpha :as s]))
#?(:spec2 (s/def …) :default (s/def …))
This way a library can maintain compatibility with both versions of spec

dominicm16:02:15

(try
  (require '[clojure.spec-alpha2 :as s])
  (catch Exception e
    (require '[clojure.spec.alpha :as s])))
No? 😄

dominicm16:02:26

doesn't work for cljs though

borkdude16:02:34

that’s the big deal, CLJS

borkdude16:02:11

we could use a goog-define for it in CLJS, but then still you cannot do anything on the namespace declaration level

dominicm16:02:21

it's a shame that it doesn't work in cljs given the work that was done to make require work there.

borkdude16:02:43

that’s only for self-hosted AFAIK

borkdude16:02:48

and REPL usage

borkdude16:02:49

anyway, it would be nice to have a way have libraries support both versions. I bet it will be 2 branches and versions with “-spec2” suffixes for a while

dominicm16:02:55

yeah, I think you're right

dominicm16:02:31

you could use a macro, and perform the require voodoo at the clojure level perhaps :thinking_face:

dominicm16:02:36

but still not great

borkdude16:02:04

oh right, I think I saw someone “abuse” a data reader for that recently. Maybe it was @U050PJ2EU?

dominicm16:02:04

it would work in shadow-cljs, which has the ability to specify custom :keywords which run

borkdude13:02:16

will spec-alpha2 be released at the same time as clojure 1.11 as one bundle?

borkdude13:02:36

I have yet to test the latest SHA. Will update you, probably in the weekend.

joefromct18:02:04

can anyone point out something i may have missed… if i try to generate from a s/keys spec un-req i get all blank maps… but if i use the same with req i can generate data. Do you need a custom generator for anything un-req ?

seancorfield19:02:47

@joefromct

user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/def ::a int?)
:user/a
user=> (s/def ::b string?)
:user/b
user=> (s/exercise (s/keys :req-un [::a ::b]))
([{:a -1, :b ""} {:a -1, :b ""}] [{:a 0, :b ""} {:a 0, :b ""}] [{:a -1, :b "75"} {:a -1, :b "75"}] [{:a 0, :b "zW7"} {:a 0, :b "zW7"}] [{:a -3, :b ""} {:a -3, :b ""}] [{:a -2, :b "Hlf"} {:a -2, :b "Hlf"}] [{:a 11, :b "M8r"} {:a 11, :b "M8r"}] [{:a -4, :b "K9Dn"} {:a -4, :b "K9Dn"}] [{:a 44, :b "o2rAl"} {:a 44, :b "o2rAl"}] [{:a -4, :b "SzHY1pTSN"} {:a -4, :b "SzHY1pTSN"}])
user=> 

joefromct19:02:55

ha, yup. that was it. very funny.

joefromct19:02:08

attention to detail is important. thanks.

clojure
(s/def :my-test/name (s/and string? #(not= "" %)))
(s/def :my-test/developer (s/keys :req [:my-test/name ]))
(s/exercise :my-test/developer 2)
([#:my-test{:name "W"} #:my-test{:name "W"}] [#:my-test{:name "7"} #:my-test{:name "7"}])

;; d'oh `un-req` not a thing.  
(s/def :my-test/developer (s/keys :un-req [:my-test/name ]))
(s/exercise :my-test/developer 2)
([{} {}] [{} {}])

seancorfield19:02:47

I'm a bit surprised s/keys doesn't complain that all its known options are missing

user=> (s/exercise (s/keys))
([{} {}] [{} {}] [{} {}] [{} {}] [{} {}] [{} {}] [{} {}] [{} {}] [{} {}] [{} {}])
user=> 

Alex Miller (Clojure team)19:02:16

s/keys is a valid spec that will validate all registered keys in the map

joefromct19:02:25

yeah i guess i did come across in the guide the mention of (s/keys) was valid and i guess the rule to fn’s are “extra keys no-big-deal”

Alex Miller (Clojure team)19:02:30

but of course it doesn’t know how to gen

Alex Miller (Clojure team)19:02:16

just use this mnemonic … “I req-un the map should have a key like this”

👍 5
jimbob20:02:57

whats the idiomatic way to mandate presence of keys? related: whats also the idiomatic way to mandate the lack of extraneous / undefined keys?

jimbob20:02:40

i suppose for spec you can do something like (s/and (contains % #{mandated-keys})

borkdude20:02:03

mandate the presence of keys: req-un not allow extra keys: this is not according to the philosophy of spec/Clojure I think

jimbob20:02:42

interesting. thanks

taylor20:02:25

there are some code snippets around for disallowing "extra" keys though @ben.borders even though it's counter to the design philosophy

borkdude20:02:48

you can always hack around the design philosophy of Clojure 😛

jimbob20:02:10

right, i figured.. seems like i should probably try to allign more with the philosophy

jimbob20:02:21

i’m sure there are valid reasons for that.

jimbob20:02:27

would just like to read about it i suppose

Alex Miller (Clojure team)20:02:31

Rich did a whole talk about it

⁉️ 5
Alex Miller (Clojure team)20:02:06

grep for “code for growth”

taylor20:02:45

I think one of the metaphors is "if you're expecting a truck to deliver your TV, you shouldn't care what else is on the truck" or something like that

borkdude20:02:49

There have been some conversations about what spec is and isn’t suited for. E.g. it isn’t designed for coercion or “closed world” assumptions (not allowing extra keys), e.g. using it as API boundary protection

Alex Miller (Clojure team)20:02:27

I think the greater point is actually that you shouldn’t design your systems that way, and if you don’t, spec will mesh well :)

jimbob20:02:08

right.. we don’t now.. we allow our maps to have whatever values they have, and we grab the ones we care about

jimbob20:02:17

which tends to be most or all of them.

borkdude20:02:34

well, sometimes I don’t want to have crap in my jsonb field, but there’s always select-keys, etc.

👍 5
Alex Miller (Clojure team)20:02:45

and you might be interested in the new stuff coming in spec 2 around that https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/MaybeNot.md

👍 5
seancorfield20:02:41

> I think the greater point is actually that you shouldn’t design your systems that way, and if you don’t, spec will mesh well :) It's fine for a (REST) API to be passed a bunch of parameters it doesn't care about. It can just ignore them. Then spec works well for parameter validation.

Jp Soares13:02:10

I started to use REST API with compojure-api 2.0.0 that can coerce data with spec and generate swagger docs from it. I thought it would be the trend for specs, but reading this conversation maybe not. It's better to use compojure-api with schema to follow the clojure philosophy in REST?

seancorfield18:02:29

I don't understand your question : what do you mean "clojure philosophy in REST"?

seancorfield20:02:41

> well, sometimes I don’t want to have crap in my jsonb field, but there’s always select-keys, etc. And you can pull the "known" keys out of a spec fairly easily (usually!) so that you can narrow your set of fields down to just what's in the spec (we do this around database insertion where we have a spec for the row representation of data).

joefromct21:02:11

how would a spec look for the output (https://github.com/clojure/clojure/blob/master/src/clj/clojure/xml.clj#L78) from clojure.xml\parse look? Below is what i’m guessing at so far, but the ::content references ::element and vice versa… not sure how to handle this. (I’m not worried about namespaces at the moment): I think its tricky for me because it’s a hierarchy.

(s/def ::tag keyword?)
(s/def ::attrs (s/map-of keyword? string?
                         :conform-keys true))

(s/def ::content-strings (s/coll-of string?))
(s/def ::content-maps    (s/coll-of ::element?))

(s/def ::content
  (s/or :content-map ::content-maps 
        :content-string ::content-strings )) 

(s/def ::element (s/keys :req-un [::tag]
                         :opt-un [::content ::attrs]))

butterguns23:02:17

(gen/sample (s/gen string?) 10000)

butterguns23:02:51

...only gives me alphanumeric, for 10,000 iterations. Why is that?

butterguns23:02:04

No symbols / unicode

seancorfield23:02:27

That's probably all the string? generator is coded to produce.

seancorfield23:02:03

test.chuck has a regex string generator that can produce a much wider range of character values, if you need to test that.

seancorfield23:02:57

For example, here are some generated "email" strings, using that generator:

(["脀@Z.Gp" "脀@Z.Gp"] ["򫑟@OG.lr" "򫑟@OG.lr"] ["\"򒅨\"@z.DtS.K.Qpu" "\"򒅨\"@z.DtS.K.Qpu"] ["񜪐񕞐.򲏨򌲞򑠡󒓶@2N8.SuYn.FY.zBf" "񜪐񕞐.򲏨򌲞򑠡󒓶@2N8.SuYn.FY.zBf"] ["񈱢󔮢񑬧򹒠􎲱@3.QBB" "񈱢󔮢񑬧򹒠􎲱@3.QBB"] ["𸣄񨮤򁍣䱝󥼭.򐋾.򞘮.󦑁𣩷󽇱𖸃.쑆󎘅񮓶􌹛𖄃񌋶.𕺻󚣐񉈀󘅙򡓎򢞙@[3.61.526.313]" "𸣄񨮤򁍣䱝󥼭.򐋾.򞘮.󦑁𣩷󽇱𖸃.쑆󎘅񮓶􌹛𖄃񌋶.𕺻󚣐񉈀󘅙򡓎򢞙@[3.61.526.313]"] ["\"񹭃𰄖𿪵򙇑\"@[78.35.80.6]" "\"񹭃𰄖𿪵򙇑\"@[78.35.80.6]"] ["\"񑙬򨍸񁞻\"@[00.5.0.8]" "\"񑙬򨍸񁞻\"@[00.5.0.8]"] ["\"񱁩\"@-Npu4W.unVx6R.DdNnk.4.A6tM.y.EJFHfw92N.ecYbisymo" "\"񱁩\"@-Npu4W.unVx6R.DdNnk.4.A6tM.y.EJFHfw92N.ecYbisymo"] ["\"򵪶򐤆𶝟򃚭𫐊񝤬򋂼񺚷񆠧򍪳\"@[0.881.6.929]" "\"򵪶򐤆𶝟򃚭𫐊񝤬򋂼񺚷񆠧򍪳\"@[0.881.6.929]"])

seancorfield23:02:22

(my editor does not render them all correctly 😞)

butterguns23:02:56

... scrolling down to "Lets set up some Generators", he uses gen/string instead of string? and it produces the full range

seancorfield23:02:06

Ah, there you go!

butterguns23:02:20

Would be nice not to have to override all my string? specs with gen/string however 🙂

seancorfield23:02:18

I guess it depends on how important it is that you exercise your code with more exotic strings?

butterguns23:02:25

Yeah, I guess so. It just surprised me. Testing against exotic strings strikes me as exactly within generative testing's wheelhouse

seancorfield23:02:38

Well... it is but this is a case where you need to opt into it.

seancorfield23:02:23

It really depends on what parts of your data processing you need to stress test. If you only pass a string through your system and into the DB, you don't really care whether it's alphanumeric or all poop emojis.

seancorfield23:02:06

If you have specific string processing that does some sort of parsing, it might well be useful to stress test it with really exotic strings.

seancorfield23:02:36

Not everything is important to test to the same level.

seancorfield23:02:36

It might be more useful, for example, to use a string generator that only ever produces an empty string, a short readable string, or a giant long string.

Jp Soares13:02:10

I started to use REST API with compojure-api 2.0.0 that can coerce data with spec and generate swagger docs from it. I thought it would be the trend for specs, but reading this conversation maybe not. It's better to use compojure-api with schema to follow the clojure philosophy in REST?