Fork me on GitHub
#clojure-spec
<
2017-02-09
>
mike_ananev11:02:11

Hello! If i have regular fn which returns some number, how to wrap it into generator for spec?

not-raspberry11:02:54

Like, random number? This is pretty much not the point of generators. To my understanding, you're supposed to extend some existing generator by using it as a seed for your generator.

not-raspberry11:02:11

look at clojure.spec.gen/fmap

mike_ananev11:02:13

@not-raspberry i know about fmap. Can you provide some example how to use it in my question? I have fn (e.g. block-number-fn) which can produce block numbers. I have a spec for block. How to express block-number-fn as gen for block spec?

not-raspberry11:02:31

what's a block number?

mike_ananev11:02:40

it is objects

mike_ananev11:02:53

more complex structure

not-raspberry11:02:58

and without clojure.spec, how would you generate it? {:www (random) :zzz (random)}?

mike_ananev11:02:41

i have a Java constructor

mike_ananev11:02:57

(Block. "bla" "bla")

mike_ananev11:02:36

more simple example

not-raspberry11:02:52

can you generate it based on 1 random integer/double?

mike_ananev11:02:47

forget blocks

mike_ananev11:02:56

another example

mike_ananev11:02:08

i need gen for uuid

mike_ananev11:02:24

i know that gen for uuid already in spec

mike_ananev11:02:38

but if there no one?

mike_ananev11:02:54

(UUID/randomUUID) fn return uuid

mike_ananev11:02:18

how to wrap fn with (UUID/randomUUID) into gen?

not-raspberry11:02:20

Let's limit ourselves to textual uuids.

not-raspberry11:02:01

you write a spec for a character allowed in uuid, e.g. a set of those, you get a gen for free

mike_ananev11:02:38

i understand it, this sounds like workaround

not-raspberry11:02:47

then you write a spec for a coll of those chars with rexactly 24 chars (let's say uuids have 24 chars, i dunno)

mike_ananev11:02:40

how to wrap fn above into gen?

not-raspberry11:02:40

then you write your own genetrator with fmap that maps the previous generator with a function that inserts hyphens blocks with

not-raspberry11:02:22

you miss the point - don't use null-ary functions in specs. That makes them impossible to reproduce.

mike_ananev11:02:33

be reproducable is mandatory?

not-raspberry11:02:50

(defn gen-string-thats-somewhat-funny []                                                                            
  (sgen/fmap
    do-something-funny-with-string  
    (sgen/string-alphanumeric)))
where do-something-funny-with-string is not funny itself but the results of it should be. Also it's pure.

not-raspberry11:02:47

Is reproducible mandatory? Nothing is pretty much. But it's inconvenient to stand out.

not-raspberry12:02:31

I haven't found an easy way to write "non-pure" generators.

not-raspberry12:02:00

you can always discard the argument the function passed to fmap gets.

not-raspberry12:02:23

But the API overall seems to have been designed to discourage that.

gfredericks12:02:04

reproducibility and shrinking are the two big features you get by constructing generators using the combinators

mike_ananev12:02:10

@not-raspberry thanks. went to "hammock"

gfredericks12:02:57

@mike1452 to address your specific example, I would generate a UUID by generating a pair of longs and calling the UUID constructor

gfredericks12:02:23

which is essentially what the builtin UUID generator does

mike_ananev12:02:16

@gfredericks can you provide source of builtin gen?

mike_ananev12:02:57

i saw macros name in sources but can't find particular code for this

gfredericks12:02:01

as the comments say, the jvm code there is lower level because it's faster; that wouldn't be a normal way to build generators

mike_ananev12:02:02

@gfredericks thank you, I will try it

ericnormand15:02:36

I have a question about the clojure.spec guide

ericnormand15:02:43

it says: > The map spec never specifies the value spec for the attributes, only what attributes are required or optional.

ericnormand15:02:00

but then it does look like the values are being specified

ericnormand15:02:39

the next sentence is: > When conformance is checked on a map, it does two things - checking that the required attributes are included, and checking that every registered key has a conforming value.

not-raspberry15:02:15

you mean s/keys?

not-raspberry15:02:31

it specifies keys that specify values

joshjones15:02:56

@ericnormand so what is the question eric?

ericnormand15:02:07

they seem to contradict

ericnormand15:02:24

does the s/keys specify values or not?

ericnormand15:02:22

it seems that it does

joshjones15:02:15

no, it does not, only keys. however, when a map itself is validated or checked, the check is done to ensure that the keys specs are present, and that the values that those key specs specify are correct

ericnormand15:02:33

ok, that's what I'm saying

ericnormand15:02:43

the first sentence says it doesn't specify the value

ericnormand15:02:54

it's a direct contradiction

ericnormand15:02:04

you just said it does

joshjones15:02:10

i’ll give an example, just a min

ericnormand15:02:30

here's my example:

ericnormand15:02:32

(s/def ::name string?)

(s/def ::person (s/keys :req-un [::name]))

(s/explain ::person {:name 3})

ericnormand15:02:51

this fails with this message: In: [:name] val: 3 fails spec: :foo.core/name at: [:name] predicate: string?

ericnormand15:02:07

the value is clearly expected to be a string

ericnormand15:02:56

perhaps it's just a problem with the wording of that sentence

ericnormand15:02:03

> The map spec never specifies the value spec for the attributes, only what attributes are required or optional.

ericnormand15:02:25

to me, that means that s/keys only deals with existence of keys, not their values

joshjones15:02:05

you quoted the guide which says: “The map spec never specifies the value spec for the attributes” — is this the part that you say is confusing?

tbaldridge15:02:52

@ericnormand it means that s/keys never contains specs for the values, aside from the values spec'd by the keys themselves

ericnormand15:02:36

@tbaldridge I see, I totally misunderstood

joshjones15:02:40

(s/def ::mymap (s/keys :req [::mykey]))
(s/def ::mykey number?)
::mymap does not specify the value spec for the attribute above. It only specifies which key is required.

tbaldridge15:02:55

My favorite analogy: A "Car" is (s/keys [::wheels ::seats ::engine]), doesn't say what a ::wheel is, it just says "A car contains these things"

ericnormand15:02:57

but it implicitly says the value has to be a number

joshjones15:02:58

::mykey can change all day long, and the spec for ::mymap doesn’t change

joshjones15:02:11

the spec for the KEY says the value has to be a number, NOT the spec for the map

ericnormand15:02:28

but I still think it's unclear enough to warrant a rephrasing

tbaldridge15:02:35

But if you say "Is this a car?", it has to check to make sure that what you say is a ::wheel is actually a ::wheel

ericnormand15:02:44

if I had trouble with it, others probably will too

tbaldridge15:02:35

@ericnormand isn't that all covered in the spec guide? From the section on entity maps:

tbaldridge15:02:48

"Clojure programs rely heavily on passing around maps of data. A common approach in other libraries is to describe each entity type, combining both the keys it contains and the structure of their values. Rather than define attribute (key+value) specifications in the scope of the entity (the map), specs assign meaning to individual attributes, then collect them into maps using set semantics (on the keys). This approach allows us to start assigning (and sharing) semantics at the attribute level across our libraries and applications."

ericnormand15:02:34

that combined with the sentence I quoted allows for the interpretation I made

ericnormand15:02:44

that keys are what matter

ericnormand15:02:54

and that if you want to conform the value yourself, you can

ericnormand15:02:05

please take my suggestion for what it is

ericnormand15:02:17

I think it's unclear and would help others if it were rephrased

ericnormand15:02:27

I don't want to argue about it 🙂

joshjones15:02:38

I can understand why it’s confusing — (s/valid ::somemapspec {…}) could only check that the keys are there. then another (s/validextra ::somemapspec {…}) could validate both that the keys are present, and that the keys themselves are valid. i get it … but that’s not how it works

tbaldridge15:02:08

I don't understand: https://clojure.org/guides/spec already explains this about 5 different ways in the section labeled "Entity Maps". I'm not trying to argue, I'm just not sure how it could be any more explicit.

ericnormand15:02:46

it explains it five different ways, but the first way it is explained can be read the wrong way

ericnormand15:02:58

and it can color how the rest are interpreted

joshjones15:02:10

i’ve not heard anyone ask this question before, but I’m sure some people have wondered. more people are confused about how keys that are in the map are checked, even if they’re not given in :req and :req-un

ericnormand15:02:41

thanks for your help!

tbaldridge15:02:19

But I mean....that's the 3rd paragraph in the section. And your initial question about "the map spec never specifies" is tied to a code example as context.

tbaldridge15:02:31

To be frank, I think "read from top to bottom taking context into account" is assumed as part of the guide. At least that's my opinion.

mpenet15:02:58

yeah I got surprised by this one once (s/def ::foo string?) (s/valid? (s/keys) {::foo 1}) -> false

mpenet15:02:35

kinda makes sense after all, but I guess doesn't come to mind at first

tbaldridge15:02:30

Yeah, that one is a bit surprising

tbaldridge15:02:43

(didn't know about it till I read the guide just now)

mpenet15:02:21

it makes namespaced kw very specific to specs

mpenet15:02:44

maybe too much, but that's a matter of opinion I guess

mpenet15:02:46

one could argue it goes against the "specify what you want in there and ignore the extras" direction spec took

not-raspberry15:02:45

specifying against extra keys makes it impossible to write backwards-compatible specs

mpenet15:02:33

yeah the argument you'll hear is "create new keys"

mpenet15:02:42

which is again, arguably, a bit odd

not-raspberry15:02:42

(actually, future-proof, not backwards-compatible - it forces you to synchronize 2 services when adding an extra key)

mpenet15:02:31

@not-raspberry not sure we're talking about the same thing tho. I was referring to the example I mentioned, not to the fact that s/keys allows extra (unespecified) keys to be valid

not-raspberry15:02:22

I was just guessing the rationale for s/keys behaviour.

mpenet15:02:27

that is actually a good thing as you mention. the example I mentioned, is, odd.

tbaldridge15:02:06

It is an interesting topic. One one hand I do like that any value of my map will be valid after I've run it through conform.

joshjones15:02:31

@mpenet why is your example surprising?

tbaldridge15:02:39

On the other hand it is silent behavior (code that is executed not based on any specs I wrote in this context).

mpenet15:02:05

it ran valid? on a key that's not present in (s/keys) (the spec)

mpenet15:02:09

and failed

tbaldridge15:02:16

@joshjones because the overall language around spec has been "specify what you want to check, and allow the rest to be whatever".

mpenet15:02:45

in my book, this is a bit too surprising. Unless you read the doc very carefully you wont know about this behavior

joshjones15:02:03

but the spec guide clearly states that all keys in a map will be checked for conformance

mpenet15:02:29

As I said, it's arguable.

joshjones15:02:54

it’s definitely a “gotcha” but it’s something that is pretty quickly found, if you’re writing specs, IME

mpenet15:02:15

well the fact that @tbaldridge didn't know about it might be revealing 🙂

tbaldridge15:02:21

Funny enough I've been writing specs for months, and although I've encountered this, I've never understood it until now.

tbaldridge15:02:39

that's really my fault though for never reading the guide 🙂

joshjones15:02:22

the usual case is, write a key spec, write a map spec that doesn’t include that key, and put an invalid value for that key — that’ll uncover the “gotcha"

tbaldridge15:02:38

For better or worse, I tend to skim guides. So I like my APIs to be explicit. I can handle (s/keys :req [...] :also-check true), since I can go look up what :also-check does. It's silent behavior that can become a "gotcha"

mpenet15:02:00

I never encountered it personally, we have probably thousand of lines of spec and it's 99% un-* keys

joshjones15:02:14

i do agree that the behavior is not the most intuitive — but because i’m dense, i usually read the guides very thoroughly 😉

mpenet15:02:16

I think I saw it mentioned here

tbaldridge15:02:27

although all of this is also in the doc string, so double bad on me for not reading that either 🙂

joshjones15:02:59

well, normally when you write specs for regular situations, you’re not going to put a key in the map that is not spec’d in the map spec. but when you’re learning and “tinkering around”, trying to figure things out, it’s pretty normal thing to do imo

mpenet15:02:05

it makes kinda useless (other than for doc) :opt for instance

mpenet15:02:16

I think that's actually what the doc says

mpenet15:02:23

but odd nonetheless

mpenet15:02:35

ah, and gen

mpenet15:02:55

so not 100% useless

mpenet16:02:32

It has one big drawback tho, if you have a "giant map" and need to only validate 1 key it will still try to do its thing on all the others

mpenet16:02:55

this won't show on synthetic benchmarks like the one that's on github, but i am not sure the other frameworks do this

mpenet16:02:05

(pretty sure they don't actually)

tbaldridge16:02:49

:opt is also useful for gen, as I don't think s/keys gens other keys, only :req and :opt

bbloom19:02:43

Just a random note on the open-maps discussion: I discovered that TypeScript allows open js objects but has an interesting feature called “excess property checks” - essentially, it only disallows extra keys at compile-time for literal values. This way it catches typos etc, but doesn’t prevent future growth. A pretty neat solution, I think.

onetom21:02:48

We just had an awesome introduction to spec at the Clojure Remote 2017 conference! Adoption will increase for sure 🙂

Alex Miller (Clojure team)21:02:13

@ericnormand just skimming the backchat, is there something that needs to be clarified in the guide? if so, let me know or file an issue at https://github.com/clojure/clojure-site/issues

Alex Miller (Clojure team)22:02:46

issue welcome even if not sure

ericnormand22:02:46

thanks, alex!