Fork me on GitHub
#clojure-spec
<
2016-08-01
>
gfredericks03:08:53

@lvh I was pondering having an "experimental" namespace, so we could probably start it there (c.g.schpec.expermental.json-schema)

gfredericks03:08:37

schpec depending on chuck is fine and probably inevitable

rickmoynihan09:08:24

@lvh: @gfredericks: Sorry, just seen the messages above about JSON Schema / spec... curious what you're doing exactly, it sounds interesting and useful!

mpenet09:08:48

same here, I'd love to see the work in progress

mpenet10:08:53

I started to dump utils I am using in our projects in https://github.com/mpenet/spex, I might make a couple of PR if there's an agreed "spec utils" project in the works (and interest)

lvh13:08:53

I’m writing a fn that takes a JSON Schema and produces an equivalent spec

dominicm14:08:39

@lvh: How possible would the reverse be do you think?

gfredericks14:08:00

should be possible for a subset of specs presumably

lvh15:08:30

@dominicm: Trickier, but like gfredericks said possible for some subset of specs; the biggest problem is that you presumably need to statically determine what a particular spec pred means, and then see how much of that you can express in JSON Schema

lvh15:08:04

it’s not going to be very nice json schema (probably larger, decent amount of duplication) but as long as you treat it like a compilation product and don’t care about bidirectional access that seems fine

dominicm15:08:16

Yeah, that was my thoughts also.

gfredericks15:08:23

the fact the predicates are stored as symbols that can't be reliably mapped to vars seems problematic for doing this kind of thing robustly

dominicm15:08:04

Could you say :foo/bar is this part of JSON schema?

rickmoynihan16:08:27

a symbol is a namespace qualified value too though right? 'clojure.core/int

gfredericks16:08:24

Not necessarily

gfredericks16:08:06

The mapping from symbols to vars is entirely relative to the current namespace setup

gfredericks16:08:00

You can setup your namespace so that clojure.core/int resolves to something entirely different

gfredericks16:08:31

But in any case I think spec would only store int? generally

rickmoynihan16:08:53

surely a macro could resolve it to its bound value though by inspecting ns-map etc... though

arohner16:08:21

(s/form (s/spec int?))
clojure.core/int?

arohner16:08:32

yes, you could set up your namespace so that int? redirects to something else, but spec does store the ns

eggsyntax16:08:20

@lvh: @seancorfield -- I just had occasion to write a spec for bigints, and was surprised to discover that (int? (bigint 1)) returns false. So I went with:

(s/def ::bigint (s/with-gen #(instance? clojure.lang.BigInt %)
                  (fn [] (gen/fmap #(bigint %) (s/gen int?)))))
Just as an FYI followup.

lvh16:08:49

What does integer? do?

lvh16:08:03

I find the two pretty confusing still :)

eggsyntax16:08:57

int? has to be fixed-precision. integer? just has to represent an integer, I guess. So I may be able to simplify that 🙂

eggsyntax16:08:40

Yes indeed, this works:

(s/def ::bigint (s/with-gen integer? 
                  (fn [] (gen/fmap #(bigint %) (s/gen int?)))))

eggsyntax16:08:50

Thanks @lvh, I would have missed that.

seancorfield16:08:21

@eggsyntax: and why doesn’t (s/def ::bigint (s/and integer? (s/conformer bigint))) work for you?

seancorfield16:08:49

(all I changed from my original suggestion was to use integer? instead of int? there)

seancorfield16:08:17

I guess the question is should (s/conform ::bigint 1) be 1N or :clojure.spec/invalid?

eggsyntax16:08:50

Yeah, good question. For my purposes, sure. But I could imagine going the other way in another context.

eggsyntax16:08:10

The conformer version works too, & is nicer actually, I'll go with that 😄

eggsyntax16:08:10

I mainly just meant to illustrate that it had to be integer? rather than int?

seancorfield17:08:45

Your predicate — #(instance? clojure.lang.BigInt %) — is not the same as integer? tho’. Your predicate rejects 1. That was my question.

seancorfield17:08:53

(and (s/exercise integer?) doesn’t seem to generate bigints in my quick tests)

seancorfield17:08:35

clojure.spec encourages us to be specific 🙂

gfredericks17:08:36

w.r.t. generating bigints, I ran into a bug once that didn't surface until the bigints were larger than Double/MAX_VALUE

gfredericks17:08:29

which makes it interesting to consider how large of a bigint a general bigint generator should test by default

gfredericks17:08:49

presumably at least bigger than Long/MAX_VALUE

gfredericks17:08:53

but Double/MAX_VALUE is quite large

dominicm18:08:43

@gfredericks: I thought some of the automatic tests were around max/min values?

gfredericks18:08:00

you're asking about default generator behavior?

gfredericks18:08:22

test.check doesn't have anything like that currently besides favoring small numbers

gfredericks18:08:37

and nan/infinity

dominicm18:08:57

Ah right. I was operating based on conference talks. That's unusual.

gfredericks18:08:01

I'm not sure what the best way to do that is

gfredericks18:08:49

e.g., if Double/MAX_VALUE should be tested frequently, does that mean the value right below it should be too? or just make that one special? and should it be easy to "shrink" to Double/MAX_VALUE from something more random? or should we always shrink downward?

glv18:08:10

@gfredericks: I don’t know if you saw the discussion I started a couple of weeks ago about the growth rate on the default spec generators … but the conclusion was that we definitely need integer generators for sizes (that grow slowly and have reasonable bounds) and generators for values used in mathematical functions (that grow rapidly and as large as they can get). Those already exist in test.check (although they’re not labeled/documented for those uses) but you have to jump through more hoops to get sensible bounds in spec.

gfredericks18:08:31

I think I saw that, yeah

gfredericks18:08:37

I wish I could just rename a bunch of generators

gfredericks18:08:55

A docstring overhaul could be good at least

xcthulhu18:08:29

@gfredericks: You probably weren't considering supporting goog.math.Integers, but if you were you might be interested to know that multiplication is broken for them https://github.com/google/closure-library/issues/703

gfredericks18:08:40

xcthulhu: ha, I fixed a couple division bugs in that class a while back

xcthulhu18:08:03

Oh man thank you I did notice that division works!

gfredericks18:08:17

I've definitely thought about Integer but I'm not sure how it ought to fit since it's pretty 2nd class in the cljs arithmetic world

gfredericks18:08:28

could definitely add some stuff in test.chuck for it though

xcthulhu18:08:10

Well... they don't work... when I need bigintegers I use SJCL and am generally disappointed.

gfredericks18:08:23

what doesn't work?

xcthulhu18:08:28

Multiplication

gfredericks18:08:38

that might be a recent bug

xcthulhu18:08:44

I ran into that particular bug when I was implementing modular inverses.

gfredericks18:08:51

I did a bunch of property-based testing on Integer in https://github.com/gfredericks/exact

gfredericks18:08:56

which is where I ran into the division bugs

gfredericks18:08:15

so either I tested on an earlier version w/ the mult bugs or else they're hard to find

xcthulhu18:08:16

Cool maybe I should port the modular inverse code I wrote since it makes great tests.

xcthulhu18:08:55

I didn't have time to spelunk goog.math.Integer to figure out the problem.

gfredericks18:08:58

I'll probably take a look at it; I've already been through their contribution process

xcthulhu18:08:53

This is a cool library btw. Would you ever want an is-prime? function? Miller-Rabin is rather fast provided you accept the generalized Riemann hypothesis is true.

gfredericks18:08:47

well you should probably call it probable-prime? then; but I'm not opposed to a namespace for number theory stuff if you find that useful

glv19:08:28

@gfredericks: Yeah, I think for test.check a documentation update is fine … the right generators are there, it’s just not obvious to a newcomer how different those two use cases are.

glv19:08:38

… or at least, to this newcomer.

gfredericks20:08:13

yeah; communicating how sizing works in general is kind of difficult

pvinis21:08:56

hello. i want to have something like

:nav {:bla 1
      :foo [{:ok true
             :sure 0}]}
how would i do the s/def on that?

lvh21:08:47

Is there a way in clojure.spec to specify xor of schemas? JSON Schema has it. Specifically, there’s a way to say that something must match all these schemas, or at least one of them, or exactly one of them (the latter being the xor I’m talking about).

pvinis21:08:10

also, what happens if i have a key like :ok in some other place, and its a string there? how do i s/def that..?

lvh21:08:20

I guess I’ll just do it with and and a predicate

gfredericks21:08:43

@lvh: add that to schpec

lvh21:08:57

sounds good

lvh22:08:53

is there a way for a pred that fails to communicate something about why it failed? specifically, i’m implementing xor as described above (aka one-of), I’d like to say the name of the spec that failed the pred

eggsyntax22:08:46

Specifically the predicate itself? For a spec it's just (s/explain ::the-spec failing-input)

lvh22:08:33

yeah; the predicate is checking that something is invalid against a particular spec

lvh22:08:40

and spec doesn’t appear to support negation natively right now

eggsyntax22:08:03

For a pred you can do (s/explain (s/spec my-pred) failing-data)

lvh23:08:02

eggsyntax: Sorry, I’m not being very clear. I know how to get spec’s opinion on why the data is bad; I’m trying to improve what s/explain produces when you give it failing data

eggsyntax23:08:17

@lvh: ah, gotcha. Maybe you can get more detail by splitting it into multiple subspecs? Dunno, just guessing.