I'm trying to spec something like s-expressions, and my current flailing in that direction involves a recursive s/tuple (something like

(s/def ::sexp
  (s/alt :arg integer?
            :expression (s/tuple fn?
                                              ::sexp
                                              (s/* ::sexp))))

s/cat?

then you can just replace ::sexp (s/* ::sexp) with (s/+ ::sexp) too

(s/def ::sexp (s/alt :arg integer? :expression (s/cat :f fn? :args (s/+ ::sexp))))

@alexmiller Thanks! When I gen/generate from an s/cat approach, though, I don't get a sequence of sequences, though, but just one flat one.

I was asking stuff in the beginners channel, but probably this is a better place for doing that. I posted this snippet of code, claiming that it doesn't work:

(let [arst {:a [1 2 3]
            :b {:c #{::z ::x ::c}}}]
  (s/keys :req-un (get-in arst [:b :c])))

(def my-bunch-of-stuff {:a {:b [:x :c :d]}
                      :z {:b vector?}}

(defmacro arst [type]
  (let [sp# (get-in my-bunch-of-stuff [type :b])]
    (if (fn? sp#)
      `(s/spec ~sp#)
      `(s/spec (s/keys :req-un ~(vec (map #(->> % str keyword) sp#)))))))

you need to use either a macro or eval:

(let [arst {:a [1 2 3]
                :b {:c #{::z ::x ::c}}}]
  (eval `(s/keys :req-un ~(get-in arst [:b :c]))))

(s/keys is a macro)

and you might actually need to call seq or vec on the result of the get-in, not sure s/keys will happily take a set as :req-un value

your macro should work (kinda, you need namespaced keys in your bunch-of-stuff and there's a paren missing there too)

I'm sorry for adding the macro after the request, I accidentally pressed enter before completing my post. My keys aren't namespaced - I "namespace" them with the (map #(->% str keyword) sp#) thing. What puzzles me is: why does this macro work if I try to do (arst :z) - which takes a function from the bunch-of-stuff and spits a spec out of it - and doesn't if I get a vector of un-namespaced keywords, "namespace" them with the dirty hack described above??? I (macroexpand '(arst :a)), and copy-paste what I get in REPL - and it works, but it doesn't if I just try to do (arst :a). I'm pretty sure that ~(vec (map #(->> % str keyword) sp#)) yields a vector of namespaced keywords...

it doesnt I think, it creates keys with ":foo" content with a leading :, so yeah ::foo but not namespaced per say

try running your spec, it will complain about this. Try changing your keys in bunch-of-stuff with ns keys you ll see

Oh... I see. Was able to make it work using this other horrible hack:

(defmacro arst [type]
  (let [sp# (get-in settings [type :b])]
    (if (fn? sp#)
      `(s/spec ~sp#)
      `(s/spec (s/keys :req-un ~(vec (map #(keyword (str *ns* "/" (name %))) sp#)))))))

how do I write a spec for an instance of Comparable?

(s/valid? (s/spec #(instance? Comparable %)) "arst")

thanks @vandr0iy . I’ll give that a try

that worked. Brilliant thanks.

If I want a generator that produces a wider range of vaues than the ones provided for double-in and int-in how would I go about getting one of those?

all the random numbers I’m getting are pretty small and I need to test against some big ones.

test.check has generators that allow you to specify a range

large-integer* and double* I think

cheers @mpenet I’ll have alook

(gen/sample (gen/double* {:min 1 :max 200})) still produces really small values

I’d like to see some values closer to 200. Is this possible?

Try sampling more values

ah ok.. looks generators probably aren’t going to be efficient enough for the way I was abusing them.

If I wanted a more evenly distributed generator would it be possible to write one?

@triss yes but you need to use the generators in test.check

@triss what's a uniform distribution over doubles?

@triss oh I think this is a sizing issue, sorry I didn't read back far enough

@gfredericks ah I can see why that might be awkward. rand seems more even than this though

@triss you should get lots of values close to 200, but gen/sample is deliberately only showing you small examples

@triss try (gen/sample ... 200)

is there any reason for why s/spec couldn't accept a third optional argument for defining a custom generator? I thin it would make it easier to statically analyze specs and might end up with some cool tooling use cases

@triss test.check has some subtleties with sizing that need to be documented better, and I have 45 minutes free right now so I think I will start on that

@gfredericks I still see a similar distribution. biased to low numbers

ah many thanks @gfredericks will be much appreciated

@triss it's not going to be uniform, but you should definitely get some larger numbers

@triss one reason I asked my original question about what a uniform distribution is, is because doubles themselves aren't uniform -- there are a lot more numbers between 1 and 50 than 50 and 200

so uniform might mean different things in different contexts

oh that’s interesting. because of the way they are represented in the machine?

you could say that

it's kind of inherent in the idea of floating point

I guess I’d like a distribution of Real numbers? does that make more sense?

^a more even distribution of real numbers

not really, but I think I know what you're trying to get at

@triss you could use large-integer to get this pretty easily

I’d love to see a flatter histogram!

or at least something close

yes that's a good way of putting it :)

ah brilliant will look at large-integer

(gen/let [x (gen/large-integer {:min 0 :max 200000000})] (/ x 1000000.0))

@triss ⇑ something like that

it won't get you every double in that range, but it might be okay for what you're doing

and it wouldn't be hard to make it fancier so that it hits most things

thanks. this will probably do nicely.

np

oh ha yes I forgot that

sorry

looks like I won’t be generating my population of genotypes with spec or generators.

or hang on I can write a custom one?

@triss try (gen/let [x (gen/choose 0 200000000)] (/ x 1000000.0))

@triss ⇑ these essentially are custom generators

it's all about composing lower-level generators to get what you want

gen/choose is one of the lowest-level ones, and gives you a uniform distribution over a range of integers

gen/choose is perfect for what I’m doing I think....

@triss now that I think about it, I might recommend the large-integer approach anyhow, but it's tricky to explain why

worst case you can cheat the thing with (gen/fmap #(do whatever you want) (gen/return nil))

ok - I need to scratch my head about composing these things for a while....

@triss this might be useful: https://github.com/clojure/test.check/blob/master/doc/generator-examples.md

wonderful thanks.

I have this one bookmarked 🙂 https://github.com/clojure/test.check/blob/master/doc/cheatsheet.md

@triss w.r.t. choose vs large-integer, the short explanation is that test.check has a strategy where it tries small things first and slowly tries larger and larger things; by using choose you're overriding that strategy, and that strategy is the reason you saw an uneven distribution with large-integer

which is best depends on your goals, but I'd say large-integer would be a good default

would you consider using generators outside of testing code foolish?

it seems very much tailored for testing.... (as the namespace indicates I suppose)

@triss clojure.spec uses them for a bit more than test.check does, but still for testing purposes; if you're using them for not-testing-at-all, it will be a little weird but not terrible; understanding the sizing subtleties could be more important though

I suppose based on that jungle music talk at the conj I should expect people will be using generators for all sorts of things

@triss the gen/generate function will probably be useful for you, since it accepts a size parameter

e.g., (repeatedly 200 #(gen/generate (gen/let ...) 200)) should give you the distribution you wanted even using large-integer

Guys, can someone help me with this: I have a long string of lorem ipsums, something like this "pellentesq dapib suscip liguldon posue augquaeti v tort ..." and now I need a spec with a generator that would generate a name of 3 to 5 words long by randomly pulling those words from that lorem-ipsum string. How do I do that?

(def s “pellentesq dapib suscip liguldon posue augquaeti v tort”)
(clojure.string/join " " (take 3 (shuffle (clojure.string/split s " “))))
;=> "augquaeti posue v”

oh you said generator… hmm

You could make a generator for sequences fitting that format. Composing in string/join is the unknown bit, but it seems doable.

You could do that in a custom generator, for instance, but then you'd have a custom generator 😕

(gen/sample (gen/bind (gen/shuffle (str/split "foo bar baz qux asd wet ab" #" ")) #(gen/frequency [[5 (gen
/return (take 3 %))] [5 (gen/return (take 5 %))]])))
;=> (("foo" "baz" "bar" "ab" "wet") ("foo" "ab" "baz") ("wet" "bar" "baz") ("baz" "bar" "ab") ("bar" "ab" "baz") ("foo" "
ab" "baz") ("foo" "wet" "asd" "ab" "baz") ("asd" "wet" "ab") ("qux" "wet" "foo" "bar" "baz") ("ab" "bar" "asd" "foo"
"baz"))

yeah but I can’t make this work, I got this far:

(gen/generate
 (s/with-gen string? #(gen/generate (gen/shuffle (clojure.string/split lorem-ipsum #”\s")))))

got it to alternate between take 3 and take 5

i think you see the pattern

oh, let me try your snippet

just add an entry for (take 4 %) with w/e freq you want

i think the only reason i came up with that so fast was the time i spent in haskell 😅

bind man...

@adambros oh this is awesome, now I think how could be gen/frequency part can be generalized

um

how about

(gen/vector (gen/elements the-words) 3 5)

@ag ⇑

should be more efficient than the shuffler too since it doesn't do lots of shuffling work just to throw it away

only difference is it won't give you distinct elements

@gfredericks oh, yeah, makes sense

thanks

thats interesting, so i guess it depends on if you want/need distinct elements

if you need distinct, wrapping it in such-that might be efficient, as long as the collection is large enough

errrh… now how I actually emit strings? str/joined? I am trying to wrap it in gen/fmap - doesn’t work ;(

nevermind got it!

(gen/fmap
    #(clojure.string/join " " %)
    (gen/bind (gen/shuffle (clojure.string/split lorem-ipsum #" "))
              #(gen/vector (gen/elements %) 1 5)))