Fork me on GitHub
#clojure-spec
<
2018-12-03
>
PB03:12:32

Does anyone know of a tool where I can give it a datastructure, and it will generate a spec for me based upon that datastructure?

PB03:12:40

Thats wonderful, thank you!

🤝 4
avi18:12:18

I feel like I’m a little rusty with spec, and getting tripped up by something… Can anyone see what I’m doing wrong here?

(s/fdef probably-diagram-yaml?
  :args (s/cat :v (s/or :is-diagram     :ss/diagram-yaml-str
                        :is-not-diagram string?))
  :ret  boolean?)
When I test this function with stest/check I’m seeing values generated via string? being validated against :ss/diagram-yaml-str — is that what’s supposed to happen? If so… why? (This is probably just PEBKAC)

seancorfield18:12:47

What's the generator for :ss/diagram-yaml-str look like?

seancorfield18:12:08

(I would expect that to also have string? in it based on its name)

avi18:12:55

It’s a little crazy

avi18:12:46

The thing is, whenever I exercise that spec it succeeds just fine. And… hmm, well, I hope that exercise does validate the generated values against the spec 😅

avi18:12:15

Generator:

#(gen/fmap
      (fn [diagram]
        (str (sometimes (str seyaml/default-front-matter doc-separator))
             (seyaml/stringify diagram)))
      (s/gen ::st/diagram))
and sometimes is:
(defmacro sometimes [body]
  `(when (< (rand) 0.5)
     ~body))

taylor18:12:22

FYI you’ll probably want to replace that sometimes with one of test.check’s generators, otherwise it’s an unbound/uncontrolled source of randomness to test.check

avi18:12:18

Great tip — thanks so much!

👍 4
taylor18:12:54

maybe something like (gen/frequency [[1 (gen/return "thing")] [1 (gen/return nil)]])

avi18:12:23

ah that’d work? equal frequencies means equally likely?

avi18:12:36

(I didn’t look at the docstring for frequency yet)

taylor18:12:03

that’s my untested assumption 🙂 and there might be an even more appropriate function in test.check I’m forgetting/don’t know about

avi18:12:05

based on the link you provided, looks like I should use (gen/return nil) rather than just nil

taylor18:12:39

oh yeah, sorry… my example had concrete values instead of generators, fixing

aw_yeah 4
taylor18:12:19

I suppose you could use gen/elements too, if your values are static/not-generated. I suppose elements given a two-element coll would choose between them equally?

avi18:12:01

I dunno what the distribution is

avi18:12:18

IIRC some test.check generators don’t use even distributions

avi18:12:38

I wonder if this is on the right track:

#(gen/fmap
  (fn [diagram]
    (str
      (gen/frequency
        [[1 (gen/return nil)]
         [1 (gen/return (str seyaml/default-front-matter doc-separator))]])
      (seyaml/stringify diagram)))
  (s/gen ::st/diagram))))

taylor18:12:12

looks pretty reasonable to me. I still wonder if elements would be better suited in this case b/c it'd definitely be more concise

avi18:12:33

probably

avi18:12:45

thanks for helping me improve my code!

4
avi18:12:32

Ah yes, peeked at the source of exercise and it calls conform on all generated values. (It’s mentioned in the docstring too.)

avi18:12:25

I think this is starting to dawn on me… I think maybe (I’d like to think because I’ve been rusty) I’ve just been not quite grokking s/or … when a value is conformed against an s/or spec it will quite naturally be conformed against each scenario in order. And after a value is generated via a generator, I guess there’s no mechanism at that point to conform it straight off against its original scenario spec… in other words, s/or isn’t like pattern matching, which I guess was for some reason how I was trying to use it.

taylor19:12:38

I think it's pretty conceptually similar to pattern matching, but I'm not sure I follow this issue: >And after a value is generated via a generator, I guess there’s no mechanism at that point to conform it straight off against its original scenario spec

taylor19:12:40

BTW one way I think s/or differs from pattern matching like in ML languages is that s/or clauses don't have to be exhaustive by default. Don't have to be non-overlapping either

avi19:12:58

er, yeah, sorry. I think it was just my rusty thinking. • An or spec is a spec, and (I suppose) most functions that work with specs will treat/use an or spec as an opaque spec, like any other spec. • Accordingly, the workflows of conform and stest/check have no affordances for taking values generated from the or spec and influencing how those values are conformed against the or spec and its internal scenarios • Accordingly, values generated by any given scenario of an or spec might be conformed against any other given scenario of an or spec, because that’s how an or spec works — values are conformed against its scenarios one by one in order • The only reason I got caught up on this is because one of my specs would sometimes throw an exception when passed an invalid value

avi19:12:05

I hope that might begin to approach making some kind of sense

taylor19:12:56

Yeah I’ve been bitten using a predicate in an or that wasn’t nil safe. Order matters in some cases

avi19:12:11

yes, exactly

avi19:12:20

my spec calls a function that throws if passed an empty string

avi19:12:23

related to a regex search

avi18:12:45

I feel like I need a couch to lie down on and someone with a pipe to nod and doodle on a notepad

🛋️ 4
🙏 4
avi18:12:53

sorry for the noise!

prozz20:12:02

hmm, have a strange question: how to actually spec that something is a sentence? or write a generator for a good sentence? (let's say lorem ipsum language with limited number of words is enough)

prozz20:12:27

just realized im specing stuff as string? but would like my generators to be clever and generate usernames as random strings but description as some sort of lorem ipsum with spaces. if this question is too beginner like for this channel, let me know pls.

andy.fingerhut20:12:41

If you mean actual grammatically correct English, or some other natural language, then you need some kind of NLP library or a product of machine learning for that. If by "sentence" you mean something else, please clarify.

noisesmith20:12:45

and spec explicitly isn't a string parsing library - anything for differentiating one string from another will be wrapped up and just be another function as far as spec is concerned

andy.fingerhut20:12:00

Sorry, you did mention "some sort of lorem ipsum with spaces", but still might need some clarification of exactly what kind of strings you want to include vs. exclude. A regular expression could be written over strings that matches most lorem ipsum like text, but rejects things that contain weird special characters.

andy.fingerhut20:12:14

If you did that, and then wanted to generate random examples for testing, you would likely only be able to do so by writing a custom string generator, because otherwise the default test.check string generator would, with very high probability, generate strings that do not satisfy the spec.

gfredericks20:12:52

test.chuck (different lib) has a regex -> generator thingamabob

4
andy.fingerhut20:12:50

cool. I probably saw that mentioned before and forgot about it. Such a thingamabob sounds pretty non-trivial, unless you found a library that transforms regexes into state machines for you.

gfredericks20:12:03

it was nontrivial

gfredericks20:12:38

the most nontrivial parts were punted on (esp. lookahead, lookbehind)

andy.fingerhut21:12:13

Makes sense. Common regex libraries include so many features like that, that complicate analyzing them (and IMO, understanding what they do)

gfredericks21:12:25

one big hard part was parsing the regexes correctly, and the other one was handling characters correctly

andy.fingerhut21:12:01

I saw a quote similar to the following (paraphrasing from memory) in the early 1990s in someone's signature on a public Internet forum and loved it: "Ain't nothin's easy when you're doing it fer real! -- a gunnery sergeant in the U.S. Army"

gfredericks21:12:04

well some things are, but they're not very interesting to talk about

prozz21:12:07

thanks for explanation, you all above confirmed what i thought. need to dive deeper into generators, check possibilities and decide what i really wanna do 🙂 basically i want a fixtures generator for particular domain. hence need for sentences.

prozz23:12:10

is there an existing generator for normal distribution from given range?