Seems to work now. I’m generating regexes using a simplified regex spec (that exists solely for generating) and I’m generating strings that match it using test.check. I’m using these combinations to test re-find, etc. Thanks for the suggestions all 🙂

https://twitter.com/borkdude/status/1095985994590027777

The only problem left is making it work on CLJS and self-hosted CLJS 😕 https://github.com/gfredericks/test.chuck/blob/master/src/com/gfredericks/test/chuck/generators.cljc#L244

I think I’ll just replace the generator on CLJS with a simpler one

@borkdude there's a PR for that 🙂

I'm not sure if it works though

PR seems out of date. Even if it’s incomplete I think it’s better than nothing? @wilkerlucio

I just realized that core.match could play nice with clojure spec conform results (in general, not related to this regex discussion)? https://stackoverflow.com/a/54687183/6264

(probably realized a little late)

i'm probably missing something obvious -- how can i define a spec so that its value conforms to one of a collection of specs? for example, a person can have a pet that's either a mammal or a fish, neither of which share a common attribute.

(s/def :animal.class/mammal (s/keys :req [:utters/count]))
(s/def :animal.class/fish (s/keys :req [:fins/count]))
(s/def :person/pet (s/or :animal.class/mammal :animal.class/fish))

(s/explain :person/pet {:utters/count 4})
val: #:utters{:count 4} fails spec: :animal.class/fish at: [:animal.class/mammal] predicate: (contains? % :fins/count)
=> nil

whoops - that was meant for #beginner

(s/or :animal.class/mammal :animal.class/fish) => (s/or :mammal :animal.class/mammal :fish :animal.class/fish)

you need to tag the alternatives with a key

ah!

i saw that in cat/alt, didn't realise it applied to /or. thanks as usual, borkdude.

borkdude, are you basically looking for bugs in the JVM's and/or JavaScript's regex library using spec and generative testing?

I wouldn't be surprised if you find bugs in those, and/or test.chuck's generate-a-string-matching-a-regex-from-a-regex code, or all of the above. Just so you realize that the time scale for fixing the JVM and/or JavaScript's regex library might be a bit long 🙂 Those libraries must be challenging to maintain.

The situation is like this: someone used Orchestra with speculative specs. Orchestra checks ret specs at runtime (in contrast to spec). It turned out the ret specs of regex functions were incomplete and I could have found this if I had used generative testing (which I have for almost all spec’ed functions except these). Now I have almost fixed this generative testing, but I ran into the CLJS limitation of test.chuck as a last issue.

I can bypass this by generating simpler strings for CLJS. It will still find the case I forgot to spec.

I’m not using it for anything more serious than this, just wanted to see how for I could go with it.

any planned changes (or options) about fn as argument triggering gen, something like wrapping their arguments with asserts instead so that they fail at invoke time ?

I think there was a jira issue about this

will revisit, haven't yet

Is there a way to provide a custom generator for a spec distant to the spec's definition/local to the test context? The docs for instrument say that the :replace only relates to fn-specs and :gen option is only for stubbed vars

@djtango do you want to test with stest/check or instrument?

I think right now want to manually generate some inputs using gen/sample

though could also do stest/check - was it stest/check that lets you also patch in your own generators?

that seems familiar

yes,

(stest/check `my-function {:gen {::my-spec (fn [] (gen/…))}})

awesome thanks

but with instrument no luck?

> :gen overrides are used only for :stub generation.

I’m not sure what :replace does, I have never used it 🙂

:replace replaces a fn with a fn that checks args conformance, then
invokes the fn you provide, enabling arbitrary stubbing and mocking.

:thinking_face: