Fork me on GitHub
Oliver George07:06:06

Hi Specy Specers. What's the most human friendly way of viewing the output from check-var? I see that reporter-fn can be provided, is there a commonly used one? (I'm using CLJS so perhaps that makes a difference).


I find myself writing lots of (s/and string? seq) to specify non-empty strings. Also, I'm missing a predicate for strings of length from n to m (analogous to int-in-range?).


Is it possible that s/with-gen alters the spec it defines a generator for? I have a simple (s/cat :base ... :children ...) spec that works fine but as soon as I wrap it in (s/with-gen <spec> #(gen/tuple (s/gen ...) (s/gen ...))), data that would previously conform to the spec now becomes invalid.


Note: I', not generating the data using the generator yet. I'm using hand-written data.


@olivergeorge: there's a discussion about that on the main Clojure mailing list. I'm very interested in the answers to that question.


@jannis: It looks like it's introducing a new regex context like spec does:

user=> (s/explain (s/cat :x ::pair) '[a [b c]])
user=> (s/explain (s/cat :x ::pair-with-gen) '[[a [b c]]])


I'm working on some code that does simulations over state machines. The transition functions tend to have a lot of detailed conditional logic, and as a result test.check doesn't seem to be a great fit for validating the functions (random inputs tend to be ignored or lead to errors, and writing generators to provide valid input is essentially the same as writing the state machine model). When instrument-all checked the return value it was very useful, since I caught a lot of errors of omission, misspelled keywords, etc. I'd like to suggest we have an option to check :ret and :fn for cases like this.


Yeah, whilst I agree in principle with the justification @alexmiller offered as to why :ret and :fn checking was removed in Alpha 6, I also agree that there is potentially a lot of value in having the option to be able to instrument functions in a way that does conform the result at least.


Thanks, I may use that to hammer out my own version of instrument for now.


I think there's gotta be something coming for spec'ing impure functions that will cover this


otherwise you could never spec them, really, or the spec would be pointless

Alex Miller (Clojure team)18:06:13

@seancorfield: Rich has some ongoing work, I'm not sure what the endpoint will be on this. you might have noticed that explain-out was made public today in master

Alex Miller (Clojure team)18:06:25

@bfabry: not every function is a great candidate for generative testing via spec (but the spec may still be useful for docs or other purposes)


@alexmiller: where was the explanation @seancorfield mentioned of why :ret and :fn checking was removed? I’m curious to read it.

Alex Miller (Clojure team)18:06:19

@seancorfield: I think Stu is looking at some testing-related mods too btw


re: explain-out — I already added a comment on that commit thanking him for that 🙂


@alexmiller: I really do appreciate the steady stream of alpha builds so we can all try this stuff out and provide feedback.

Alex Miller (Clojure team)18:06:46

I'm sure there will be more :)


Having an option on instrument to use the old version of spec-checking-fn with :ret and :fn conforming would be very nice. I think checking just :args is the right choice for most cases of instrumentation, but I think being able to "fully instrument" certain functions would be very valuable — especially for functions that cannot easily be tested the generative way.


For example, working on java.jdbc’s specs, they can’t reasonably be tested generatively because many of them are side-effecting (updating the database) and writing generators that conformed to the database schema would be … a huge amount of work, if it’s even feasible (e.g., unique key constraints etc?).


So losing the ability to conform the :ret and :fn specs there is kind of a big deal, IMO.


I’ll be interested to see how this all ends up since any given code base is going to have a mix of functions that can reasonably be tested generatively and functions that can’t, so (clojure.spec.test/run-all-tests) needs a way to distinguish those, right?


(mind you, right now I can’t run generative testing on java.jdbc because the system doesn’t know how to generate a java.sql.Connection… which might be an interesting exercise 🙂 )


(and I’d also need a generator for a java.sql.PreparedStatement)


What is the recommendation for stuff like that? How would you even write a generator for some of these Java objects?


Would anyone be willing to offer some guidance on conventions for attributes shared by entities of different types? For instance, I have "question" and "survey" entities, each of which can have names. Right now I'm using (s/def :entity/name string?) and entities of either type can have an :entity/name attribute along with their type-specific attributes. The alternative of course is to define both :question/name and :survey/name. I haven't seen the :entity/attr pattern elsewhere, and I'm wondering whether there's a reason for that.


@tomc: I believe you can share the attribute as long as the semantic is the same, for example, a car name may have a different semantic from a person name, but that will depend on the requirements on your system


@wilkerlucio: thanks a lot, that's helpful.


I’m trying to generate dates within a range with spec, and I’m stucked with this piece of code which generates #inst whose year is above 9999, making them unrecognized when i try to def them manually afterwards : (gen/sample (s/gen inst?) 60)


ex of data generated: #inst"26138-06-03T15:09:43.670-00:00


(def d1 #inst"26138-06-03T15:09:43.670-00:00")
CompilerException java.lang.RuntimeException: Unrecognized date/time syntax: 26138-06-03T15:09:43.670-00:00, compiling:(/Users/gilles/dev/try-spec/src/try_spec/core.clj:23:46) 


@gphilipp: you can try something like this:


the first number, 100000 is a cap to limit the increment, and the 1465391285642 is the ms for a start date, adjust those to met your needs


ah, and this example is for CLJS, please change the Date initialisation if you are using on Clojure


@wilkerlucio: thx, I will try this


@gphilipp: There is also an clojure.spec/inst-in macro.


The default 100 tests for test.check can take a really long time on some fairly simple looking specs...


…running 50 tests wasn’t too bad but it seems to be taking more than linearly longer to do 5, 10, 25, 50, 100...


@seancorfield: Yeah, as well as being very dependent on ordering. The caveats about "generate-and-test" style vs. constraint satisfaction are in full effect here.


On the plus side, I figured out how to write generators for stuff like java.sql.Connection etc 🙂


And I also finally figured out how s/keys and the test.check integration hang together… which answered a question I’d asked someone else here before I understood what was going on...


Thanks @leifp, (gen/sample (s/gen (s/inst-in #inst "2016-01-01" #inst "2016-12-31")) 100) did the trick


@gphilipp: The end date is exclusive, remember.


ah, correct


I still wonder why (gen/sample (s/gen inst?) 60) generates invalid instants.


See? Generative testing is going to help us avoid the Year 26k Problem.


(and it took me a while to even get so far as to make run-all-tests actually start running… with-gen taking a 0-arity function that returns a generator is very counter-intuitive and I kept getting that wrong 😞 )


@seancorfield: The (s/* (s/or ...)) combo seems to be the culprit. It would probably be very fast if you could limit it to a max size. I don't know how to do that, though.


I guess I could always add a custom generator around it just to avoid that?


(and, thanks for the pointer on that!)


@seancorfield: This works: (s/def ::column-spec (s/with-gen (s/cat :spec (s/* (s/alt :kw keyword? :str string?))) #(g/vector (g/one-of [g/string g/keyword]) 0 6))) Still not what I'd call lightning-fast, but you can bound the generation time by decreasing the max. And note that g/ refers to clojure.test.check.generators, the equivalent using clojure.spec.gen didn't work.


Kind of clunky, though, and you'd need to do that everywhere you have (s/* (s/alt ...)). Maybe if enough people run into performance problems, rep will be made public, or the regex generators will be optimized.


Assume I parse with conform. Then I have functions that operate on the value returned by conform. I can't get a spec for the value returned by conform (so that I can spec said functions) - It seems like this could be automated though. Imagine (s/conform-spec ::my-spec) would return the spec of the return value of calling (s/confom ::my-spec 42)