hey @gfredericks et al - have you explored concurrent (ie non-monadic, maybe applicative) generation in any way?

@bbloom what would you use that for

I'm assuming you're not talking about "running my tests in parallel"

no, i’m talking about conceptual concurrency

not necessarily parallelism

i’m mostly thinking about the effort required to make generators that behave well

the generate & test approach is kinda cumbersome for some more interesting properties

Oh you're talking about that awkward separation

For when you wanna do some testy things and then generate some more stuff?

trying to come up with a good example, way to explain my thought here

1 sec

ok so if i do something like:

(s/exercise (s/and int? #(> % 50000000000)))

this still works

but only b/c the int generator is producing lots of values and then the s/and generator is filtering them

i’m wondering if there’s any experimentation done where the filter can feed information back to the generator

rather than being a strictly forward pipeline

Ohright. I feel like rich might have wanted something like that. But I have no idea what direction you would go for a general solution to that. Did you have something specific in mind when you said concurrent?

something earlier had reminded me of the art of the propagators stuff

and i re-read the section in that report about the truth maintenance systems and the search problem

once you add predicates, generation changes from an enumeration problem in to a search problem

numbers with inequalities are a pretty easy search space - but can get rough if you did something like “is prime” or whatever

so you wind up having to override the generator for the whole thing

and the more interesting the predicates, the more generator work you need to do

Yeah, it smells like declarative programming, where you can do much more than you might think, but way less than you need in real life

heh

i was just wondering if you guys had thought about this sort of thing much at all - since i keep trying to use generative testing, and it keeps being a lot of work. so far i’ve only really been truly successful with it in the form of “simulation testing” on a service

but i had to do a lot of work to build a good model / generator / validators / etc - made sense for a whole system, but s/exercise etc is fun and rapidly loses utility as i enhance my specs

Yeah I agree about all of that

I haven't thought about it too much because I haven't had any reason to be hopeful that it would be fruitful. I'm definitely open to being persuaded otherwise.

i guess that’s why i specifically mentioned “applicative” - as it seems like there might be some way to improve generators in a compositional way using binary combinators

the two interesting ones being intersect and union

so like maybe rather than defining a generator override for a spec by name

you can specify an override for a tuple of union/intersect and a pair of generators

which is already sorta what s/and and s/or do, but the do it left to right

although it’s not quite clear to me how to go about this without immediately winding up in the land of SAT/SMT 🙂

yeah; and inevitably it would only work for a few lucky combinations of things and otherwise you'd be on your own

if there are common combinations of specs that are easy to generate automatically, that can, I assume, be automated by writing special spec macros for them

ok - well i may noodle on this some more at some point, but for now i’ll just resign myself to abandoning generative testing as soon as it gets tough and then re-adopting it as soon as my system gets big enough to justify the simulant sort of thing 🙂

thanks for the discussion

haha

np; good luck

While writing specs for some existing data structures I encountered a case where a structure has two fields with the same unqualified keyword (nested on different levels, obviously). One field is a collection, the other is a string. Is there a way to make this work with spec? I tried searching online, but couldn't find anything and it seems like generally the answer would be "your data structure is wrong, fix it". Other than splitting the structure into two namespaces (so that each keyword and its associated specs is in a different ns), is there anyway to address this?

either you need 2 different ns'ed keywords as spec ids, or use a predicate instead of s/keys

The interesting question is: why s/keys has been designed this way?

I mean: probably the case raised by @jrychter is not idiomatic

Maybe, it is not idiomatic to have different keys with the same name?

I’m asking because I have also encountered the same issue

Well, I can see why "your data structure is wrong, fix it" makes sense — I was just hoping for a reasonable workaround. In the meantime, I am writing db migration code to rename one of the keys.

Actually, I don’t understand why it is wrong - but I’ve got the feeling that it’s not the clojure.spec way to shape a map

But I don’t understand why s/keys doesn’t support passing pairs to it

Something like: (s/keys :req-un [[::aa integer?] [::bb string?]])

Then the name of the key would be decoupled from its meaning

Well, spec by being tied to namespaces assumes that a spec key is "global" within the data structure, as long as everything is within a single ns. I can see (and like) the advantages of that, but there are cases where this becomes problematic.

the workaround is two keywords with different namespaces

yep ☝️ that

@viebel the whole point of spec is that the key of a map is a semantically enduring thing with meaning and that maps are nothing but sets of those attributes

This makes perfect sense, except sometimes you run into maps where there is a :name and a :seller :name, and you don't necessarily want to introduce a whole new seller namespace. Spec pretty much forces you to. Note, I'm not saying it's a bad thing, just noting that this is one of the few places where Clojure steps in and says: this is the one right way to do it. I got used to Clojure being very lenient (want objects? fine. want pure functions? fine. want state? fine. want to mutate it? fine.)

a lot of stuff about spec would have to change to accommodate that, I think

spec has aspirational and opinionated goals

@alexmiller: Yes, that's what I'm noticing 🙂 I've already learned to appreciate some of those. In particular, spec encourages reuse, and I'm already making use of it.

I guess it could just add a new kind of map spec; that'd be pretty independent

we’re not going to do that

yes I do realize that 🙂

but I think a library could provide that without undermining anything

@alexmiller do you know if Rich has an opinion about http://dev.clojure.org/jira/browse/CLJ-2112 and the form it's taking currently?

I am tempted to rewrite a bunch of macros for generating specs+json-schemas on that base, and would like to know if it's viable

@mpenet I would not base anything off of that patch - everything there has changed multiple times and is likely to change again

it has been through one round of review from both Rich and Stu but there are still a number of open questions about it

Alright, thanks for the info. Any sort of ETA for news on this proposal?

no

Hello here 🙂 Just a question for my information: has anyone ever tried (or plan to try) to use clojure.spec to validate form answers or models?

We use spec to validate REST API arguments — which is pretty similar to form answers (in that they are both user-provided over HTTP).