Fork me on GitHub

Hi! If I need (gen/sample ::somespec) about 1M or more, how to parallelize this operation? is there build-in tools? or just pmap or something?


@mike1452 what do you need them for?


@gfredericks for data sample generation. I want to give samples to other team, which needs shape of our data for dev process.


@alexmiller what's the extra form parameter in s/valid? ?


doesn't seem to be documented


well, it's not used actually

Alex Miller (Clojure team)14:02:44

It's an alternate value to return if not valid

Alex Miller (Clojure team)14:02:46

Or maybe I'm reading that wrong

Alex Miller (Clojure team)14:02:57

Can't say I've ever used it


doesn't seem used at all but in the exception

Alex Miller (Clojure team)14:02:04

that’s not the current version of that code


@mike1452 you can parallelize it however you like, that's kind of independent


@alexmiller it's the one linked from the api docs I think

Alex Miller (Clojure team)14:02:02

@mpenet looks like its used internally via pvalid?


I ll have a look

Alex Miller (Clojure team)14:02:22

yeah, the autodoc updating was broken by an issue in spec, so they are not the latest

Alex Miller (Clojure team)14:02:35

I have a patch pending to fix the code and get that working again

Alex Miller (Clojure team)14:02:43

that valid? form is used when you have an alternate form to supply for the error reporting


that's exactly what I wanted it to be 😄


I ll try it out

Alex Miller (Clojure team)14:02:42

I think maybe the only place it’s used is from within keys

Alex Miller (Clojure team)14:02:52

used with that extra form that is


so it's not supposed to be "public" ?


I mean subject to changes/removal


not sure it helps actually


I am wishing for a explain-info I think. Sometimes I need to add some context to explain data


I know I can wrap the whole spec and add this at a higher level, but it's a bit gross

Alex Miller (Clojure team)14:02:55

the valid? function should be considered public and I don’t think there are any expectation that api will change

Alex Miller (Clojure team)15:02:37

my impression is that Rich doesn’t want to provide extensions to modify explain (other than by writing your own spec and taking care of it there)


😞 care about more detail of my use case for a potential suggestion?


in short: that spec predicate is quite resource heavy, and upon failure the function wrapped by that predicate provides a lot of metadata about the failure (think a query parser output, with line/col num, explain traces etc). Right now not only I loose all this data but I need to re-trigger a parse to get back the metadata (or throw upstream, bypassing spec, which is ugly). In theory, one could just wrap and report the error manually, but this values are "deep" in a (spec validated) map.


open to suggestions on how to better handle this

Alex Miller (Clojure team)15:02:46

sounds like a good use case for writing a custom spec


you mean implementing the internal protocol?


is that "safe"?


I mean allowed

Alex Miller (Clojure team)15:02:00

it’s available to do but is subject to change while in alpha


ok, I guess I might just do this. That's how it used to work with Schema btw, seems quite an elegant way to do it

Alex Miller (Clojure team)15:02:10

it’s something I would avoid doing as much as possible


yes, same feeling.


works like a charm


@mpenet I was under the impresion that explain-data was meant to provide that functionality:


no explain-data just returns the data behind a failing spec, doesn't allow you to "customize" said data


same diff as ex-info vs ex-data (kinda)


but isnt the data behind a failing spec the one that your parser already returned? I mean if your parser would be a speced fn then the data that you get already contains that custom information that you want. Or am I missing something?


no it's the spec form : something like #(try (parse q) (catch ExceptionInfo e ...))

Alex Miller (Clojure team)16:02:30

well, you’ll also get the failing value

Alex Miller (Clojure team)16:02:38

but not all the work that went into evaluating the failing value


hmm I don't think so


valid will return false, conform ::invalid


ex-data the form

Alex Miller (Clojure team)16:02:17

each problem in explain-data has the value that failed the spec


yes, the initial value true, but not the "detail" of the failed predicate (in my case an ex-info instance)

Alex Miller (Clojure team)16:02:55

maybe the exception case in particular is interesting


even then, that wouldn't make it as easy as it is not with the custom spec, it just feeds that extra data to the explain map


yes, imho it's something that might make sense in spec itself.

Alex Miller (Clojure team)16:02:37

well, what if when conforms fails with an exception, the ex-info was conveyed

Alex Miller (Clojure team)16:02:57

normally it just returns a truthy/falsey value (and nil/false can’t convey additional info)


that would be nice, that's what I meant with explain-info , it could return that


either accept a explain-info return, or the kw


but the reify Spec option is actually nicer in my case (for other reasons)


how about (or (s/explain-data spec x) ::ok )


returns ::ok when everything when well or the ex-info from your spec if it failed


@alexmiller using explain-info you would basically write your own conformer( try/catch potential exception in my case) and return the (explain-info {...}) on failure. I am not sure doing by default for any exception (or ex-info instance) is desirable


Is core.spec able to use something else as keyword qualifiers besides the current ns? As I say that out loud, I realize how silly it would be if that were not the case, but I see a lot of the example code using the double colon keywords, which auto expand to current ns


I don't suppose there's some kind of dynamic binding that would change the way the double colon autoexpands?


::keyword => :com.mycompany/keyword

Alex Miller (Clojure team)19:02:04

(alias ‘a ‘com.mycompany)

Alex Miller (Clojure team)19:02:35

the tricky part is that right now, com.mycompany must be an actual namespace


that's great!

Alex Miller (Clojure team)19:02:11

you can work around that by doing (create-ns ‘com.mycompany)

Alex Miller (Clojure team)19:02:29

that is something we’ve talked about changing though


guys, I’m kinda stuck again. I’ve done it before (I think) now I can’t remember the right way of doing that. How do I properly generate things based on vector? So If I have generated vector of maps, and then for each item I need to generate something else


passing vector generator with fmap into a function with for kinda works, but I think this isn’t right approach, because I need to call gen/generate inside for


since it can’t return generator


> For example, say we wanted to generate a vector of keywords, and then choose a random element from it, and return both the vector and the random element.


@zane here in fn in just grabs random element from generated vector and then…


actually that may work for me


Random element?


Why did you generate a whole vector of them just to pick one out?

Alex Miller (Clojure team)22:02:55

well one reason you might do that is if you need a vector AND an element from that vector


actually @gfredericks comment in my case make sense.


ok, there’s still something I can’t figure out for the second case, where I have a predefined vector of maps and for each item I need to generate (something based of that map) and attach in a key to that map and finally return that modified vector


my solution requires to call gen/generate within for, which I don’t like.


I though test.chuck’s for would work for me, but I can’t get it right


Calling gen/generate in general is not a proper way to build a generator


And it shouldn't ever be necessary


If you want to describe what you're doing I'm happy to help work it out


so I have list-of-accounts (predefined list) which is a vector of maps, e.g. [{:id 1 :type :customer} {:id 2 :type :investor},,,]. Now, for each item in that vector I need to generate a balance-update data and add it in :bal key and return that modified vector