This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2017-02-21
Channels
- # arachne (1)
- # aws-lambda (50)
- # beginners (10)
- # boot (59)
- # capetown (4)
- # cider (9)
- # cljsjs (27)
- # clojure (249)
- # clojure-berlin (8)
- # clojure-finland (7)
- # clojure-germany (1)
- # clojure-italy (6)
- # clojure-nl (7)
- # clojure-russia (91)
- # clojure-spec (100)
- # clojure-uk (61)
- # clojureremote (2)
- # clojurescript (171)
- # core-async (11)
- # cursive (31)
- # data-science (1)
- # datascript (2)
- # datomic (11)
- # dirac (2)
- # emacs (16)
- # events (1)
- # hoplon (142)
- # juxt (4)
- # lein-figwheel (9)
- # leiningen (10)
- # luminus (7)
- # lumo (44)
- # mount (3)
- # off-topic (150)
- # om (18)
- # onyx (5)
- # perun (12)
- # planck (12)
- # protorepl (13)
- # re-frame (28)
- # reagent (8)
- # ring (1)
- # ring-swagger (10)
- # spacemacs (2)
- # specter (11)
- # sql (14)
- # untangled (99)
- # vim (18)
- # yada (2)
Hi! If I need (gen/sample ::somespec) about 1M or more, how to parallelize this operation? is there build-in tools? or just pmap or something?
@mike1452 what do you need them for?
@gfredericks for data sample generation. I want to give samples to other team, which needs shape of our data for dev process.
@alexmiller what's the extra form
parameter in s/valid?
?
It's an alternate value to return if not valid
Or maybe I'm reading that wrong
maybe I misunderstand something then: https://github.com/clojure/clojure/blob/d920ada9fab7e9b8342d28d8295a600a814c1d8a/src/clj/clojure/spec.clj#L672-L683
Can't say I've ever used it
that’s not the current version of that code
@mike1452 you can parallelize it however you like, that's kind of independent
@alexmiller it's the one linked from the api docs I think
@mpenet looks like its used internally via pvalid?
yeah, the autodoc updating was broken by an issue in spec, so they are not the latest
I have a patch pending to fix the code and get that working again
that valid? form is used when you have an alternate form to supply for the error reporting
I think maybe the only place it’s used is from within keys
used with that extra form that is
I am wishing for a explain-info
I think. Sometimes I need to add some context to explain data
the valid? function should be considered public and I don’t think there are any expectation that api will change
my impression is that Rich doesn’t want to provide extensions to modify explain (other than by writing your own spec and taking care of it there)
in short: that spec predicate is quite resource heavy, and upon failure the function wrapped by that predicate provides a lot of metadata about the failure (think a query parser output, with line/col num, explain traces etc). Right now not only I loose all this data but I need to re-trigger a parse to get back the metadata (or throw upstream, bypassing spec, which is ugly). In theory, one could just wrap and report the error manually, but this values are "deep" in a (spec validated) map.
sounds like a good use case for writing a custom spec
it’s available to do but is subject to change while in alpha
ok, I guess I might just do this. That's how it used to work with Schema btw, seems quite an elegant way to do it
it’s something I would avoid doing as much as possible
@mpenet I was under the impresion that explain-data
was meant to provide that functionality: https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/explain-data
no explain-data just returns the data behind a failing spec, doesn't allow you to "customize" said data
but isnt the data behind a failing spec the one that your parser already returned? I mean if your parser would be a speced fn then the data that you get already contains that custom information that you want. Or am I missing something?
well, you’ll also get the failing value
but not all the work that went into evaluating the failing value
each problem in explain-data has the value that failed the spec
yes, the initial value true, but not the "detail" of the failed predicate (in my case an ex-info instance)
maybe the exception case in particular is interesting
even then, that wouldn't make it as easy as it is not with the custom spec, it just feeds that extra data to the explain map
well, what if when conforms fails with an exception, the ex-info was conveyed
normally it just returns a truthy/falsey value (and nil/false can’t convey additional info)
but exceptions can
@alexmiller using explain-info you would basically write your own conformer( try/catch potential exception in my case) and return the (explain-info {...}) on failure. I am not sure doing by default for any exception (or ex-info instance) is desirable
perhaps not
Is core.spec able to use something else as keyword qualifiers besides the current ns? As I say that out loud, I realize how silly it would be if that were not the case, but I see a lot of the example code using the double colon keywords, which auto expand to current ns
any ns is fine
I don't suppose there's some kind of dynamic binding that would change the way the double colon autoexpands?
you can use aliases
(alias ‘a ‘com.mycompany)
::a/keyword
the tricky part is that right now, com.mycompany
must be an actual namespace
you can work around that by doing
(create-ns ‘com.mycompany)
that is something we’ve talked about changing though
https://www.deepbluelambda.org/programming/clojure/know-your-keywords covers this
guys, I’m kinda stuck again. I’ve done it before (I think) now I can’t remember the right way of doing that. How do I properly generate things based on vector? So If I have generated vector of maps, and then for each item I need to generate something else
passing vector generator with fmap into a function with for
kinda works, but I think this isn’t right approach, because I need to call gen/generate
inside for
> For example, say we wanted to generate a vector of keywords, and then choose a random element from it, and return both the vector and the random element.
Random element?
Why did you generate a whole vector of them just to pick one out?
well one reason you might do that is if you need a vector AND an element from that vector
actually @gfredericks comment in my case make sense.
ok, there’s still something I can’t figure out for the second case, where I have a predefined vector of maps and for each item I need to generate (something based of that map) and attach in a key to that map and finally return that modified vector
Calling gen/generate in general is not a proper way to build a generator
And it shouldn't ever be necessary
If you want to describe what you're doing I'm happy to help work it out