Fork me on GitHub
#clojure-spec
<
2017-10-16
>
bbrinck13:10:47

I’m unclear on how conformers are intended to be used.

bbrinck13:10:26

I’ve seen cases like: - converting a string into a seq so spec regex operators can validate a string - converting a string into an int - converting a string into a UUID

bbrinck14:10:13

But from my understanding of https://dev.clojure.org/jira/browse/CLJ-2116, conformers aren’t intended to be used for transformation

bbrinck14:10:18

Are the above cases unintended uses of conformers? What would be an intended use case? I ask because I’m trying to figure out how to include support for conformers in Expound

Alex Miller (Clojure team)15:10:04

the intended use case was for writing new composite spec types (like s/keys*)

Alex Miller (Clojure team)15:10:23

so basically not what most people try to use them for

jrychter07:10:17

FWIW, I also use them for things like converting strings to keywords in JSON.

jrychter07:10:45

I think that while this might not have been the intended usage, it shows that there is a need here. I am having hopes that Rich will take this on as something to think about in the future.

bbrinck22:10:05

For what it’s worth, using conformers in this way also makes it harder to give good error messages. I’m not sure what the best solution is, but perhaps two phases where first the data is validated, then transformed

mpenet14:10:54

to give you information about what you are validating for reuse "later", this makes more sense in the context of validating/destructuring data passed to a macro such as defn

bbrinck14:10:23

Do you happen to have an example? My main source of examples of macros specs is https://github.com/clojure/core.specs.alpha, but I don’t see any conformers there

mpenet14:10:27

my bad I read conforming and I got carried away

mpenet14:10:16

I am guilty of using it for transformation, not sure my example is clean but here you go: we have a dsl a bit like sql for a rule engine, we validate the syntax with spec and use a conformer to spill the ast (parsed form)

bbrinck14:10:19

@mpenet Hm, that is interesting. Is the DSL originally a string then? Do you use spec to parse the string?

mpenet14:10:22

also guilty of using it do to json transforms, "old" stuff

mpenet14:10:47

the spec is basically a conformer that wraps the parse fn

mpenet14:10:13

in reality it s a lot hairier because we have custom explain and whatnot but that s the idea

bbrinck14:10:49

I see, thanks for the example. One reason it’s hard to add support for conformers is that a) I’m unclear on the intended use and b) in practice, it seems like they are often uses for transformation. I’m not sure if it’s practical to not support a common use case.

mpenet14:10:30

it s made for transformations, I guess their validity depends on the context

bbrinck14:10:20

This is the part I don’t fully understand: conformers do transform values, but what are examples of recommended transformations? @alexmiller said: “I don’t think we are interested in turning spec into a transformation engine via conformers”, but I’m not sure what that includes/excludes

mpenet14:10:34

it's a good question indeed

mpenet14:10:10

neither the guide or rationale pages mention it

bbrinck14:10:30

@mpenet In any case, I appreciate you explaining how you use them in practice. This is very helpful for my research, since I may end up wanting to support different use cases

mpenet14:10:30

you're welcome. love expound. thanks for that

Alex Miller (Clojure team)15:10:17

the guide page intentionally does not mention conformers as we consider them to be primarily useful for writing new custom composite spec types (not for general data transformation)

bbrinck15:10:07

By “composite spec types” do you mean new spec types that work similar to or or regex specs? i.e. where I would want to name the parts?

bbrinck19:10:03

Whoops, sorry I missed your comment addressing exactly this question in the thread above. Thanks!

carocad10:10:30

@alexmiller would you be so kind to put an example of the “expected use” in ? 🙂 Currently only the “unintented use” is documented so I guess that this leads people in the wrong direction (and might continue to) http://clojuredocs.org/clojure.spec/conformer

Alex Miller (Clojure team)15:10:04

I think having a tool for transforming data informed by specs would be awesome. I don’t think conformers are that tool.

ikitommi17:10:41

@alexmiller any ideas how could we find such a tool? It looks like we could do that with conformers, with just small changes to how it works. Or is there something new coming from Rich?

danielneal15:10:46

I'm trying to use spec to document a map that contains a :key-fn that returns a collection i.e. on this data {:aliases #{:bob :robert}}, {:key-fn :aliases} would be suitable. However this happens

(s/def ::key-fn
  (s/fspec :args (s/cat :item any?)
           :ret (s/coll-of any?)))

(s/def ::my-map (s/keys :req [::key-fn]))

(s/valid? ::my-map {::key-fn :aliases}) ;;false
(s/valid? ::key-fn :aliases) ;; false
Am I doing this all wrong?

taylor15:10:28

I think the problem is that a keyword (`:aliases`) won’t conform to your fspec

taylor15:10:16

You could add an s/or case for keywords in your ::key-fn spec

danielneal15:10:50

hmm I did try (s/valid? ::my-map {::key-fn (fn [x] [:a :b])}) ;;false and got the same thing though

taylor15:10:54

(s/or :kw keyword? :fn (s/fspec ...))

danielneal15:10:23

the thing is it's the fact that the function returns a collection that is the important thing I'm trying to capture

danielneal15:10:54

I wouldn't mind if I have to help it along with a generator or something

danielneal15:10:09

but I don't know where or how or if I'm approaching the whole thing in a stupid way

danielneal16:10:20

the inlined function with constants worked

danielneal16:10:25

I would like to specify that if you're gonna give a keyword it's got to look up data that is a collection

taylor16:10:42

hmm maybe need another approach if you need that guarantee… a keyword could return anything that happens to be in the map

danielneal16:10:16

as far as I can make out specifying the return types of functions <as> parameters is a bit of an edge case

danielneal16:10:55

but would love to be able to provide it to help readers understand the contract

taylor16:10:26

so I guess you’d need more context if you were going to validate that the supplied keyword would return a collection; you’d need to know 1) what the keyword is and 2) the value for that keyword in the other map it’d be applied to?

danielneal16:10:20

yeah you're right

danielneal16:10:46

I suppose I could just put it in the doc string

taylor16:10:15

one weird idea: if you had all this info (including the “subject” map) in one data structure, you could write a spec for the whole structure to ensure any ::key-fn refers to a collection value in the “subject” map

danielneal16:10:34

ooh interesting

taylor16:10:23

Anyway, the return value spec for your function isn’t going to matter for valid?. It’d be useful if you were using test.check to check the function though

akhudek20:10:47

Is there a way to write a spec that checks properties of pairs of elements in a sequence? E.g. in [{:a 1 :b 2} {:a 2 :b 3} {:a 3 :b 4}] I’d like to check that :b is equal to :a for each pair of items from the sequence.

akhudek20:10:56

I can do this as a function on the sequence.

akhudek20:10:13

But then on failure explain returns the entire sequence as the value causing the problem.

akhudek20:10:32

erm, let me fix that

ghadi20:10:00

you can do it but not as a function on the sequence, but each element of it

ghadi20:10:39

unless I understand you wrong

ghadi20:10:08

Do you mean there is some relationship between adjacent maps? or within individual maps?

akhudek20:10:42

I mean that for {:a 1 :b 2} and {:a 2 :b 3} :b of the first and :a of the second must match, and then for {:a 2 :b 3} and {:a 3 :b 4}, we have the same check and so on

ghadi20:10:50

you have a tradeoff here

ghadi20:10:37

You can individually check pairs by running (map #(s/valid? ....) (partition 2 1 collection))

ghadi20:10:52

iff you want more targeted error messages

ghadi20:10:09

otherwise you can continue to check the whole collection and get errors at the collection level

ghadi20:10:22

depends on what you want

akhudek20:10:32

ok, thanks, that’s what I thought but wanted to make sure there wasn’t some other way 🙂

ghadi20:10:53

I don't believe that (at this time) there is a have-your-cake-and-eat-it-too...