@bfabry no promises that it's the most elegant approach. I'm looking forward to seeing how others work with spec forms.

heh, I'm actually working on something to define graphql forms from specs definitions

unrelated: you’re using graphql? d’you like it?

i haven’t used it myself but the talks i’ve seen on it are pretty compelling

@mattly that's on my radar too. We are replacing our simple "datomic style" pull api with graphql soon. Here's what I came up with generating recursive pull specs from clojure.spec forms: https://gist.github.com/olivergeorge/9d822447e4838092d07138ea51af5782

oh neat

(again, no promises that it's a good approach... really want to see other peoples code and learn some effective techniques)

I'm going to keep my pull-style api, but aim to hookup a graphql parser to it

That makes sense.

mostly because some of my query endpoints, I need expressive arguments

We bolted a "pull" arg to our REST endpoints which was a practical step forward at the time. Same old REST params etc but more flexibility around what data we return. Now that GraphQL is looking fairly stable we're intrested in standardising on it's interface. That decouples our frontend from the backend which should allow us some flexibility.

I'm fairly lucky in that mine is a greenfield project

Nice. We're in the enterprise space so typically the backend tech stack is dictated to us.

I may try to abstract out my resolver into a library

it has a separate planning stage to help with the n+1 problem

@mattly I'd love to see what you come up with.

I'm not sure it's related but I did some thinking around translating graph queries into flat sql queries.

This paper gave me the idea: http://homepages.inf.ed.ac.uk/slindley/papers/shredding.pdf

eh, removed the link, there's a bug in that

I need to properly extract it

🙂

@olivergeorge any thoughts about reversing direction and generating spec definitions from a sql schema?

@mikeb I've had some fun with that actually. It's easy to produce something for each column in a database but the relations are trickier.

Here's some related code. https://gist.github.com/olivergeorge/468464ce82b8da486736fe725a4b6ff8

The slightly fiddly bit is that you can't really do it dynamically. At least I can't work out how to have my script generate and run macros. In the end I generated some forms and committed them as code.

I was looking into clojurescript code and I saw that clojure/core/specs.clj from clojure is not there. Why clojure/core/specs.clj hasn’t been ported into clojurescript?

@dnolen is it on purpose?

Just haven't gotten to it yet

would you be interested in a patch?

I see this in the spec guide: "check also takes a number of options that can be passed to test.check to influence the test run, as well as the option to override generators for parts of the spec, by either name or path." Any examples of the "path" option for generator overrides?

i’d love to see an example of generator override usage, path or no

@jrheard here's a basic example with override by name:

(defn foo [x] (println "FOO" x) (inc x))
(s/fdef foo :args (s/cat :x ::x))
(s/def ::x clojure.future/int?)
(stest/check `foo
             {:gen          {::x #(gen/return 10)}
              ::stestc/opts {:num-tests 1}})

thanks!

gen comes from clojure.test.check, not from spec, right?

But not sure how to do it when the spec is defined like this:

(s/fdef foo :args (s/cat :x clojure.future/int?))

[clojure.spec.gen :as gen]

nice, thx

@mlimotte I think you need to do (s/fdef foo :args (s/cat :x (s/spec clojure.future/int?)))

The foo spec above seems to work even w/out (s/spec ...). I'm still not sure how to do a generator override for this case where the spec is not named, though.

Oh, I see - sorry, misread the conversation!

np

i'm sorry about this naive question: if i were to spec a map that has strings as keys or even another type, i would have to convert them to keywords first before passing them into spec?

i don't think so, do you have an example of what you're trying to do?

@naomarik no. you need keywords for keys to use s/keys but there's still lots of other predicates that you can use to validate maps, including writing your own

i've been struggling to find an example for this, let's say just validating something simple like {"type" "tweet" "user" {"type" "registered"}}. I would know how to go about doing this from reading the official guide if these were keywords, but not like this

i think you're saying that those are required keys. So, like bfabry said, you can't use s/keys. you have to write your own predicates to do that.

@naomarik it depends on what you want to validate. if you know what keys you're going to get in advance and what their semantics are and which ones are required then I'd recommend converting them to keywords. it just makes sense. if the keys are dynamic then I would leave them as strings and validating other properties about the map

@naomarik if it's just required keys, a simple predicate isn't too bad:

(s/def ::my-map (s/and map? #(every? (partial contains? %)  #{"type" "user"})))
(s/valid? ::my-map {"type" "tweet" "user" {"type" "registered"}})
=> true
(s/valid? ::my-map {"type" "tweet" "NotUser" {"type" "registered"}})
=> false

ah i see

s/keys is a lot more convenient at that point

yeah like I said, if you know what keys your map contains I would convert them to keywords. for lots and lots of good reasons including that you get nice spec tools for working with them

i always write all my maps as keywords, but is there a discussion online why namespaced keywords are enforced so i can be enlightened?

there's been a few, I think maybe the cognicast podcast with rich hickey talking about spec might be the best bet. they're not enforced though

s/keys has corresponding :req-un and :opt-un options that you can use for un-namespaced keywords

Yeah but some stuff is brittle with un- keys ex broken conforming with s/merge

eh?

See recent discussion about it in history

Clj-1981

alex's comments seem to indicate that's confusion over how conforming works with merge, and isn't related to namespaced keys

Wrong ticket, my google fu failed me

http://dev.clojure.org/jira/browse/CLJ-2033

he's saying only the last spec in the merge conforms

ohh wait no I understand

ok, but there's no way to fix that

Never saw the point in ns keys personally.

you get behaviour for free when using namespaced keys. but it's just not possible if they're not namespaced

well. this would be one of the points. you get conforming for free

I think perhaps alex's example hasn't explained what's happening properly. this might help

boot.user=> (require '[clojure.spec :as s])
nil
boot.user=> (defn convert [n] (if (double? n) n (double n)))
#'boot.user/convert
boot.user=> (s/def ::value (s/conformer convert))
:boot.user/value
boot.user=> (s/conform (s/keys) {::value 5})
#:boot.user{:value 5.0}
boot.user=>

so even though s/keys doesn't specify ::value as req or opt, because ::value is namespaced and has a spec associated it automatically gets conformed and validated

so it's not that merge behaves differently with namespaced keys, it's that s/keys behaves differently with namespaced keys (it gives you free stuff if they're namespaced)

Something weird about s/and:

user=> (s/def ::ff (s/cat :start integer? :end integer?))
:user/ff
user=> (s/def ::gg (s/* ::ff))
:user/gg
user=> (s/explain-str ::gg [4 7 2 1])
"Success!\n"
user=> (s/def ::ff (s/and (s/cat :start integer? :end integer?)))
:user/ff
user=> (s/explain-str ::gg [4 7 2 1])
"In: [0] val: 4 fails spec: :user/ff predicate: (cat :start integer? :end integer?)\n"

Why in the second case - after adding s/and the vector doesn’t conform?

Regex.

Your ::gg spec is zero or more integer-followed-by-integer patterns.

Because s/* and s/cat combine as a regex sequence.

When you insert s/and you isolate the the pair spec so ::gg becomes zero or more pairs

i.e., [[4 7] [2 1]]

What is the way not to isolate the pair?

My use case is that I need extra-validation

(s/def ::ff (s/and (s/cat :start integer? :end integer?)
                   #(< (:start %) (:end %))))

(s/def ::gg (s/* ::ff))

Makes sense?

@seancorfield

So you want a sequence of integers that is even in length and when you partition that sequence into pairs, each pair is ordered…?

exactly!

So (s/and seq-of-ints length-is-even every-partitioned-pair-is-ordered)

@bfabry Yep, thats what I meant. The fact that ns keys are so deeply rooted in spec makes this stuff (merge for instance) counter intuitive.

@seancorfield But I want to do it in an idiomatic clojure.spec way

again, I'd say it's (s/keys) that's counter-intuitive, not s/merge, but yeah possibly the automatic conform/validate nature of namespaced keywords could be called out more

Actually, my real need is to build a spec for sequence of integers that are palindromes...

I tried:

(s/def ::short-palindrome 
  (s/and (s/cat :a integer?
                :rest ::palindrome
                :a integer?)
         #(= (:a1 %) (:a2 %))))

(s/def ::palindrome (s/* ::short-palindrome))

I know, I never said merged was bugged, but that s one of the places where this behavior shows through in a bad way

But it only accepts nested sequences like [[1 [ 2 2] 1]] but not flat sequences like that: [1 2 2 1]

how would you write the spec for palindrome @seancorfield ?

For a sequence that is a palindrome? (s/and (s/coll-of integer?) #(= % (reverse %)))

(untested)

testing it...

It works (except for empty sequences)

But...

I would like to write it like a context-free grammar

Maybe that’s not the idea with clojure.spec @seancorfield ?

You probably want to ask @alexmiller ...

Now, I’m trying to implement the grammar for algebraic expressions - as described here https://en.wikipedia.org/wiki/Context-free_grammar#Algebraic_expressions:

Here is a context-free grammar for syntactically correct infix algebraic expressions in the variables x, y and z:

S → x
S → y
S → z
S → S + S
S → S - S
S → S * S
S → S / S
S → ( S )

Do I have a chance to succeed @seancorfield and @alexmiller ?

I would have written something like this for palindromes (& is much better than and here as it stays in regex)

(s/def ::pal 
  (s/alt :0 (s/cat)
         :1 int?
         :n (s/& (s/cat :a int? :b ::pal :c int?) (fn [{:keys [a c]}] (= a c)))))

you can use the same approach for your algebraic expressions

great

I’m trying it now

::pal works well

working on ::algebraic-expression...

@alexmiller what’s the exact meaning of (cat) with no arguments?

Is it like empty??

yes

it’s a sequential collection with no contents

but it’s a regex so will compose better with other regex ops

in particular, it will be treated in the same sequential context here, not be a separate (nested) collection

thx

Ah, yes, I still forget about s/&...

@alexmiller Now, I’m having stackoverflow issues

😞

(s/def ::arithmetic
  (s/alt
    :var int?
    :plus (s/cat :a ::arithmetic :op #{"+"} :b ::arithmetic)))

(s/explain-str ::arithmetic [1 "+" 1])

yeah, that's not going to work

there are other typical approaches to writing left-recursive grammars like this

left-recursion is not (yet) supported in clojure.spec?

this is just a consequence of how the regex logic works

you mean the deriv and accept-nil? functions

- that implement the ideas of the “Parsing with derivatives” paper ?

spec regex walks all viable alternatives in parallel - in this case, that becomes an ever-growing set

but you can rewrite those kinds of grammars like

(s/def ::ar (s/cat :a int? :r (s/? (s/cat :op #{"+”} :b (s/alt :i int? :e ::ar)))))

in reality, a) these problems don’t come up that often and b) prefix-factoring is an option

By “prefix-factoring” you mean that the recursion appears last or that it doesn’t appear first?

@viebel you can see why I deferred to Alex on the "context-free grammar" aspect of clojure.spec, eh? 🙂

Yeah! It’s hard stuff

Now, I’m struggling with the parentheses

(s/def ::my-int (s/* #{\0 \1 \2 \3 \4 \5 \6 \7 \8 \9 \x \y \z}))
(s/def ::ar (s/alt :operation (s/cat :a ::my-int      
                                     :r (s/? (s/cat :op #{\+ \*  \- \/}
                                                    :b (s/alt :i ::my-int :e ::ar))))
                   :parentheses (s/cat :o #{"("}
                                       :b (s/alt :i ::my-int :e ::ar)
                                       :c #{")"})))

It works for (s/explain-str ::ar (seq "x+(y*x)”))

But with (s/explain-str ::ar (seq "(y*x)+x”)) I get:

In: [5] val: ("+" "x") fails spec: :my.spec/ar predicate: (alt :operation (cat :a :my.spec/my-int :r (? (cat :op #{"+" "*" "-" "/"} :b (alt :i :my.spec/my-int :e :my.spec/ar)))) :parentheses (cat :o #{"("} :b (alt :i :my.spec/my-int :e :my.spec/ar) :c #{")"})),  Extra input

@alexmiller please help (again) 😰

Sorry I'm off duty for the night :)

No problem: Actually I’m in Israel and it’s 12:30 AM - I’m too tired to continue with this tough topic. We’ll catch up later. If you find something please mention me on slack.

Is there an idiomatic way to define a spec for a constant value? Something like (s/def ::some-constant #(identical? % "foo"))?

You can use set like this: #{“foo”}

because sets behave like functions

That works, thanks

yes, sets are the idiomatic way to do that, sets also work as generators (while #(identical? % "foo") does not)