Fork me on GitHub
#clojure-spec
<
2020-11-03
>
borkdude12:11:11

Is there any existing work combining clojure-spec conform + core.match or datalog? Like: I want ?x and ?y from (s/conform ...)? I could see this being useful for grasp

delaguardo12:11:57

take a look at mine matchete https://github.com/xapix-io/matchete if meander is not a good fit

borkdude12:11:48

thanks. can you describe the difference between meander and yours?

delaguardo13:11:40

• no macro • less power -> cleaner pattern syntax (this is just my opinion, btw) • much smaller codebase -> much easier to read through and explore

borkdude14:11:46

@U04V4KLKC How do I express this meander usage in your lib?

(m/find (first conformed) {:clauses {:clause {:sym !interface} :clauses [{:sym !interface} ...]}} !interface)

borkdude14:11:01

The output should be something like:

[clojure.lang.IDeref clojure.lang.IBlockingDeref clojure.lang.IPending java.util.concurrent.Future]

borkdude14:11:17

And the input:

{:reify reify, :clauses {:clause {:sym clojure.lang.IDeref, :lists [(deref [_] (deref-future fut))]}, :clauses [{:sym clojure.lang.IBlockingDeref, :lists [(deref [_ timeout-ms timeout-val] (deref-future fut timeout-ms timeout-val))]} {:sym clojure.lang.IPending, :lists [(isRealized [_] (.isDone fut))]} {:sym java.util.concurrent.Future, :lists [(get [_] (.get fut)) (get [_ timeout unit] (.get fut timeout unit)) (isCancelled [_] (.isCancelled fut)) (isDone [_] (.isDone fut)) (cancel [_ interrupt?] (.cancel fut interrupt?))]}]}}

delaguardo15:11:04

almost the same for that particular case

(require '[matchete.core :as mc])

(def data
  '{:reify reify
    :clauses {:clause {:sym clojure.lang.IDeref
                       :lists [(deref [_] (deref-future fut))]}
              :clauses [{:sym clojure.lang.IBlockingDeref
                         :lists [(deref [_ timeout-ms timeout-val] (deref-future fut timeout-ms timeout-val))]}
                        {:sym clojure.lang.IPending
                         :lists [(isRealized [_] (.isDone fut))]}
                        {:sym java.util.concurrent.Future
                         :lists [(get [_] (.get fut)) (get [_ timeout unit] (.get fut timeout unit)) (isCancelled [_] (.isCancelled fut)) (isDone [_] (.isDone fut)) (cancel [_ interrupt?] (.cancel fut interrupt?))]}]}})

(def pattern
  {:clauses
   {:clause {:sym '!interface}
    :clauses (mc/scan {:sym '!interface})}})

(mc/matches pattern data)

borkdude15:11:28

This returns:

({!interface [clojure.lang.IDeref clojure.lang.IBlockingDeref]} {!interface [clojure.lang.IDeref clojure.lang.IPending]} {!interface [clojure.lang.IDeref java.util.concurrent.Future]})

borkdude15:11:40

Is there a way to get a list of only the symbols like with meander?

delaguardo16:11:01

sorry, I was too fast copypasting ) instead of mc/scan please use mc/each

delaguardo16:11:43

scan is for inspecting items in collection individually (in separate decision branches) each is for walking over a collection

delaguardo16:11:03

(def pattern
  {:clauses
   {:clause {:sym '!interface}
    :clauses (mc/each {:sym '!interface})}})
this pattern should work

borkdude16:11:27

I get:

No such var: mc/each

delaguardo16:11:14

you are using experimental 2.0.0 version check out 1.2.0

borkdude16:11:24

no, I was using the version that is suggested in the README

$ clj  -Sdeps '{:deps {io.xapix/matchete {:mvn/version "1.1.0"}}}'
:)

delaguardo16:11:27

argh, cljdoc badge got updated for master branch as well,

delaguardo16:11:19

then this is my bad, I should update it there

borkdude16:11:11

no problem!

borkdude16:11:56

Excellent, it works now!

user=> (def conformed (map #(s/conform ::reify %) matches))
#'user/conformed
(def pattern
  {:clauses
   {:clause {:sym '!interface}
    :clauses (mc/each {:sym '!interface})}})
#'user/pattern
user=> (mc/matches pattern (first conformed))
({!interface [clojure.lang.IDeref clojure.lang.IBlockingDeref clojure.lang.IPending java.util.concurrent.Future]})

borkdude17:11:59

Nice, it seems your lib also works with babashka:

$ bb -cp "$(clojure -Spath)" -e "(require '[matchete.core :as mc]) (mc/matches '{:x ?x} {:x 1})"
({?x 1})

delaguardo17:11:27

another pusher to polish documentation 😉

borkdude12:11:21

Maybe meander is a good match for this

Jim Newton15:11:04

I'm starting looking at clojure.spec.alpha can someone explain the different between the s/valid? function and the s/spec? function? Their docstrings look pretty similar.

borkdude15:11:58

@jimka.issy spec? is more like an instance check, valid? validates a value against a spec

borkdude15:11:36

in fact, it is an instance check :)

Jim Newton15:11:40

As I understand the normal use case is the user associates certain tags, normally starting with :: with some DSL object which tells how to check validity? Does spec? check whether the symbol has been associated with a spec object, or does it check whether this is such an object?

Jim Newton15:11:01

How can I know whether a symbol such as ::xyzzy designates a spec or not?

borkdude15:11:44

user=> (s/def ::foo int?)
:user/foo
user=> (s/spec? ::foo)
nil
user=> (s/spec? (s/get-spec ::foo))
#object[clojure.spec.alpha$spec_impl$reify__2059 0x4d48bd85 "[email protected]"]

borkdude15:11:30

So get-spec returns the spec for a keyword or a symbol (in case of fdef)

borkdude15:11:08

@jimka.issy There is also s/spec:

user=> (s/spec ::foox)
Execution error at user/eval162 (REPL:1).
Unable to resolve spec: :user/foox
user=> (s/spec ::foo)
#object[clojure.spec.alpha$spec_impl$reify__2059 0x4d48bd85 "[email protected]"]

borkdude15:11:48

fwiw, I don't use these functions a lot, except maybe when I'm writing tools around spec. Not in normal daily spec usage

borkdude15:11:25

s/def, s/valid? and s/conform are probably the most common ones I use

Jim Newton16:11:18

After something like the following:

(s/def ::big-even (s/and int? even? #(> % 1000)))
can I call some function on ::big-even to get back (s/and int? even? #(> % 1000) ?

borkdude16:11:34

almost:

user=> (s/def ::big-even (s/and int? even? #(> % 1000)))
:user/big-even
user=> (s/form ::big-even)
(clojure.spec.alpha/and clojure.core/int? clojure.core/even? (clojure.core/fn [%] (clojure.core/> % 1000)))

Alex Miller (Clojure team)16:11:29

s/describe returns a shorter form, useful for printing (but not fully resolved so not too useful as data)

Jim Newton16:11:38

seems like s/form does what I want. But it's not certain.

Alex Miller (Clojure team)16:11:03

research question - if you are using specs with aliases, do you tend to use one alias, a few, or many in any given namespace?

borkdude16:11:18

what do you mean by an alias?

borkdude16:11:41

like (alias 'foo 'bar) and then ::bar/spec ?

borkdude16:11:05

@alexmiller I think grasp could be used to find this out

Lennart Buit16:11:39

one reason for me to use many aliases would be if many keys in a s/keys require a same generator. For one, we have a present string spec, with corresponding generator that we alias a lot

Lennart Buit16:11:34

ah nevermind then, I interpreted this as an alias:

(s/def ::my-spec ...)
(s/def ::key-2 ::my-spec)
(s/def ::key-1 ::my-spec)
(s/def ::map (s/keys :req [::key-2 ::key-2]))

borkdude16:11:17

hmm, never mind, grasp returns the fully qualified keyword, so you can't see if it was aliased or not

Alex Miller (Clojure team)16:11:04

but really wondering what people are doing in private code bases

borkdude16:11:31

we usually don't use alias for this, since a lot of our specs are in CLJS. We create namespaces on disk for this

borkdude16:11:08

@alexmiller typically we have this in our backend, using on disk namespaces:

(s/def ::system
  (s/merge ::dict/system
           ::adis/system
           ::annotator/system
           (s/keys :req [::database/db ::cache/cache])))

Jim Newton17:11:57

More spec questions. When I define ::big-even as in https://clojure.org/guides/spec.

(s/def ::big-even (s/and int? even? #(> % 1000)))
I'm guessing that the function #(> % 1000) is a closure in its current lexical context. Right? So later when I call s/form I don't get back that object, but some new text, which doesn't know the original lexical context. Right?
(s/form ::big-even)
 ==> (clojure.spec.alpha/and
 clojure.core/int?
 clojure.core/even?
 (clojure.core/fn [%] (clojure.core/> % 1000)))

borkdude17:11:38

@jimka.issy The text #(...) is not representable as an s-expression, it's something built into the reader which expands to (fn [...] ...)

borkdude17:11:23

What simply happens is that s/def is a macro which captures the input and by that time the s-expression has already expanded to the fn form

Alex Miller (Clojure team)17:11:14

the spec function will capture the lexically scoped values at the point of definition (but the form will not - it will have the names)

Alex Miller (Clojure team)17:11:32

user=> (let [x 100] (s/def ::foo (s/and int? #(= % x))))
:user/foo
user=> (s/valid? ::foo 100)
true
user=> (s/valid? ::foo 99)
false
user=> (s/form ::foo)
(clojure.spec.alpha/and clojure.core/int? (clojure.core/fn [%] (clojure.core/= % x)))

Alex Miller (Clojure team)17:11:54

^^ form is broken there

Alex Miller (Clojure team)17:11:05

this is all somewhat more consistent in spec 2 (namely, this will not work at all as the divide between symbolic specs and spec objects is clearer) but there is built-in support for creating parameterized specs (like the one above) if needed

Jim Newton08:11:46

@alexmiller, thanks for the explanation. however, i'd like to get the actual function object which spec would use to validate with. For example given something like (s/and int? #= % x)), I can use int? to get the int? function, but I can't use the s-expression form of the closure to recuperate the closure. However, I'm pretty sure spec has the closure somewhere, otherwise s/valid? would not work.

Jim Newton08:11:52

What I'm trying to do is figure out which spec incantations can be expressed in a simple type system which I have implemented, called genus , normally abbreviated gns/. For example, a naïve first attempt is able to translation ::big-even as follows

clojure-rte.rte-core> (require '[clojure.spec.alpha :as s])
nil

clojure-rte.rte-core> (s/def ::big-even (s/and int? even? #(> % 1000)))
:clojure-rte.rte-core/big-even

clojure-rte.rte-core> (s/get-spec ::big-even)
#object[clojure.spec.alpha$and_spec_impl$reify__2183 0x61966631 "[email protected]"]

clojure-rte.rte-core> (s/form ::big-even)
(clojure.spec.alpha/and
 clojure.core/int?
 clojure.core/even?
 (clojure.core/fn [%] (clojure.core/> % 1000)))
The important result being the next step:
clojure-rte.rte-core> (gns/canonicalize-type '(spec ::big-even))
(and
 (or Long Integer Short Byte)
 (satisfies clojure.core/even?)
 (spec (clojure.core/fn [%] (clojure.core/> % 1000))))
So genus can unwind the spec into types and predicates, if the predicates are symbolic. But what's left (clojure.core/fn [%] (clojure.core/> % 1000)) is not something which is useable.

Jim Newton09:11:26

I could try to write a translator/compiler for expressions such as (clojure.core/fn [%] (clojure.core/> % 1000)) which try to convert them back to a closure, in the case that they don't have any free variables. But that work seems redundant since spec already has the closure in its internal data structure.

Jim Newton09:11:04

I just realized something interesting. If I eval the list (clojure.core/fn [%] (clojure.core/> % 1000)) then I get a function object which seems to have the same semantics as the original function, in case there are no free variables.

clojure-rte.rte-core> ((eval '(clojure.core/fn [%] (clojure.core/> % 1000)))  12)
false
clojure-rte.rte-core> ((eval '(clojure.core/fn [%] (clojure.core/> % 1000)))  100000)
true
clojure-rte.rte-core> 
Perhaps that is good enough for my proof-of-concept.

Jim Newton09:11:56

And if there's a free variable, eval throws an exception

clojure-rte.rte-core> (eval '(clojure.core/fn [%] (clojure.core/> x 1000)))
Syntax error compiling at (clojure-rte:localhost:51477(clj)*:45:51).
Unable to resolve symbol: x in this context

Alex Miller (Clojure team)14:11:53

Well eval is literally the function that takes a form and returns a function

Alex Miller (Clojure team)17:11:07

user=> (s/defop narrow-int [x] (s/and int? #(= % x)))
#'user/narrow-int
user=> (s/def ::foo (narrow-int 100))
:user/foo
user=> (s/form ::foo)
(user/narrow-int 100)
user=> (s/valid? ::foo 100)
true
user=> (s/valid? ::foo 99)
false