Fork me on GitHub
#clojure-spec
<
2016-06-10
>
sparkofreason02:06:32

Is there a straightforward to define a function spec and reuse with multiple fdef's?

mpenet07:06:37

@rickmoynihan @danielcompton : same here, also free type hinting, custom explain messages, performance and the list goes on. So far we have no reason to migrate really, we're waiting to see where clj.spec is going, it's probably a bit too early

ikitommi13:06:37

Besides coercions, my top feature request to spec: helper-fn to create a spec from a vanilla clojure (function) var. It would understand the Clojure destructuring syntax. Something like this:

(require '[clojure.spec :as s])

(s/def ::age integer?)
(s/def ::fullname string?)
(s/def ::role keyword?)

(defn doit [{:keys [::fullname ::age ::role] :or {:boss ::role}}] [fullname age role])

(-> #'doit s/extract-spec s/describe)
; => (keys :req [:user/age :user/fullname] :opt [:user/role])
* the responsibility to extract (and use) the specs would be on the user - I would use these on the web-tier to auto-extract docs & do coercion in the web-api tier with our libs * would not add new meta-data to vars (arguments are already in :arglists) * no need to describe the shape of the data twice (both for the function arguments & for it's spec) * (optionally the :ret and :fn could be read from the Var metadata too) Thoughts?

angusiguess14:06:03

I think this is rad, my only real question is why might we rely on spec to do it?

angusiguess14:06:34

There are pretty good hooks to pull validation information out of a defined spec and an alternate defn like this should be a macro.

angusiguess14:06:13

I don't work on spec so this is grain of salt stuff, but it seems like the opinion of spec is that people could conceivably bring their own sugar but under the hood there's a common language for validation.

alexmiller14:06:14

@ikitommi: you could build that from what exists now. since it wouldn't generically apply, I don't think we would do that as part of core or anything. One thing that would help is a spec for destructuring, which I have and which will be released at some point in some form (details TBD still)

ikitommi14:06:25

@alexmiller: spec for destucturing sounds cool. Did a dummy version of the extractor, will play more with it. Are there any caveats in playing with :arglists?

alexmiller14:06:25

there are a few cases where people have abused it a bit in what was put in it (data.generators is one that comes to mind) but generally should be fine

alexmiller14:06:32

I think I would also consider allowing overrides via lookup in the registry - s/fdef registers stuff there under the fn symbol and those can be obtained via s/fn-specs

alexmiller14:06:51

so you have an existing registry for overrides of things you couldn't build automatically

alexmiller14:06:08

also note that CLJ-1919 will add a new syntax for namespaced key destructuring.

alexmiller14:06:55

your example there is not syntactically correct btw - the keys of :or should always be unqualified symbols (matching the bindings that are created). there are some bugs in this area in current Clojure that will be fixed in CLJ-1919.

alexmiller14:06:20

so that is, what you have there probably works now, but by accident not intent, and will change

ikitommi14:06:13

uh, copy-paste error in the code. But thanks! will check out your pointers.

wilkerlucio15:06:16

what's the correct way to express to an fspec that a function takes no arguments?

gfredericks16:06:36

(s/cat) I'd guess

gfredericks16:06:54

maybe #{()} would work too

gfredericks16:06:46

alexmiller: there's no reason not to add specs to test.check is there?

gfredericks16:06:09

assuming it accounts for older clojures

gfredericks16:06:52

I guess this question is a superset of seancorfield's question on the ML, but also about test.check in particular since it's used in clojure.spec

seancorfield16:06:04

I haven't moved forward with that since the first cut. Want to see more discussion on the ML first.

seancorfield16:06:52

(sorry if I don't follow up for a few hours -- doors closing en route for a cat show!)

alexmiller16:06:36

None other than that it then requires Clojure 1.9

alexmiller16:06:53

Which requires test.check

gfredericks16:06:41

okay, cool; I'll probably do a separate .specs namespace like sean did

jcf17:06:55

Anyone tried writing specs for stateful objects like database connections, or a Datomic database?

jcf17:06:51

I can't think of a nice way to specify a function takes a db, and some other args. To generate a db I need a database connection that isn't available when I define my specs.

jcf17:06:13

Imagine a trivial example like this:

(s/fdef load-entity
  :args (s/cat :db ::d/db :tx ::entity-tx)
  :ret ::entity)

(defn load-entity
  [db tx]
  (d/entity db [:entity/id (:entity/id tx)]))

jcf17:06:49

I can't generate Datomic entities either. I need a DB, which needs a connection, which needs a URI.

jcf17:06:12

Is there a way to say a spec can't be generated automatically so I can test other specs in this namespace maybe?

arohner17:06:20

isn’t your DB predicate just #(instance? datomic.whatever.Db %)?

pheuter17:06:26

@jcf: i just went through that exercise of creating a Datomic Db

pheuter17:06:38

(s/def ::db
  (s/with-gen #(instance? datomic.db.Db %)
    (fn []
      (gen/fmap (fn [facts] (-> (helpers/empty-db)
                               (helpers/transact facts)))
                entities-generator))))

jcf17:06:48

I need a generator to go with that spec @arohner.

pheuter17:06:25

(s/def ::db
  (s/with-gen #(instance? datomic.db.Db %)
    (fn []
      (gen/fmap (fn [facts] (-> (helpers/empty-db)
                               (helpers/transact facts)))
                entities-generator))))

jcf17:06:29

I've got basic predicate fns like these:

(defn db?
  [x]
  (instance? datomic.Database x))

(defn entity?
  [x]
  (instance? datomic.Entity x))

jcf17:06:47

@pheuter: what's facts, and what does (helpers/empty-db) look like?

pheuter17:06:53

where entities is a vector-distinct-by of :db/id values

pheuter17:06:16

empty-db just creates an empty datomic db that has been primed with a schema

jcf17:06:40

So you've got some hardcoded Datomic URI or something?

jcf17:06:56

You must have global state floating around, right?

pheuter17:06:18

No, we just generate random URIs, we use in-memory databases for development / testing

jcf17:06:11

That is global state. My Datomic connection is managed with components, and they're stopped/started around tests.

jcf17:06:22

I could have two Datomic databases with separate connections. That wouldn't be possible with (helpers/empty-db).

pheuter17:06:23

yes, we use mount to start and stop our connections as well

pheuter17:06:03

we dont use mount for testing though, and that’s when we generate db values

pheuter17:06:41

actually, we do use mount for certain state, and rely on dynamic values using test/use-fixtures

pheuter17:06:53

your predicates look fine, but you won’t be able to use them for generating out-of-the-box, will need to use s/with-gen

dominicm17:06:45

https://clojurians.slack.com/archives/clojure-spec/p1465578891000846 I've been trying to figure this out also. My solutions have involved macros and sideband data. Not elegant at all. I also ignored generators.

gfredericks18:06:31

seancorfield: were you thinking of having a conditional require in the .jdbc namespace? otherwise you'd have the problem of up-to-date users having to opt-in to the specs

gfredericks18:06:50

when there are use cases for the specs besides explicit testing, e.g. clojure.repl/doc

jcf19:06:02

Is there a way to merge two s/keys specs?

jcf19:06:17

I must be missing something. Back to the manual!

donaldball19:06:30

I’ve been idly thinking about the problem of db args as well, though in the sql context. The problem seems the same for datomic and jdbc though: a spec saying the db arg is e.g. a jdbc connection isn’t sufficient. You really want a spec that says this value is a jdbc connection to a database with at least a certain schema and maybe even a certain set of entities.

jcf19:06:42

Oh wait, just me being stupid apparently.

jcf19:06:43

@donaldball: I'm using this at the mo:

(defn- with-datomic
  [f]
  (let [running (-> (config/read-config :test)
                    (assoc :uri (str "datomic:mem://" (UUID/randomUUID)))
                    map->Datomic
                    component/start)]
    (try
      (f running)
      (finally
        (component/stop running)))))

(defn entity?
  [x]
  (instance? datomic.Entity x))

(s/def ::d/entity
  (s/with-gen
    entity?
    (fn [] (with-datomic (fn [{:keys [conn]}]
                           (gen/fmap
                            #(d/entity (d/db conn) %)
                            gen/int))))))

jcf19:06:19

Don't love it if I'm honest. Creating a new connection for every test is pretty inefficient, but it works.

jcf19:06:19

If you've not used Component or Datomic that's probably meaningless.

gfredericks21:06:29

I hadn't thought about this generator setup encouraging people to use stateful resources in their generators

dominicm21:06:35

@gfredericks: I'd say that the encouragement and bias towards namespace level hoisting (with s/def) takes away our ability to lexically scope and generally pass around explicit arguments. Diving too deeply into that statement takes you to mount vs component.

gfredericks21:06:12

dominicm: the way I've used generators in the past is purely data-driven, so there's not even component-like stuff until the test starts running; but if you have a spec that's explicitly for a stateful thing, then you can't write a generator for it that way

gfredericks21:06:01

my gut would be to try to keep doing the data-driven thing, and so not use generators for specs that describe stateful things

gfredericks21:06:11

not sure how easy that is with datomic

gfredericks21:06:51

a datomic entity is a map of attributes, is that right?

gfredericks21:06:03

if that's the case you could make most of your specs just expect maps

gfredericks21:06:17

and could do low-level testing with maps instead of entities

dominicm21:06:17

@gfredericks: It's definitely difficult to manage with generators. The discussion (in my opinion) transcends just testing. Doing explicit s/explain-data on a runtime database is also a use-case.

gfredericks21:06:57

that should work with map specs though I would think?

dominicm21:06:27

(s/explain-data {:email ""} (s/keys :req-un [:unique-checker/email])) The unique email checker needs a database to make it's check. If I make it part of the map data (assoc m ::db (d/db conn)), then when I check, the returned data gives me an incorrect path to the error position.

dominicm21:06:46

I have just got a macro working, which would take the error generated, and readjust the path for you. But it's still somewhat unnatural.

gfredericks21:06:09

so your spec is making database queries?

ghadi21:06:04

that is a bad idea

gfredericks21:06:12

I think specs should be pure functions

ghadi21:06:31

absolutely

gfredericks21:06:58

something that's not a pure function can be a plain ole test :)

eggsyntax22:06:46

I need to write a script to generate some Datomic seed data, and I'm experimenting with using spec to do so. Two questions: 1) One part of the seed data I need to write is the key/val that'll generate a temporary db/id. If I were hand writing the seed data, it would look like {:db/id #db/id[:db.part/user -1015948] ... }. Can anyone help me understand how I would go about creating a spec for that? 2) ideally, it'd be nice to just spec that part of the schema, and then somehow use that to generate the seed data, but I'm not quite sure how go about that. Any hints?

eggsyntax22:06:45

In other words, how can I spec a tagged literal like that?

eggsyntax22:06:49

Got it.

(defn db-id? [v] (instance? datomic.db.DbId v))
(s/def ::db-id db-id?)

eggsyntax22:06:15

(or at least that's the spec for the tagged literal itself. I'm not quite sure how to get from that to a spec that'll produce seed data as above. I guess I'll need to write a generator for it, but it's gonna take some experimentation for sure.

hiredman22:06:38

using spec just to generate data seems super weird, why wouldn't you use the generators from test.check or https://github.com/clojure/data.generators directly?

eggsyntax22:06:45

Well, if I can figure out the relationship between a spec for the schema and a spec for the seed data, I can use the spec to do validation as well as generating seed data, and maybe find other uses for it as well.

eggsyntax22:06:37

And it's also partly a clojure.spec learning exercise for myself 🙂