Fork me on GitHub
#clojure-spec
<
2019-10-24
>
ag18:10:35

do we have any libraries that can generate specs based on DB schema, e.g.: postgress DDLs?

ag20:10:10

has anyone dealt with type of a problem where you need a spec for a map with underscored keys (for data coming from DB) and absolutely same spec with the only difference being that keys are hyphenated? Maybe someone wrote a macro?

ikitommi20:10:00

There is no way to close a Spec2 Select? e.g. dont’ allow any extra keys, right?

ikitommi20:10:29

So, if i have s Schema with ::id, ::name, ::description and ::secret keys and I would like to make an api where you must send ::id & ::name, optionally ::description, but not ::secret, I would first have to make an (api) Schema with just ::id, ::name and ::description close that (so the ::secret becomes illegal) and make a select for that schema with just ::id and ::name defined so the ::description becomes optional. Right?

seancorfield20:10:51

Spec 2 uses an option hash map to specify which specs are closed -- at checking time (`s/valid?` s/conform etc)

seancorfield20:10:09

select is always open. It just says what is required.

ikitommi21:10:20

Thanks, that’s my understanding too. So, to support a use case that one can’t send the ::secret, I need to create a concrete new Schema to be able to mark is as closed.

seancorfield21:10:39

The checking code can determine which specs are treated as closed -- independent of schema / select.

ikitommi21:10:30

(s/def ::id int?)
(s/def ::name string?)
(s/def ::description string?)
(s/def ::secret boolean?)

;; the internal schema
(s/def ::user (s/schema [::id ::name ::description ::secret]))

;; the api schema
(s/def ::user2 (s/schema [::id ::name ::description]))

;; the api select (id & name mandatory, description optional)
(s/def ::user3 (s/select ::user2 [::id ::name]))

;; closed validation
(s/valid? ::user3 {::id 1, ::name "kikka"} {:closed #{::user2}})

vlaaad21:10:04

i wonder why closed specs are implemented as options instead of as another type of spec

seancorfield21:10:26

@vlaaad Alex talks about that on the Inside Clojure blog as Spec 2 went down the "closed spec" path first and then changed to "closed checking".

vlaaad21:10:14

(s/close-specs ::s) this is some mutability thing, I was talking about having closed specs as different specs, no mutability attached

vlaaad21:10:20

like that:

(s/def ::open-keys (s/keys :req-un [::some-key]))

(s/def ::closed-keys (s/closed ::open-keys #{::open-keys}))

vlaaad21:10:08

I think that makes sense: being able to refer to your data specification by name, instead of by combining a name and an arguments to s/valid?

seancorfield21:10:49

I'll leave it up to @alexmiller to respond. I prefer the path they've taken where "closedness" is just an option on checking.

vlaaad21:10:52

If you have it as another spec it's still opt-in, but now it's also composable

Alex Miller (Clojure team)21:10:46

It’s not composable - negation is inherently problematic as specs evolve

vlaaad07:10:45

I don't get it, can you explain? I thought about closed specs for a bit, and the way I see them, they can be just a s/select counterpart, where s/select describes lower bound of data shape ("be at least that"), while s/closed describes upper bound ("be at most that"). By composable I mean using same specs both as open and as closed in same s/valid? etc. checks. I think it might make sense, for example, if I were to spec a function that cleans user input before putting it into db, it's spec would look like that:

(s/fspec
  :args (s/cat :user ::user)
  :ret (s/closed ::user))
Speaking of specs evolution, do you mean being able to tell if change of spec from a to b is accretive or not? I don't see problems here as well: - changing (s/closed (s/keys [::name])) to (s/closed (s/keys [::name ::age])) in return position provides more — it returned ::name, now it also can return ::age; - changing (s/closed (s/keys [::name])) to (s/closed (s/keys [::name ::age])) in argument position requires less — it didn't accept ::age, now it accepts it; - changing (s/closed (s/keys [::name])) to (s/keys [::name]) in return position provides more — it used to return only ::name, now it may return more.

👌 8
Alex Miller (Clojure team)12:10:45

those words do not match what you're saying in the specs. the closed version says "MUST NOT contain ..." 1/3. is probably ok from the consumers point of view but is contextual to use 2. if you provide age on the first one, then forbid on the second one, you've broken users that previously passed age

Alex Miller (Clojure team)12:10:32

this notion of context is critical and is driving the changes in schema/select/fdef, so I wouldn't rule out some way to make a more restrictive statement at the point of use, although we currently think of that again as more of a "check", not part of a spec

vlaaad13:10:56

> those words do not match what you're saying in the specs what words?

vlaaad13:10:47

maybe we have different definitions of closed? My (s/closed [::name ::address]) says that map may be empty, or contain ::name, or contain ::address, or contain both, but no extra keys are allowed

vlaaad13:10:13

like select, that requires at least this and allows more, but in different direction: requires at most this and allows less. in different contexts select, closed and a combination of both might be useful

vlaaad13:10:52

so in "2" you could not provide ::age before: this is invalid, and version after allows it

ikitommi21:10:51

What if they were part of the select?

Alex Miller (Clojure team)22:10:17

well, again the principle here is that "closedness" is part of the checking, never part of the spec

ikitommi04:10:31

I agree on that, but have thought that select is a utility of making checking context (values!) from schema. It would allow one place & syntax to easily close any of the nested maps too. The current syntax of closing specs by name in s/valid? with different (nested maps) syntax adds more things for the developer to keep in his/her head. Need to jump to both schema and select definitions to see what subkeys are including in the select and need to be closed. Also, if schemas & selects get refactored, the closed options might get out-of-sync.

ikitommi21:10:15

(s/select ::user [::id ::name s/closed])

ikitommi22:10:44

Also:

(s/select ::user [::id ::name [s/optional ::description] s/closed])

ikitommi22:10:25

In the code I pasted above, keysets need to be introduced in three places: the root schema, the api schema and the select. Also, partially in the s/valid? call to make it closed.

seancorfield22:10:45

I was a bit surprised by this:

user=> (require '[clojure.spec-alpha2 :as s])
nil
user=> (s/def ::thing (s/schema [::a ::b ::c]))
:user/thing
user=> (s/valid? (s/select ::thing [::a ::b]) {::a 1 ::b 2 ::c 3} {:closed #{::thing}})
true ; fine -- ::c is optional but present
user=> (s/valid? (s/select ::thing [::a ::b]) {::a 1 ::b 2} {:closed #{::thing}})
true ; fine -- ::c is optional and omitted
user=> (s/valid? (s/select ::thing [::a ::b]) {::a 1 ::b 2 ::d 4} {:closed #{::thing}})
true ; huh? -- ::d is not in ::thing -- why doesn't this fail?
user=> (s/valid? (s/select ::thing [::a ::b]) {::a 1 ::d 4} {:closed #{::thing}})
false ; because ::b is required but omitted... see next...
user=> (s/explain-str (s/select ::thing [::a ::b]) {::a 1 ::d 4} {:closed #{::thing}})
"#:user{:a 1, :d 4} - failed: (fn [m] (contains? m :user/b))\n"
user=> 
@alexmiller is that expected or a bug?

seancorfield22:10:34

user=> (s/def ::sub-thing (s/select ::thing [::a ::b]))
:user/sub-thing
user=> (s/valid? ::sub-thing {::a 1 ::b 2 ::d 4} {:closed #{::sub-thing}})
true
user=> (s/valid? ::sub-thing {::a 1 ::b 2 ::d 4} {:closed #{::thing}})
true
(still puzzled)

Alex Miller (Clojure team)22:10:55

there's some lingering tech debt in the schema/select code where some things are duplicated rather than having select lean on schema's validation code. I think once that's cleaned up, these will fail as you expect

4
ikitommi04:10:31

I agree on that, but have thought that select is a utility of making checking context (values!) from schema. It would allow one place & syntax to easily close any of the nested maps too. The current syntax of closing specs by name in s/valid? with different (nested maps) syntax adds more things for the developer to keep in his/her head. Need to jump to both schema and select definitions to see what subkeys are including in the select and need to be closed. Also, if schemas & selects get refactored, the closed options might get out-of-sync.

vlaaad07:10:45

I don't get it, can you explain? I thought about closed specs for a bit, and the way I see them, they can be just a s/select counterpart, where s/select describes lower bound of data shape ("be at least that"), while s/closed describes upper bound ("be at most that"). By composable I mean using same specs both as open and as closed in same s/valid? etc. checks. I think it might make sense, for example, if I were to spec a function that cleans user input before putting it into db, it's spec would look like that:

(s/fspec
  :args (s/cat :user ::user)
  :ret (s/closed ::user))
Speaking of specs evolution, do you mean being able to tell if change of spec from a to b is accretive or not? I don't see problems here as well: - changing (s/closed (s/keys [::name])) to (s/closed (s/keys [::name ::age])) in return position provides more — it returned ::name, now it also can return ::age; - changing (s/closed (s/keys [::name])) to (s/closed (s/keys [::name ::age])) in argument position requires less — it didn't accept ::age, now it accepts it; - changing (s/closed (s/keys [::name])) to (s/keys [::name]) in return position provides more — it used to return only ::name, now it may return more.

👌 8