This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-04-20
Channels
- # beginners (27)
- # calva (32)
- # cider (9)
- # clojure (111)
- # clojure-spec (71)
- # clojure-uk (7)
- # clojurescript (22)
- # cursive (20)
- # devcards (1)
- # emacs (4)
- # fulcro (3)
- # hyperfiddle (3)
- # off-topic (8)
- # pathom (26)
- # planck (19)
- # quil (4)
- # re-frame (1)
- # reitit (43)
- # rewrite-clj (9)
- # shadow-cljs (13)
- # spacemacs (7)
- # uncomplicate (5)
so, we use records and have a macro that generates a spec which includes checking fields as well as asserting the object is an instance of the backing record class (the latter part can also be used in isolation as a fast check using instance?). because of this, I didn’t realize keys generates open specs
what’s the use case for a spec that describes keys but permits maps that have none of those keys?
@mrevelle One observation about records is that they are open too -- you can assoc in any additional keys you want and it remains an instance that record type.
yeah, and that makes sense. but my expectation is that a record (or keys spec) provides a base set of keys that should be present
My view of :opt in s/keys is that it mostly exists for generative testing (and partly for documentation).
(s/def ::f string?)
(s/def ::l string?)
(s/def ::s (s/schema [::f ::l]))
(s/valid? ::s {::x 10}) ;; "extra" keys are ok
;;=> trueselect is how you specify required-ness in spec2.
"Schemas do not define "required" or "optional" - it's up to selections to declare this in a context."
user=> (s/valid? (s/select ::s [::f ::l]) {::x 10})
false
user=>user=> (s/valid? (s/select ::s [::f ::l]) {::x 10 ::f "F" ::l "L"})
true
user=>
The schema/`select` stuff is going to be very helpful for us -- it will simplify our API specs quite a bit. Instead of needing to have different specs for otherwise very similar API calls, we can have a schema for the "family" of parameters that can be passed in and then use select to specify what a particular API requires.
I'm still thinking about how it can help with the specs around our database, but I think it can help with the required-for-insert vs required-for-update scenarios.
yeah, sub-spec selection seems like a good idea. I think it’s just that the undefined default behavior is to not include any of the keys/fields
@mrevelle This is not valid because ::f's value is not string? per your spec for ::f. That makes perfect sense to me.
s/keys works that way today.
Oh, I don’t disagree. Was pointing out that a unselected spec field becomes active when it matches a key even though it’s not selected.
Yes, which makes sense in the context of the way Spec has always worked.
If you added a spec for ::x as string? then (s/valid? ::s {::x 10}) would fail -- because 10 is not string?.
That's how s/keys works today. That's how s/schema and s/select work in Spec2: if you provide additional keys, they're still checked if there's a spec for them.
user=> (s/def ::y string?)
:user/y
user=> (s/valid? (s/select ::s []) {::x 10})
true
user=> (s/valid? (s/select ::s []) {::x 10 ::y 11})
false
user=> (s/valid? (s/select ::s [::f]) {::x 10 ::y 11 ::f "s"})
false
user=> (s/valid? (s/select ::s [::f]) {::x 10 ::y "a" ::f "s"})
true
again, I understand and am not disagreeing with you. But you pointed out that the docs for spec2 say use of a schema without select is undefined and I’m providing examples of how saying something is undefined doesn’t mean there isn’t a default
I didn't say it was undefined (as in "undefined behavior"). I quoted the part that says schema says nothing about required-ness -- that's what select is for -- and that decoupling is exactly what Rich was talking about in Maybe Not.
What I do think is odd, right now, is that you can select keys that are not in the schema and that makes them required.
I disagree. It's exactly what Rich talked about.
This, on the other hand, seems wrong:
user=> (s/def ::s (s/schema [::f ::l]))
:user/s
user=> (s/valid? (s/select ::s [::f ::x]) {::f "s"})
false
user=> (s/valid? (s/select ::s [::f ::x]) {::f "s" ::x 10})
true
(there's no spec for ::x but the s/select call makes it required, even tho' it isn't part of ::s)
I guess my point is, based on Rich’s talk and the current spec2 docs, (s/valid? ::s x) shouldn’t be permitted since the schema is being used without a select
In the first cut of schema/`select` that landed in the repo, schemas were not specs, so that was not allowed.
I'm not sure why the change was made to turn schemas into specs.
This is the commit that changed that https://github.com/clojure/spec-alpha2/commit/1aa141dcdadca88c25afa73e486b3707eaed9d99
I’m mostly just bummed that spec seems to be going away from being a more-flexible structural typing substitute and into something that I’m not sure fits at all with how I use Clojure
I'm not sure I follow you... select is much more powerful than s/keys -- and now you can define the overall shape of your data once, and then validate against the relevant parts of it as needed, without having to write multiple s/keys specs that all overlap.
Reading over the wiki page again in more detail, I wonder if the drive for that change (the commit above) came from wanting to treat schemas like specs because of data generation?
There's an example of generating against a schema, which produces random selections of keys from the set in the schema. Which would all be "valid" in any context that accepted that schema without placing any required-ness constraints on the contents. Seems like a slim driver to me tho'...
that’s true, it’s not that something can’t be done. but it’s unclear how compatible it is with other use cases. don’t want to end up working against the language
Alex responded to my question on that commit: "The big thing that tipped it is the unqualified key support which does enforce the specs of unqualified keys (in addition to checking the values of any qualified keys against the registry)." -- because the unqualified key schemas include the spec and don't exist outside the schema (so the schema is a useful spec mostly for unqualified keys which would not otherwise be checkable).
@alexmiller About the closed specs. I agree on @dominicm on the open/closed on the borders. We want to drop out extra keys at the borders (recursively) and also coerce the values. Inside they can be open. Spec-tools (and by so, reitit) does both stripping extra keys & coercion already, but using the s/form parsing, which feels wrong. There is CLJ-2251 I wrote about the different ways to walk the Specs from inside, not outside. Also the fast path of just validating.
i.e., based on what Rich presented and my lurking on here, I’d expect select to be used to choose a subset of keys when the full record isn’t expected or needed. having it default to none seems silly when it’s much easier to write (s/select ::s []) when none are required than enumerate all the fields in a spec when you want all to be required. I also think it’s less surprising, but that’s subjective
> when it’s much easier to write (s/select ::s []) when none are required than enumerate all the fields in a spec when you want all to be required
FYI (s/select ::s [*]) is the "all fields required" case
I would put the closed-checking into select: Schemas would be open by default, but one can define select default to open or closed maps, which can be explicitly defined to different (sub)selects too.
(s/select ::user [::id ::name ::address {::address [::street]}] [s/closed]) ;;3rd arg will be merged into all submaps
maybe moving to the select get back on the closed problem again, it will always be closed, I agree with @borkdude in this one, maybe the point to make this decision is s/valid?, this way you keep the possibility to stay open, but can do closed checks when it makes sense
with s/valid? is would be all or nothing. In s/select, one could close it just partially. Not sure if there is a real life case for that thou.
It seems you register which spec are to be validated as closed and open. So the s/valid could be partially closed as well, if you broke it up into named specs.
Independently wrote a response to the open/closed approach which pretty much echoes the same things other folks are saying above — global-stateful-ness bothers me, what about threads, etc. The need for closedness seems to me like it’s usually context-specific and scope-limited, so I was expecting a limited-scope approach. My first inclination would be to be able to do something like
(s/valid? (s/closed-variant ::bar) my-bar)
s/closedly-valid? (or strictly-valid? or strictly-correct? or whatever)? And maybe an equivalent for s/explain. Or I like what I think @wilkerlucio is suggesting, making it available as an extra-arity arg to s/valid? (off by default so it doesn’t break existing code).@eggsyntax My first thought was s/valid-closed? but then you need a variant of each of the explain* functions and conform and probably several others... which gets ugly fast...
What about maybe a separate s/closed? function that just verifies whether the data structure contains only the keys in the spec? It doesn’t seem like a big burden to do
(and (s/closed? ::spec x) (s/valid? ::spec x))@mrevelle This is not valid because ::f's value is not string? per your spec for ::f. That makes perfect sense to me.
If closed/open status becomes an option to some s/valid* fn then I don't see how the closed spec can be specified to validate args in defn-spec or something like that. So "closedness" should be attached to the spec instance.
And I guess I'd also intuitively expect (s/close-specs ::s) to return a new instance of the spec. Even though I don't yet have cases where I need to open or close the already created spec, I'd expect it to return new values instead of "mutating" existing ones. Otherwise, there's a risk to have all the old problems with mutable data like threading, reasoning about code, etc.
Should I be using spec for server side form validation or http://funcool.github.io/struct/latest/ or something else?
@somedude314 We use Spec for that. We use it for API parameter validation, server-side form validation, and several other places in our production code.
@seancorfield do you use any helper libraries built on top the spec or just vanilla spec?
Just vanilla spec. We wrote some code on top of explain-data to turn Spec failures into domain-specific error messages but that's it.
And we mostly do it like this:
(let [params (s/conform ::my-spec (:params req))]
(if (s/invalid? params)
(report-error (s/explain-data ::myspec (:params req)))
(process-form-data params)))Cool thanks. I will give it a try. This is the first time I am trying to validate stuff in Clojure so still unsure what is practical and what is not.
What about maybe a separate s/closed? function that just verifies whether the data structure contains only the keys in the spec? It doesn’t seem like a big burden to do
(and (s/closed? ::spec x) (s/valid? ::spec x))Did a demo in Clojure/North about closed specs, with spec.alpha, spec-tools, spell-spec and expound, using the new reitit error formatter: https://vimeo.com/331602826. The code calls spec-tools.spell/closed-spec function on a spec and gets a closed (spell-)spec back.