Fork me on GitHub
#clojure-spec
<
2017-12-01
>
Shantanu Kumar05:12:28

Hi, has anybody successfully used spec for web validation and translated the validation failures to HTTP 400 for nested data models? Any example or references would be great to have.

ikitommi06:12:34

Don’t know about success, but there are routing libs like compojure-api and reitit, which support spec as web (backend) validation. There is an example in https://github.com/metosin/c2. For invalid input, produces http 400 with this kind of body:

{
  "spec": "(spec-tools.core/spec {:spec (clojure.spec.alpha/keys :req-un [:c2.spec/x :c2.spec/y]), :type :map, :keys #{:y :x}, :keys/req #{:y :x}})",
  "problems": [
    {
      "path": [
        "y"
      ],
      "pred": "clojure.core/int?",
      "val": "a",
      "via": [
        "c2.spec/y"
      ],
      "in": [
        "y"
      ]
    }
  ],
  "type": "compojure.api.exception/request-validation",
  "coercion": "spec",
  "value": {
    "x": 100,
    "y": "a"
  },
  "in": [
    "request",
    "body-params"
  ]
}

Shantanu Kumar06:12:47

I was looking at https://github.com/metosin/spec-tools meanwhile and thought of pinging you!

Shantanu Kumar06:12:02

We are using a different data-driven router, so looking for a standalone validation mechanism using spec. Is there any such library?

ikitommi06:12:26

Well, reitit is kinda modular, if you take the metosin/reitit-spec is’s flattened deps tree is: spec-tools, clojure.spec & meta-merge

ikitommi07:12:32

it bring the ring-module but it has no dependencies. Could move the Coercion protocols into own module.

ikitommi07:12:35

do you think the coercion should be totally out of reitit (modules)? I would be more polite towards other routing libs at least.

Shantanu Kumar07:12:06

Thanks, let me try out Reitit-spec. I’m sort of new to spec. And yes, compat with other routing libs would be very useful.

Shantanu Kumar07:12:05

Though can’t comment now about code organization.

ikitommi07:12:56

there is already compat, but needing to include another routing lib to get coercion for another is kinda odd. Plan is to ship coercion interceptors too (as they don’t require any new deps), so they should be usable from “pedestal-style interceptor web libs”

ikitommi07:12:12

are you using mw or interceptors? and if interceptors, which kind?

Shantanu Kumar07:12:05

I am using Ring with a routing lib, but right now just using plain app code to do validation (it’ a new project) - no m/w for now.

ikitommi07:12:42

ok, I think I removed the “vanilla ring coercion middleware” in favour of the reitit “compiled” one (much faster). Could push them back to support other routing libs. This: https://github.com/metosin/reitit/commit/7979c9de9d42afd611e13da40502ac8ea1c0c0e3

Shantanu Kumar07:12:24

I think I can form an opinion about the commit after some more experience with spec.

Shantanu Kumar07:12:56

I’m looking at https://metosin.github.io/reitit/ — is there a section that describes reitit-spec in particular?

ikitommi07:12:45

I could cook up examples of that with plain ring. Most likely have time on the weekend.

Shantanu Kumar07:12:15

Thanks! Looking into it. More examples would be cool for sure.

Andreas Liljeqvist10:12:54

How would I define keys and sharing them between specs? like `(def base [::id ::something]) (s/keys :req base)`

Andreas Liljeqvist10:12:13

I suppose it fails because of compile-time evaluation

andre.stylianos10:12:56

You can define a base spec and merge that into others

Andreas Liljeqvist11:12:30

@andre.stylianos Thank you, that seems to be exactly what I am looking for

mpenet13:12:05

most of the time you use s/merge you probably want s/and instead

guy13:12:42

why do you think that is?

guy13:12:51

Just want to understand thats all

mpenet13:12:35

from what @andreas862 asked, he wants to be able to combine specs, not necessary merge them

Alex Miller (Clojure team)13:12:41

I don’t think I would agree with that

Alex Miller (Clojure team)13:12:55

This is exactly when you want to merge

mpenet13:12:04

what are the advantages ? genuinely curious. I know about gen

mpenet13:12:35

I remember also merge having arguably odd behavior with conforming

Alex Miller (Clojure team)13:12:03

in that conform doesn’t flow

Alex Miller (Clojure team)13:12:02

gen is the big one though

ikitommi13:12:14

Is there a reason that conform doesn’t flow?

Andreas Liljeqvist13:12:42

Just adding that what I wanted was to merge even if my wording was a bit unclear

ikitommi13:12:15

(s/def ::a string?)
(s/def ::b (s/and (s/conformer str/upper-case) string?))

(s/conform
  (s/merge
    (s/keys :req-un [::a])
    (s/keys :req-un [::b]))
  {:a "kikka", :b "kukka"})
; => {:a "kikka", :b "KUKKA"}

(s/conform
  (s/merge
    (s/keys :req-un [::b])
    (s/keys :req-un [::a]))
  {:a "kikka", :b "kukka"})
; => {:a "kikka", :b "kukka"}

mpenet14:12:06

another thing I prefer not to use lately: custom conformers 🙂

gfredericks14:12:21

speaking of schema-related transformations, I've been experimenting with reviving the stuff in https://github.com/gfredericks/schema-bijections as a library that only deals with bijections, not referring directly to schema/spec at all I'm curious how useful something like that would be for transforming data to the format that a spec expects (and back? e.g. jdbc)

gfredericks14:12:04

I think generating bijections from specs would be the ideal result

mpenet14:12:11

that'd be useful

gfredericks14:12:30

I still have to decide if/how to incorporate surjections though

mpenet14:12:34

I tend to write this kind of stuff with specter these days, but that's quite gory

gfredericks14:12:37

since those are also useful in some cases

mpenet14:12:09

tbh I didn't really like the form it took in schema, but that was practical. I am not sure I have seen a solution I like for these kind of transformations yet

gfredericks14:12:36

the form what took in schema?

gfredericks14:12:56

oh right -- I don't like them because they represent surjections everywhere, even for uses when bijections can work and it makes it easy to test your function-that-coerces-its-inputs with the canonical form of the data, since that's easier, and never actually test the production form that has to be coerced

danielneal14:12:25

bijections from specs would be awesome

gfredericks14:12:16

I am envisioning that you can define different styles for other representations, like json and jdbc, and then get bidirectional functions for converting your canonical spec form from one to the other

gfredericks14:12:35

e.g., "I want camel-cased keys, I want date-times serialized like this" etc.

gfredericks14:12:06

that category of things, but hopefully arbitrarily flexible

gfredericks14:12:59

so if you say your json representation has camelCased keys while your internal representation is kebab-cased, then a json input with kebab-cased keys would be considered incorrect and cause an exception rather than just silently be accepted

mpenet14:12:37

There was a hint of a next iteration of spec coming out at some point (I think in Rich's talk), I am wondering if there are changes related to these kind of things (and others often mentioned).

mpenet14:12:23

I personally am curious to see how spec forms conforming (or whatever replaces it) will take form

mpenet14:12:39

related to CLJ-2112

gfredericks14:12:51

oh yeah, I forgot about that

mpenet14:12:56

and specs metadata too, could help for a lot of things

gfredericks15:12:57

you mean enabling specs to have metadata attached?

gfredericks15:12:14

I assume they already can so you must mean something else

mpenet15:12:16

you can't attach metadata to specs atm

ikitommi17:12:38

@alexmiller related to the s/merge problem, there is a PR request in spec-tools of a merge that seems to fix the problem. Could that be pushed to Spec itself? Comments welcome: https://github.com/metosin/spec-tools/pull/91

ikitommi17:12:49

My comment on 15:53, that can’t be right?

ikitommi17:12:57

e.g. merge merges that values, so the unconformed value override the conformed ones.

Alex Miller (Clojure team)17:12:34

it doesn’t override, it’s just that only the last spec in the merge controls the conformed result

Alex Miller (Clojure team)17:12:32

that is the intent, not a bug, but it can have some non-obvious effects, particularly with unqualified keys

Alex Miller (Clojure team)17:12:14

I think there is a ticket for this already in jira, but not sure what should be done with it

ikitommi17:12:17

oh, so that works with qualified keys. That’s interesting.

ikitommi17:12:28

Woudn’t @arttuka’s PR fix that? Could the issue be re-opened?

Alex Miller (Clojure team)17:12:43

I wouldn’t re-open that, but if there is a good statement of a problem and a patch, we can take a look at it

Alex Miller (Clojure team)17:12:44

that is, in a new ticket

Alex Miller (Clojure team)17:12:13

I have no idea what that PR is doing at a glance

Alex Miller (Clojure team)17:12:42

Rich has a pretty extensive re-work coming for spec (post 1.9 release) and I suspect we won’t really look at any fixes till we’re on the other side of that

danielneal16:12:09

Ah interesting! What kind of rework? Is it on the implementation side or is there going to be some kind of change to the interface?

ikitommi17:12:10

ok, I’ll ask Arttu to post a new issue out of that. And looking forward to the re-work, whatever that is. thanks!