Fork me on GitHub
#clojure-spec
<
2017-08-31
>
tjscollins02:08:25

I think I'm slowly losing my mind trying to get my function specs to work right.

(s/fdef example-fn
    :args string?
    :ret string?)

(defn example-fn [s] s)
This does not work if I try stest/check example-fn. Why? I'm clearly missing something. I've been re-reading the guide for two days, but something's not clicking, and I can't see where why s/fdef's are going wrong.

seancorfield02:08:09

@tjscollins :args is always a sequence of argument specs so you want (s/cat :s string?) there.

tjscollins02:08:42

How does s/cat work? Do the keys have to match the variable names, or are they just used as informational labels in the output of s/explain?

seancorfield02:08:59

Have a read of this https://clojure.org/guides/spec#_spec_ing_functions -- it has some examples that should help.

tjscollins02:08:49

Yeah, I've read through that repeatedly. It's just not making sense to me how s/cat and s/alt work.

seancorfield02:08:56

The keywords in s/cat don't have to match the argument names but it's common convention that they do (otherwise your error messages will be less than helpful).

tjscollins02:08:51

So what is actually happening when the args are checked against s/cat? Do the args pass if they match any spec that's "s/cat"ed together? Match every spec?

seancorfield02:08:18

s/cat is a "sequence regex" so it matches the sequence of arguments passed into the function

tjscollins02:08:19

The guide calls it a regex op, but I don't know what that means in this context

tjscollins02:08:43

So the first pair passed to s/cat has to match the first arg?

tjscollins02:08:51

And the second pair matches the second arg?

seancorfield02:08:17

Sort of... consider

(s/fdef clojure.core/declare
    :args (s/cat :names (s/* simple-symbol?))
    :ret any?)

seancorfield02:08:51

That matches zero or more simple symbols as an argument sequence and labels that sequence as :names

seancorfield02:08:32

So in (declare foo bar quux), (s/* simple-symbol?) will match the three symbols and :names will be (foo bar quux)

seancorfield02:08:59

It's quite literally "regex" for sequences (instead of strings).

tjscollins02:08:33

So something like (s/cat :s string? :i int?) would match both ("1" 1) and (1 "1")?

seancorfield02:08:44

No, it's sequential.

seancorfield02:08:59

It would only match "1" 1

seancorfield02:08:27

How familiar are you with regex for strings?

tjscollins02:08:37

Alright, would (s/cat :s (s/* string?) :i int?) match both ("1" 1) and ("1" "1" 1)?

seancorfield02:08:48

Yes, I believe it should

tjscollins02:08:56

Yeah, I'm familiar with string regexes

tjscollins02:08:05

Okay, that makes more sense now

seancorfield02:08:04

You can try it out directly in the REPL, without worrying about arguments: (s/conform (s/cat :s (s/* string?) :i int?) ["1" "1" 1])

seancorfield02:08:49

and (s/conform (s/cat :s (s/* string?) :i int?) [1]) => {:i 1} because there were no strings at the beginning

tjscollins02:08:27

Alright that makes a lot more sense now. Is s/alt basically s/or for a sequence?

tjscollins02:08:26

That helps a lot. Thanks.

seancorfield02:08:41

Also from that guide:

(s/cat :prop string? :val  (s/alt :s string? :b boolean?))

tjscollins02:08:31

First element has to be a string, second can be boolean or string, right?

favila13:08:21

I'm trying to spec a sequence where there is a fixed predicate/spec based on position, but the sequence may omit things at the end; and I'd like it to conform to a flat map, one key per position, nil or no-key for the omitted items

favila13:08:25

(s/conform ??? [1 ""]) => {:pos1 1 :pos2 ""}

favila13:08:45

(s/conform ??? [1 "" :kw]) => {:pos1 1 :pos2 "" :pos3 :kw}

favila13:08:11

(s/conform ??? [1 :kw]) => ::s/invalid

favila13:08:50

I can't use cat plus nested s/? because that backtracks (last example would be accepted)

favila13:08:07

the best I can think of is a linked-list type structure of nested s/or that a conformer flattens

favila13:08:19

any other ideas?

favila13:08:04

s/tuple won't work because variable-width

Alex Miller (Clojure team)13:08:18

I’m saying you could s/or multiple s/tuple together

Alex Miller (Clojure team)13:08:27

it’s unclear how much variability there is here

Alex Miller (Clojure team)14:08:02

I guess maybe s/alt of s/cat would be better for your conformed value

Alex Miller (Clojure team)14:08:49

I think I’d say: don’t fight the conformed value. write the right spec, then transform the conformed value to what you want.

Alex Miller (Clojure team)14:08:34

always come back to: how can I state the truth about the data?

favila14:08:52

that's the part that is difficult

favila14:08:34

I essentially want s/tuple, with keyed names for each slot (not numbers), and possible truncation on the right side for optional values

favila14:08:23

for background, this is parsing an edi segment, if you are familiar with those

Alex Miller (Clojure team)14:08:39

sounds like you really want s/cat with nested s/?

Alex Miller (Clojure team)14:08:16

(s/cat :1 int? (s/? (s/cat :2 string? (s/? (s/cat :3 keyword?))))) etc

favila14:08:34

wait that's legal?

Alex Miller (Clojure team)14:08:43

the downside of that is the result will be nested maps

favila14:08:55

i thought it had to be key+spec pairs

Alex Miller (Clojure team)14:08:21

oh, I missed the keys in there (always do that)

Alex Miller (Clojure team)14:08:38

(s/cat :1 int? :more (s/? (s/cat :2 string? :more (s/? (s/cat :3 keyword?)))))

favila14:08:08

(s/conform
  (s/cat
    :a number?
    :tail/next (s/? (s/cat
                          :b string?
                          :tail/next (s/? (s/cat :c keyword?)))))
  [1 "" :k])

favila14:08:15

was the best I could come up with

favila14:08:30

and I was thinking maybe an s/& could flatten the keys out

Alex Miller (Clojure team)14:08:28

I would separate that from the spec, but yes should be able to regularize both the creation of these (with a macro) and the transformation to something you want

bbrinck14:08:01

@alexmiller When using a predicate as a spec (which can occur in, say, s/assert), the value of :clojure.spec.alpha/spec is a function. However, if a proper spec is defined, the :pred is a symbol. https://gist.github.com/bhb/de31b73133d6322158a3c0a5d47d67a2

bbrinck14:08:24

Do you happen to know why there is a difference?

bbrinck14:08:58

(My goal is to provide a consistent printer that can say “<value> did not match <predicate>” in all cases)

bbrinck14:08:37

I did not see a JIRA ticket, but would be happy to create one if it is helpful 🙂 I’m not sure if there is a technical limitation here.

bbrinck14:08:52

I suspect this is related to the fact that s/def is macro whereas explain-data is a function, so it only gets the function value

bbrinck14:08:12

i.e. the symbol is already resolved by the time explain and explain-data see it. Hm. I wonder if the right solution is to spend time making a pretty-printer for functions that works reliably across CLJ and CLJS

Alex Miller (Clojure team)14:08:14

there is a ticket and a patch for that

Alex Miller (Clojure team)14:08:25

but your reasoning is correct

bbrinck14:08:21

My ticket searching skills need work 😉

Alex Miller (Clojure team)14:08:53

but maybe it’s not covering what you’re asking

Alex Miller (Clojure team)14:08:31

you should not have any expectation about the concrete type of :clojure.spec.alpha/spec, other than that it can be used as a spec

bbrinck14:08:42

No, you’re correct, this is what I want: I am not concerned about the concrete type, I just want a way to consistently pretty print the problem.

Alex Miller (Clojure team)14:08:03

that ticket fixes the :pred in the explain-data

Alex Miller (Clojure team)14:08:10

by unmunging the function name

bbrinck14:08:19

Although in my case, I’d probably print the set differently (instead of a set literal, I’d enumerate options), but that should not be a problem

Alex Miller (Clojure team)14:08:27

I’m hoping it will be in the next spec release

bbrinck14:08:01

Does it apply the same unmunge to :clojure.spec.alpha/spec at the top level of the explain-data?

bbrinck14:08:33

IOW, would it work for (s/explain odd? 10)?

Alex Miller (Clojure team)14:08:55

user=> (s/explain odd? 10)
val: 10 fails predicate: odd?

Alex Miller (Clojure team)14:08:12

(running with the patch)

bbrinck14:08:51

Much appreciated! I will definitely be using this in expound

Alex Miller (Clojure team)14:08:34

I have been too busy to look at expound, but hope to soon

bbrinck15:08:13

Don’t look too closely: the internals are a bit of a mess, but I hope to refactor soonish 🙂 . Most importantly, I want to cleanly separate the pretty printing part (hopefully useful to end users, but not really reusable) from the stuff I did related to adding information to the “in” paths, which could be useful for any library that wants to consume explain-data

didibus16:08:02

Can I trust describe to be consistent on specs of the same kind. Like will all s/keys describe itself in a consistent way?

didibus16:08:39

I'm going to build reflective logic on spec descriptions, but I want to be sure its reliable.

hmaurer16:08:56

Hi! Is there a built-in function to conform a spec and throw an error if the spec does not conform?

hmaurer16:08:29

My use-case is basically this: I want to pipe data through a series of steps, and I would like to throw an error if a spec does not conform

hmaurer16:08:35

I could easily write a function to do this

bbrinck16:08:38

@hmaurer Would s/assert work?

hmaurer16:08:41

just wondering if it’s in clojure.spec

bbrinck16:08:04

followed by conform?

hmaurer16:08:36

actually s/assert does exactly what I want; I don’t care too much about conforming

hmaurer16:08:39

thank you!

bbrinck16:08:02

@hmaurer Just make sure to call (s/check-asserts true) to turn on assert checking for spec

hmaurer16:08:07

@bbrinck ok. Is it sensible to use s/assert when validating user input, e.g. an API payload?

bbrinck16:08:51

@hmaurer I would prefer to just use explain-str in that case personally, since a) assert will raise an exception that you’ll need to catch and b) turning on asserts will enable asserts throughout your program, which you probably don’t want. I’d only use assert for dev-time checks/invariants

bbrinck16:08:57

I’m assuming the flow is check data against spec if it works, proceed else, show message to user?

hmaurer16:08:05

pretty much

hmaurer16:08:37

I would like to have some flow like this: check data again spec => validate through authorization filters => assign UUID => transact to database

bbrinck16:08:39

If your user is a dev at dev-time, I think assert is fine. But if you want to show end-user error messages, I’d avoid it

hmaurer16:08:51

any step can throw an error which should be reported back to the user

hmaurer16:08:57

end-user, not dev

bbrinck16:08:29

Right, in that case, I’d use explain-str and check to see if it success.

bbrinck16:08:48

Then return that string to whatever is going to show it to a user

hmaurer16:08:50

I am quite new to Clojure; would you in general avoid to throw errors and return error conditions instead? Aka use a “Either”-like type, similar to Haskell/etc?

hmaurer16:08:10

e.g. return {:ok ...} or {:fail ...}

bbrinck16:08:31

Good question. I don’t see a lot of use of “either” in Clojure code. Exceptions are fine if it’s actually an exceptional case , but AIUI, in your case, you want to validate user input so you expect things to not conform

bbrinck16:08:35

(correct me if I’m wrong)

hmaurer16:08:16

You are right. I would like to be able to chain my pipeline steps with (-> ...), but breaking out of the pipeline on error seems hard to do without exceptions

bbrinck16:08:17

I don’t have great advice here for general error patterns, but I’d lean towards “use exceptions for exceptional cases, not control flow”

bbrinck16:08:43

There are libraries for this, but I don’t know how idiomatic they are. I haven’t used them myself

didibus16:08:19

It's more idiomatic to use exceptions with ex-info. I would wrap explain-str or explain-data in a function which throws an ex-info, where I handle it at the place where my code can return an error msg to the user. There I'd parse out what I need from the exception into the msg I want to show the user and return it.

hmaurer16:08:30

Thanks! I’ll read this. I like the philosophy of using errors for exceptional cases only though; considered following that approach in a previous javascript projct

hmaurer17:08:02

@didibus yep, that sounds a bit more idiomatic/practical

didibus17:08:43

What you do is in the ex-info map, you put a key such as :type :validation-error and so where you catch it, you know this isn't a bug, but a validation error. Normally, the exceptions should not log error or push failure metric, as they are actually expected behaviour

bbrinck17:08:54

Even if you use exceptions in this case (not necessarily bad IMO), I’d still probably throw an exception manually instead of relying on assert since turning on/off asserts is a global thing. IOW, do you want to enable all asserts in production?

hmaurer18:08:03

That seems reasonable

bbrinck17:08:15

That’s a valid question, but you probably don’t want it tied to your user-facing errors messages

didibus17:08:18

I mean exceptions of type validation-error

didibus17:08:49

Agree with bbrinck

bbrinck17:08:51

Good suggestion @didibus

bbrinck17:08:54

@hmaurer Shameless plug, but if you are considering showing spec error messages to users, take a look at https://github.com/bhb/expound

hmaurer18:08:33

I stumbled on it yesterday actually! It’s looking very useful

didibus17:08:23

Big fan of expound, the default spec errors are so full of noise.

didibus17:08:16

I'm actually disappointed in spec for improving clojure error msgs. They promote the idea, but it really doesn't help because the error is so long. Its only slightly friendlier then a stack trace

bbrinck17:08:35

IMO, the default error messages are a good default, since they need to be useful “out of the box” in a variety of contexts: shown to a end user, shown to a dev, logged, returned as an API response, etc. Expound makes a different set of tradeoffs that are very much optimized for the “display to dev” case - *much* more verbose, and omits some information, but hopefully helps devs find the root problem faster

bbrinck17:08:22

Luckily we can swap out the default for custom printers like expound when we want to customize behavior 😉

bbrinck17:08:04

Expound is probably still not quite optimized for end users, but I am looking to make it more suitable for this in the future.

didibus17:08:45

I don't know. Id like more beginner friendly defaults in Clojure. Easier to swap out to something else when you're not a beginner anymore. Its a lot to ask to a beginner to learn how to setup expound to figure out why his code doesn't work.

bbrinck17:08:37

@didibus I agree with you that it’d be nice to get something easier for beginners. I have been wondering if it would be possible to create a lein plugin or library that would set up beginner-friendly REPL / logs by default

bbrinck17:08:47

Adding a single dependency that loads core specs, turns on instrumentation (with orchestra?) and configures spec to use expound without any further config could be really useful

bbrinck17:08:18

maybe pretties up/shortens stack traces as well

didibus17:08:37

That would be really nice. Maybe if it can also add a coloured repl and preload clojure.repl

didibus17:08:56

But since at my work we use Ant, it doesn't help me promote Clojure to other teams

bbrinck17:08:03

What editor would you recommend for other teams? Just curious about the constraints.

didibus17:08:29

Well, I tend to recommend eclipse + counterclockwise or cursive with intelliJ.

didibus17:08:13

The latter less so, because of its paid nature

didibus17:08:10

You'd just be surprised how many top class engineers don't bother to learn how to setup any environment properly.

didibus17:08:24

So when I tell them, if you want to do Clojure and like it, you need to learn at least how to setup a working repl. And already I've lost the interest of half of them

gdeer8118:08:56

@didibus I think @sekao is the best person to talk to about this since he has written three IDEs so far and each one tries to lower the burden of getting everything set up

gdeer8118:08:56

this is a side topic not really related to spec so I broke it out into this thread

didibus20:08:34

Nightlight looks pretty cool

gdeer8118:08:04

my favorite is nightmod since I can send someone the download link and they can be in a new project with a repl, starter code, and a live game environment in less than 5 minutes

favila18:08:33

In clojure.spec.alpha, s/invalid? is defined like (identical? ::invalid ret). I thought we couldn't rely on keyword interning anymore?

ghadi18:08:07

it still is a thing

ghadi18:08:12

(not in cljs afaik)

favila18:08:47

I thought at some point even in clj it was no longer guaranteed

favila18:08:05

so (identical? (sym|kw) x) became a bad idiom

ghadi18:08:12

symbols are not interned

ghadi18:08:14

just keywords

favila18:08:27

ah maybe that's what I'm thinking of

favila18:08:32

or I'm just confused with cljs

favila18:08:37

(I do a lot of cljs)

ghadi18:08:39

though the codepath goes still through a method called intern

ikitommi19:08:03

hi. I have a problem creating a recursive spec. the s/spec requires the spec to exist:

(require '[clojure.spec.alpha :as s])

(def data
  ["/api"
   ["/ping" {}]
   ["/pong" {}]])

(s/def ::route
  (s/cat
    :path string?
    :arg (s/? (s/map-of keyword? any?))
    :childs (s/* (s/spec ::route))))
; CompilerException java.lang.Exception: Unable to resolve spec: :user/route

ikitommi19:08:10

Setting something temporary as a spec and double-evaluating the original spec seems to work thou, but don’t think this is the way to do this 😉

ikitommi19:08:30

;; declare'ish
(s/def ::route int?)

(s/valid? ::route data)
; false

;; resetting the spec works now (but the childs is wrong)
(s/def ::route
  (s/cat
    :path string?
    :arg (s/? (s/map-of keyword? any?))
    :childs (s/* (s/spec ::route))))

(s/valid? ::route data)
; false

;; now it might work?
(s/def ::route
  (s/cat
    :path string?
    :arg (s/? (s/map-of keyword? any?))
    :childs (s/* (s/spec ::route))))

;; ....
(s/valid? ::route data)
; true

favila19:08:49

@alexmiller My attempt to spec the s/cat s/tuple hybrid with optional trailing positions https://gist.github.com/favila/67c869b3c92b1a40559f639b065734e2

favila20:08:58

works for me (so far)

ikitommi20:08:29

Oh, my recursive spec works when I wrap the child spec into something, like s/and

(s/def ::route
  (s/cat
    :path string?
    :arg (s/? (s/map-of keyword? any?))
    :childs (s/* (s/spec (s/and ::route)))))

(s/valid?
  ::route
  ["/api"
   ["/ping" {}]
   ["/pong" {}]])
; true
Is this the correct way to do this?

hmaurer21:08:43

Hi! Quick question: I have a spec for a map which has required and optional keys (`(s/keys :req [..] :opt [..])`). One of my functions takes “partial” versions of those maps, which essentially means that every key is optional (and every key that was optional in the full spec can be nil, but I can handle that separatly)

hmaurer21:08:24

it there a neat way to do this, how should I define two versions of the map specs? (e.g. :foo/my-map and :foo.partial/my-map)

plins21:08:48

hello everyone, im reading the spec docs (`https://clojure.org/guides/spec`) is it possible to achieve with spec/fdef something similar to this? (catching spec errors at run time):

(defn person-name
  [person]
  {:pre [(s/valid? ::person person)]
   :post [(s/valid? string? %)]}
  (str (::first-name person) " " (::last-name person)))

didibus04:09:42

You can instrument, but it doesn't do the post

didibus04:09:26

You can use orchestra a 3rd party library, that one will instrument the post also

plins14:09:10

any recommendations on libraries that already do that?

wilkerlucio22:08:39

hello, I'm having an issue trying to define the following specs:

wilkerlucio22:08:42

(s/def :user/accessors (s/keys :opt [:user/id :user/name :user/address]))
(s/def :user/id string?)
(s/def :user/name string?)
(s/def :user/address :address/accessors) ; <- trouble maker

(s/def :address/accessors (s/keys :opt [:address/street :address/user]))
(s/def :address/street string?)
(s/def :address/user :user/accessors)

wilkerlucio22:08:18

this is a simplified case where I need to set a spec to something that still are going to be defined

wilkerlucio22:08:31

right now this throws an error, as:

wilkerlucio22:08:36

CompilerException java.lang.Exception: Unable to resolve spec: :address/accessors, compiling:(2017.clj:10:1)

wilkerlucio22:08:26

I remember reading that the definitions should be lazy to allow this kind of things, am I missing something here?

didibus04:09:29

I always known it as not being lazy. Can't you just reorder your definitions? Being lazy would make it risky I feel, what if specs are accross namespaced and lazy, and you forgot to load another namespace.

wilkerlucio22:08:14

@hmaurer you can define multiple versions, create a simpler one and then you can merge it with other to create a more fully-featured, example:

(s/def :foo.partial/my-map (s/keys :req [:foo/a :foo/b]))
(s/def :foo/my-map (s/merge :foo.partial/my-map (s/keys :req [:foo/c])))