Fork me on GitHub
#clojure-spec
<
2019-03-04
>
jsa-aerial16:03:09

Is there a preferred idiom for the use of 'world information' in predicates and specs? This can occur in data validation. To clarify some, you have a data value, which you can check the 'shape/type' of via predicate and spec involving that value. Once that checks OK, a further validation requirement would be to see if what the value represents exists (in a DB or directory or ...) and is 'properly configured'. Those checks require looking outside the value so a predicate doing these checks needs more than the value. Maybe that is access to an in memory cache or a DB key or whatever. I can think of several ways to do this, but they all feel a bit off. So, is there an idom / preferred way to have that information available in predicates and specs?

borkdude16:03:21

@jsa-aerial My gut feeling tells me that it’s not a great idea to do side effects in specs

Alex Miller (Clojure team)16:03:58

specs are not a great match for these kinds of validations

Alex Miller (Clojure team)16:03:36

in general, I've seen better success with dynamically generated static specs (like running some code to register a spec with a set of allowed values pulled from a db) than with dynamic specs (which somehow close over state required to dynamically check validity in some way)

jsa-aerial16:03:18

I see - I can understand that. Basically this is 'out of scope'.

Alex Miller (Clojure team)16:03:09

might be easier to implement as an explicit validation implemented outside of spec

jsa-aerial16:03:03

Yeah, that would be one of the ways I've considered

jsa-aerial16:03:35

I will say there isn't any side effecting here, but it does use / need 'outside/ world' access

jsa-aerial16:03:03

Yeah, that would be one of the ways I've considered

borkdude16:03:38

maybe something like this would work though:

(defn foo [conn]
  (let [vals (query conn)]
       spec (my-custom-spec vals)]
  (s/assert spec ...))
This keeps the spec entirely pure

borkdude16:03:24

depending on how you use spec -- this won’t work for e.g. fdefs

jsa-aerial17:03:53

Some other ways: augment the value first with the 'key' for outside data so that the 'value' is wrapped and contains the necessary information. That's not too bad. If you have a map, augment with the 'outside key' and explicitly check the map contents (as opposed to using spec'd keys). Again, not too bad but kind of not using spec idiomatically...

jsa-aerial17:03:24

Rather worse: use a dynamic var to hold the outside 'key'... This actually feels a bit dirty.

seancorfield20:03:52

It's also worth mentioning that, strictly speaking, reading values from a DB is side-effecting because if you repeat the call, you will not necessarily get the same result. It's not mutating anything directly, but it is not a pure operation either. I think a lot of people tend to think that SELECT * FROM whatever is "readonly" and therefore not subject to side-effects...

jsa-aerial22:03:12

If nothing changes (anywhere), I don't see how it is 'side effecting' to read. The only way that would make sense is if the database were to change out from underneath you. In general that could happen, but in this case nothing of the sort is happening. Even more to the point, if it is already read and cached, nothing associated with the operations at hand is effecting anything anywhere.

seancorfield23:03:25

That's why I said "strictly speaking" -- in the general case, where a database is a mutable thing that other processes or other threads could be updating, when you do a live read from the DB, you may get different results from different calls over time. Thus, the function doing the reading behaves as if it has side-effects because it is not pure. I was specifically excluding the case where the data is read once and cached.

seancorfield20:03:25

(unless you're using Datomic where you can get the entire DB as an immutable value for whatever scope of request processing you want)

ag21:03:35

how do I write fdef args for a function that takes: [[a b] _]?

Alex Miller (Clojure team)21:03:32

there are several ways to potentially spec "[a b]" - what have you tried?

Alex Miller (Clojure team)21:03:58

could be an s/tuple, s/coll-of with :count 2, a nested s/cat

ag21:03:08

(s/cat (s/cat :a int? :b int? ) :_ any?)

Alex Miller (Clojure team)21:03:21

nested regexes describe the same collection

Alex Miller (Clojure team)21:03:33

wrap s/spec around the inner s/cat

Alex Miller (Clojure team)21:03:46

to describe a nested regex collection inside the outer regex collection

Alex Miller (Clojure team)21:03:01

(s/cat (s/spec (s/cat ...)))

ag21:03:02

not working 😞

borkdude21:03:12

@ag

user=> (s/valid? (s/cat :v (s/spec (s/cat :a int? :b int?))) [[1 2]])
true

ag21:03:12

ah okay… now I get it!

ag21:03:31

awesome! thank you @alexmiller and @borkdude!

borkdude21:03:12

in spec2 this will be called (s/nest ..)

robertfw22:03:46

apologies if this isn't the right place to ask - I've recently started using the replacement instrument function from jeaye/orchestra. I'm getting an error while generating one of my specs, the error is

Execution error - invalid arguments to orchestra.spec.test/spec-checking-fn$conform! at (test.cljc:115).
(\8 \9 \7 \9 \2 \5 \4 \2 \0 \1 \0 \8 \1 \6 \9 \3 \1 \7) - failed: string? at: [:args :digits]

robertfw22:03:56

er, one moment while i clean that formatting

robertfw22:03:38

I'm wondering if I'm missing something as I would expect to get a more detailed/specific error about what spec failed

robertfw22:03:51

whereas the error seems to be something internal to orchestra

seancorfield22:03:58

@robertfrederickwarner That looks to me like it expected a string but was passed (seq s) -- i.e., a sequence of characters instead.

robertfw23:03:35

Yup - that's the problem within the spec. My confusion is more that the error from orchestra isn't what I'd expect to see - looking at https://github.com/jeaye/orchestra/blob/2019.02.06-1/src/clj/orchestra/spec/test.cljc @ line 115 (the line from the error above), I'd expect to see an error prefixed with "Call to..." followed by details about the spec that failed

robertfw23:03:51

The problem itself - passing a sequence of chars instead of a string - is occurring within a generator for a spec so I am wondering if that is causing problems for the usual reporting

robertfw23:03:53

I saw some discussion of orchestra here earlier so thought I would throw it out incase anyone here recognized something obvious, I'll dig around some more but may open a bug to see what @jeaye thinks

robertfw23:03:30

did some further digging and tried out the standard instrument. Orchestra was hiding the error location a little bit but otherwise returns similar to the standard error

jeaye23:03:20

@robertfrederickwarner So the issue exists in the upstream spec instrumentation as well?

robertfw23:03:57

The upstream error I got was

Execution error - invalid arguments to ws.data/luhn-check-digit at (data.clj:39).
(\8 \9 \2 \2 \5 \0 \7 \9 \2 \3 \9 \6 \4 \5 \3 \2 \6 \8) - failed: string? at: [:digits]

robertfw23:03:10

So at least that let me see the line in question (data.clj:39)

jeaye23:03:34

Ok, it's likely an issue with the way your spec is written then.

jeaye23:03:53

Will you show your code, or a minimal repro case?

robertfw23:03:29

sure, let me just strip out some specifics here