so when we’re not using generative tests, what do people use for utilizing specs from tests?

i find s/assert annoying b/c of the check-asserts compile time flag - need to forcibly reload everything that might have asserts when working interactively

instrument is nicely dynamic, but i understand why the asserts are static

i still don’t really understand why :ret and :fn are not checked when instrumented

The guide now says "Note that the :ret and :fn specs are not checked with instrumentation as validating the implementation should occur at testing time.” - but instrument are in the stest namespace, soo presumably you only instrument at test time anyway?

I get the desire to not instrument ret and fn for other peoples code in some cases, but honestly, i also want to validate that the libraries i’m calling respect their specs

i don’t trust them

i’m just going to call s/assert* for manual (non-generative) tests and deal with it if it breaks at some point 😉

Hello, I've just started using Spec! I'm trying to use it in a particular situation that I can't see covered in the documentation but I may be wrong. I have two datastructures:

(def created
 {:event-type "TxnCreated"
  :data {:id "foo"
         :amount 6}})

(def deleted
 {:event-type "TxnDeleted"
  :data {:id "foo"
         :reason "bar"}})

I'd like to have a different spec for each of the different :data items, but from what I can see s/keys will use one spec for one key...

So that a deleted item would have different required keys to a created item. Is the problem that my keys are not namespaced? Or is this going against: > "the map spec never specifies the value spec for the attributes, only what attributes are required or optional"

Thank you for any help

Yeah, this is exactly the use case for namespaces. You want to re-use the same keyword to mean 2 different things, put them in different namespaces and you’ll be fine

@manutter51 the only problem I have with this is that I currently have functions that accept either of these structures, and rely on the presence of the data key.

Hi, just as an idea/question I had... is it possible the do clever things with date/times in spec. I assume you can say that a certain value should be an date/time instance... but wouldn't it be nice if you could express in a spec that a certain date/time should come before/after a different date/time ?

for instance {:start #inst 2017-01-01 :end #inst 2017-01-5} and if the end is before the start it fails to validate?

I guess you can do that with s/and

(s/and (s/keys ....) #(start<end? %)) where the latter function checks the values

ok cool @mpenet that looks like a good solution.

(s/def ::fancy (s/and (s/keys :req-un [::start ::end]) (fn [{:keys [start end]}] (> end start))))
:user/fancy
user> (s/def ::start int?)
:user/start
user> (s/def ::end int?)
:user/end
user> (s/valid? ::fancy {:start 0 :end 1})
true
user> (s/valid? ::fancy {:start 0 :end -1})
false
user> (s/valid? ::fancy {:start 1 :end 0})
false

aah cool... I hadn't realised you can attach the function to the spec. that is very handy indeed

specs are just predicates really

well the building blocks at least

http://blog.cognitect.com/blog/2017/1/3/spec-destructuring

@alexmiller your article is related to this issue right? http://dev.clojure.org/jira/browse/CLJ-2003

in what way?

that is, no not in any way obvious to me :)

One reproduction of the issue is shown by @bbloom

(s/unform :clojure.core.specs/defn-args (s/conform :clojure.core.specs/defn-args '(f [& xs])))
;;; (f ((& xs)))

oh, just another case. the bug is in spec itself, not in what’s in the post.

I never meant that there was a bug in the post

😊

well, not sure why you’re asking then

I’m asking because after reading your post, people might try to do a “conform unform” loop and they might be surprised by the result...

I’ve been using collection-check https://github.com/ztellman/collection-check a while ago — is something like that becoming easier with spec? Maybe there are generators etc for ·built-in data structures?

@dacopare Given your data, you can still use unqualified keys in your existing data. The spec keys must be qualified, but not the keys in your map. They would look something like this:

(s/def :generic/event-type string?)
(s/def :generic/id string?)
(s/def :created/amount pos-int?)
(s/def :deleted/reason string?)

(s/def :created/data (s/keys :req-un [:generic/id :created/amount]))
(s/def :deleted/data (s/keys :req-un [:generic/id :deleted/reason]))

(s/def :created/event (s/keys :req-un [:generic/event-type :created/data]))
(s/def :deleted/event (s/keys :req-un [:generic/event-type :deleted/data]))

(s/conform :created/event created)
(s/conform :deleted/event deleted)

@martinklepsch I’m not sure spec adds much over collection-check, which has a pretty specific purpose. spec does allow you to gen from collection specs, but not sure if that’s easier. depends what you’re trying to do.

@alexmiller in the blogpost the result for the 3rd "WORK IN PROGRESS" block should be: (s/conform ::seq-binding-form '[a b & r :as s]) {:elems [a b], :rest {:amp &, :form r}, :as {:as :as, :sym s}}

ah, thx

@alexmiller I basically just want to check if a thing behaves like a set, a vector, a list. Guess collection-check it still is then 🙂 Thought maybe through specs for core it would have become easier to generate operations

yeah, nothing at the level of collection-check

one thing i learned from your blog post was that simple-symbol? existed 🙂 i had defined my own unqualified-symbol?

500 fake internet points to the person that spots the problem (took me all morning to debug this and thought it'd be more fun to make it a challenge )

(spec/fdef run-thread!
           :args (spec/cat :fn (spec/fspec :args empty? :ret nil?)
                           :name string?)
           :ret #(instance? Thread %))

(defn run-thread!
  [f n]
  (let [t (Thread. ^Runnable f ^String n)]
    (.start t)
    t))

(defn my-runnable
  []
  (try
    (while true
      (Thread/sleep 1000)
      (println "Hello, World!"))
    (catch Exception e
      (println "Got an exception!" (.getMessage e) "Exiting..."))))

(clojure.spec.test/instrument)

(def t (run-thread! my-runnable "my-thread"))

name is optional in your fdef but not in your function?

Ah, good catch. That's an artifact of copy/paste (my original run-thread! had 2 arities where name was optional). But, that's not the issue that got me...

fixed it 🙂

I think that's worth 50 fake internet points, though 😉

hint: it is related to spec...

other than that seems to work for me. except of course there's not really any reason why either sleep or println would throw an exception

well, the catch exception is for killing the thread with (.interrupt t)

try pasting the function definitions in the repl first and then hit enter. after that paste the instrument and def separately...

(just to make sure you are instrumenting on the newly added run-thread! spec)

doesn't that need to be `run-thread! for that?

ah, nope

iunno, nothing particularly interesting is happening for me

Some more detail: What this does is runs the my-runnable in the current REPL thread and never returns from run-thread! when I would expect the run-thread! to return immediately with the Thread object and the printlns happen in the background.

Reason: spec/instrument attempts to validate the my-runnable lambda per my fspec, before calling the function, which starts a continuous loop and the run-thread! fn is never called!

Lesson learned: don't instrument a "runnable" fn that doesn't terminate...

ah. I see. yes don't instrument functions with side-effects

Yep. Makes perfect sense now, but took me a while to figure it out 🙂

yeah I totally forgot, but it has been mentioned before

Thought I could save someone a potential headache by mentioning here...

there was talk of more coming for dealing with spec+side-effects maybe, but I'm not sure if it still is

Yeah, it's one of those things you know in the back of your mind, but forget when in the thick of debugging a weird error

i’m not sure i understand what you say is happening here...

i did this:

(s/fdef f :args (s/and #(do (.printStackTrace (Exception.)) %) any?))

instrument

then call f or (.start (Thread. f))

from what i can tell, the spec-checking-fn is called by Thread.run

which seems right to me… sooo what’s the problem?

what are you trying to accomplish?

When I do this I get a stack trace, is this not what you want?

I was trying to understand @manderson and bfabry’s discussion above

@bbloom problem is with the fspec. It's not intended for fn's with side effects, which a while true looping runnable most certainly is. When instrument is turned on, it attempts to validate the fspec by testing it out with different data to ensure it conforms. This ends up running the loop in the current thread and never actually executing the originally intended function.

Threw me for a while, because it looked like it was running as intended, but was running in the current thread.

As part of instrumentation

huh? fspec should work just fine for functions with side effects outside of generative testing

what “different data” is it using?

instrumented functions are only called once - with the same args they are given, and presumably on the same thread they would normally have been called on

@bbloom instrumenting uses random data to validate functions passed as an arg to the instrumented fn

wait … what?

e.g., if you spec clojure.core/map

whose first arg is a function

then you instrument map

then you call it with some function

that function will be test.checked as part of the arg-validation step

whoaaaa OK now it finally makes some sense why :fn and :ret aren’t checked by instrument

yeah? I haven't connected these two things

i was under the impression the purpose of instrument was to make sure i wasn't calling other people’s stuff wrong

and that it was also used during generative testing to validate my stuff

i had no idea generative data was happening if i did instrument normally

if somebody wrote a clickbaity listicle of facts about clojure.spec this would be the "number seven will shock you"

it's not generatively tested the instrumented function

it's generatively testing fspec args to the instrumented function; so for non-HOFs this doesn't apply at all

“number three: ALL spec’d keys in a map are validated, not only those explicitly stated as :req‘d …” 🙂

yep, i discovered this ^^^ when i had some println's for debugging and saw a bunch junk printed out i wasn't expecting. just didn't connect it with my issue above, but it hits home the point as to why fspec isn't for side effects

the generative testing of fspec, that is

i’m pretty confused now.

is that the advice? don't use fspec at all for side-effecting functions?

ignore what i said - i still don’t understand what is happening here

if i use stest/instrument

and then call some instrumented functions from the repl

are generators involved at all?

my impression was no

possibly

… when? why?

think about the spec for clojure.core/map

(s/cat :f (s/fspec :args (s/cat :x any?) :ret any?) (s/coll-of any?))

something like that

imagining it only took 2 args

if you instrument map, that means you're asking for its args to be validated

so you instrument and then from the repl you call (map inc [1 2 3])

so you must be expecting spec to validate that [1 2 3] matches (s/coll-of any?), but what do you expect it to do with inc?

it could merely check that inc is an IFn

holy hell - that’s what i would have expected: to just check ifn

i expected only instrumentation of var calls

number seven will shock you

i would not expect instrumentation of higher order functions - that’s crazy

clojure is just not prepared for that

heh

in the absence of chaperones, proxies, and all that other crazy platform stuff that racket contracts have

in the presence of pervasive effects

that’s just nuts

nnnow i understand - here i am trying to recreate the problem discussed above with thread/start etc, but it was only a problem with higher order functions in there

well, that’s what fspec is mostly concerned with — hof

As example:

(spec/fdef test-fn 
           :args (spec/cat :str string? 
                           :fn (spec/fspec :args (spec/cat :x string?) :ret nil?)) :ret nil?)

(clojure.spec.test/instrument)

(test-fn "hi" #(println %))

U
O

6K1Y
sqD
4PsKAL4
1wbc5
1kq9bvc
wqgAN0gb
kdKAI1aE24
22r
2
6TUvuD2x8c
95NKQmTX6c7kdTO
98O0dGrT9vr65
PE5712237F7
6
Rwc93xcj510gn7
MOvNSAJl
Received x: hi
hi
=> nil

So, if #(println %) was instead a fn with side effects (looping, hitting a db, etc), then you can see the problem...

Interesting. So, long story short: HOFs are "instrumented" via generative testing?

yeah

seems really weird to me

I'd never considered it directly. I think deep down I was thinking they would be proxied.

so, in my case, i was seeing my runnable fn executing, and it looked like it was functioning as normal, except it was running in the current thread instead of in a new thread.

yeah proxied makes more sense

But I understand the argument for non-proxied verification.

ie you check it’s ifn? or fn? and then wrap it with clojure.spec.test/spec-checking-fn (private function that does the proxying)

i don't

this makes instrument much less useful for repl use

yep, i went back and changed some of my specs to #(instance? IFn %)

rich talked about the proxying approach in a ML thread somewhere

TBH, I'm not sure what utility there is in specing a side-effecting function

before i look for that thread - the reason i wouldn’t expect proxying is b/c it breaks equality potentially

I think he thought it was difficult to design well because of having to track the proxy as it flies around the program

Yeah that's^ what I was thinking.

once you use proxying, the validation of your function can happen anywhere at any time, not just at the moment of calling it

"We do verification here and only here."

e.g., with map, it would immediately return a lazy seq

and then blow up later when somebody consumed it? or not blow up because the function call already ended?

Creates separation in execution.

if you unstrument before consuming the lazy seq what should it do?

this is all why my assumption was that instrument only instrumented vars

right

ie it was a better than nothing interactive tool

cover the 90% case, rather than attempt the 100% case with fallout such as “can’t use instrument if you fspec any side effecting functinos"

Yeah the fact that it does that by default and "transparently" is... unexpected.

instrument does only instrument vars

well, i mean only calls to vars and not use the generators

but, instrumentation means validation of arguments to a function, if one of your argument is an fspec, validation necessarily means generative testing

found http://dev.clojure.org/jira/browse/CLJ-1936

code is data, data is code bruh

so, don't write an fspec for functions that could potentially be side-effecting 🙂

@bbloom I think I'm remembering a long thread on the clojure ml (not -dev) from 3-9 months ago

I suppose the other thing you could do is stub out specs for functions that are potentially side-effecting

That seems to be the intended path^

hopefully the next screencast/blog on spec is advice on impure functions 🙂

Yeah.... it's still interesting to me that they would commit to verifying HOF args. I think see the argument (pun definitely intended), but "if 'x is on classpath' do 'y' else 'z'" seems complex.

I mean, I think it brings back up the question: "What's the relative utility of specing side-effecting fns?"

If the answer is "so low it's not worth specing them," then committing to verifying HOFs makes a lot more sense to me.

it’s only low value if you view spec-ing as a testing mechanism and not as developer guard rails

Yeah perhaps. OTOH, if you view side-effects as poison, no guard rail will save you.

i don’t view side effects that way tho

Yeah, Rich has said exactly that I think (in Simple Made Easy)

you can say that the function passed to reduce should be effect-free, but it’s frequently useful to gather side data or whatever

(Not saying it's right or wrong. But the decision makes sense when viewed with that mindset I think)

i think the decision makes sense when you consider that conform is trying to say “yup, this thing is valid"

vs “as far as i can tell, this thing isn’t invalid"

the former requires checking everything, the later doesn't

if you want to validate some data off the wire for security reasons, you need the former

but that doesn’t work in the presence of higher order functions or mutation - as the racket contracts folks learned, hence proxies, chaperones, etc - and still, they have problems with side effects b/c they don’t have an effect handlers system

@gfredericks I realize I’m a heretic in clojure-land, but i abuse side effects quite frequently 😛

I've never used it that way either 🙂 I've use reductions when I'm curious what happens during reducing, but not side-accumulation.

I can also say I’ve only ever used the reducing function to .. well, reduce the collection at hand

@bbloom you're a monster

@bbloom can you give an example of a side-effecting reducing function? (like a common use case for what you’re describing)

@bbloom do you sprinkle local atoms all over the place and swap them a bunch and then deref them to figure out what happened

i’ve done stuff like that - yeah

okay.

if it’s local to a function or a concrete process, it’s totally fine

“tree falls in a forrest” and all that

Anyways, not sure I understand what @bbloom was trying to say about conformers with regard to the decision to validate higher order fns, but it makes some amount of sense to me having thought about it a minute.

especially when perf is involved

using map/filter/reduce etc in 9 passes over a long sequence is just not OK for some use cases

i’d love to be able to tell the compiler “hey, these three things traverse the same data structure, do loop fusion” but that’s just not realistic

transducers in the 9 pass case! 🙂 (if applicable)

Yeah, I'm curious how that's more useful than loop/recur or doseq+accumulator.

i do the loop/recur or doseq thing plenty too

Yeah me too

but sometimes yo ujust already have a function, so you just call reduce 🙂

lol

anyway - this is waaaay besides the point

point is that valid? wants to return true, but really the best it can do in the presence of fspec is say “maybe true"

also, i gotta run

thanks all for the discussion