Fork me on GitHub

good feedback, thanks seancorfield


working title is "Rigorous Testing of Time-Sensitive Systems"


while testing is the broad focus, the testing of the scheduler is what the talk'll be about, so the abstract deserves a rewrite to focus on that.


Sounds interesting!


maybe something like "testing in clojure 334: mocking, dependency injection, and a deep dive on rigorous testing of a reentrant scheduler"

Alex Miller (Clojure team)02:08:05

My theory is that any title with a : or a - in it should be cut in half :)


I have just submitted my proposal to the conj 🙂 I’m pretty excited; it’s a new venue for me


(My proposal is about calling C code, so I don’t know how many people care — but I think it’s an excellent example of how Clojure’s strengths are really quite general and it turns out to do well in corners where you might not expect it)


whoa c code?


@alexmiller: alas I comically missed the SL CFP; my mom is visiting so I won't be able to make that :(


@bvulpes glad to hear; my main concern is audience; it doesn't seem very popular right now


Could be a chicken-egg problem :)


Feedback on this: "Specifying the World (Singles) World Singles runs around 100 Internet dating sites, with over 40,000 lines of Clojure at the core. As part of a massive refactoring to support a new API-based version of our platform, we're focusing on our data model and the associated validation and data transformation rules. Find out how we're using clojure.spec across multiple application layers to engage our business team and clarify our rules, improve robustness and confidence, and simplify our code. I'll show how we use clojure.spec to support three different data models (input, domain, and persistence), to automate validation and testing, and to ease the transition between them."


@seancorfield: I'm especially interested in testing using clojure.spec


have you come up with good ways of tying clojure.spec into clojure.test?


@ericnormand: so far folks seem to mostly be treating it as a distinct test architecture, ie different lifecycle from clojure.test testing, because it may take much longer if you're pushing lots of generated data through your code.


That's just based on anecdotes, though, mostly in #C1B1BB2Q3


We're thinking about primarily having our CI server do it.


I definitely used test.check with clojure.test


it makes some sense, though


with a CI server, you could have it continuously run the tests


and log any failing cases somewhere


Yep, or just announce them in build notifications.


well, the problem is you might find some after the build has been deployed


(but yeah, with drillable detail of course -- especially useful with shrinking 😄 )


Sure, yeah, depends on your branch/deployment choices I guess.


But I'd expect to have regular unit tests to catch common or critical failures, with the generated stuff primarily catching edge cases. You could also have separate configs such that some amount of generated data is pushed through your code pre-deployment, and then lots more on a less frequent basis to catch rare edge cases before users do.


I feel like best practices around spec are still very much in flux, though.


that's why I was asking about how they do testing


I think you're right about pushing a small # of generated data through before is a good idea


and I had good experiences doing that with test.check


And I'd trust Sean's opinion way more than my own here, since he's been using it in prod for an extended amount of time, whereas our usage is still pretty experimental.


so far, my testing with clojure.spec has been limited


it has usually taken too long to run the tests


I did not experience that with test.check


usually because I was writing custom generators


@ericnormand: we already use test.check with Expectations for property-based testing so this isn’t terribly different for us: we expect certain things of the summarized result of running check...


and how do you print out the results of failures?


We’re currently rebuilding our CI environments and planning to have a suite of generative tests that run there and run longer (bigger test size) that either don’t run as part of the DEV tests or only use small test sizes.


@ericnormand: We use expect more-of and run the output of stest/check thru stest/abbrev-result — Expectations does a good job of pretty printing the :failures key if there are any.


I may look at enhancing Expectations so that it can do it more clearly, but it’s pretty good already


So… not much feedback in terms of changes to my title / abstract? Doesn’t look like @alexmiller is around for feedback...


@seancorfield: maybe I'd put clojure.spec in the title


"Specifying the World (Singles) with clojure.spec"?


I think that loses some snappiness — and @alexmiller already hinted that titles with : and - are often too long and should be split / shortened. I think that applies to with a bit as well.


> (clojure.)Specifying the World (Singles)


@ericnormand: That might contain too many parens, even for lispers 🙂

Alex Miller (Clojure team)18:08:51

Your last title is ok although it seems like you have a more specific focus on your abstract re layered data models that maybe could be the focus of the title


"Specifying Layered Data Models with clojure.spec"?


comments welcome: Testing stateful, concurrent, and async systems using test.check Generative testing is great for testing pure functions. But it is also used to test the behavior of stateful systems that change over time. They’re often designed for highly concurrent usage, such as databases and web services. And then sometimes your code is also asynchronous, such as in JavaScript. Learn several patterns for testing these more interesting situations. These find race conditions, corner cases, and inconsistencies across platforms.


actually, please rip it apart


would love to put it through the gauntlet

Alex Miller (Clojure team)18:08:11

so this is a good hook. the part people often don’t do as well is to tell the reviewers in the private comments what the “several patterns” are explicitly - that is, what are you actually going to talk about.

Alex Miller (Clojure team)18:08:53

you don’t need to do so here, but that’s what I (as a reviewer) want/need to see - because that’s actually more useful to us in understanding the talk

Alex Miller (Clojure team)18:08:08

(and believing you actually have specific things and didn’t just write the abstract :)

Alex Miller (Clojure team)18:08:18

if a proposal comes in early enough that I can ask that question, I will often go back to the submitter and do so, but most proposals come in in the last few days and it’s not possible to read and respond to them. In that case, reviewers have to make their best guess. And I guess conservatively. :)


how much introduction to test.check will I need?

Alex Miller (Clojure team)18:08:08

in the abstract or in the actual talk?


and is it still earlier enough in the cfp for the back and forth you just mentioned?

Alex Miller (Clojure team)18:08:19

yes re back and forth :)

Alex Miller (Clojure team)18:08:49

I think in the talk it depends what expectation you want to set for the attendees

Alex Miller (Clojure team)18:08:05

your abstract should make that clear

Alex Miller (Clojure team)18:08:25

the #1 cause of an attendee not liking a talk is because it was different than their expectation

Alex Miller (Clojure team)18:08:58

if you don’t state some level of assumed knowledge in the abstract then it’s on you to explain it (or not meet some people’s expectations)

Alex Miller (Clojure team)18:08:24

it’s totally fine though to say “assumes basic knowledge of generative testing or test.check” or something like that

Alex Miller (Clojure team)18:08:38

assuming we are multi-track for the conj, which I think we are (don’t remember)

Alex Miller (Clojure team)18:08:46

then it’s on me to schedule it right :)

Alex Miller (Clojure team)18:08:13

if it’s single track then I think a certain amount of explanation is required


ok, thanks for this


would you recommend putting in a link to some place where they can get the basic knowledge?


like a prior clojure/west talk?


oh, and now I'm filling out the form and it says maximum 5 sentences for the abstract


would you rather me cut it down or do some runons? 🙂


Once we’ve submitted a talk, do we get a chance to update it if we think of new stuff?


Like expanding on the main points or whatever (your response to Eric above made me wonder if I needed to be more expansive in that section).

Alex Miller (Clojure team)19:08:36

@ericnormand: link/reference is fine. I’d treat what the form says as general guidance re length - yours looks fine.

Alex Miller (Clojure team)19:08:12

@seancorfield: if you want to send me a replacement I can update your entry, or you can also just resubmit. I usually run a pass before we do anything and delete old revs of a talk

Alex Miller (Clojure team)19:08:33

in either case, a heads up would be helpful


ok, submitted!


@ericnormand: Mostly just positive feedback: I have used test.check for exactly that purpose and it was pretty great 🙂