Fork me on GitHub
#conf-proposals
<
2016-08-11
>
bvulpes01:08:22

good feedback, thanks seancorfield

bvulpes01:08:38

working title is "Rigorous Testing of Time-Sensitive Systems"

bvulpes01:08:30

while testing is the broad focus, the testing of the scheduler is what the talk'll be about, so the abstract deserves a rewrite to focus on that.

seancorfield01:08:53

Sounds interesting!

bvulpes01:08:54

maybe something like "testing in clojure 334: mocking, dependency injection, and a deep dive on rigorous testing of a reentrant scheduler"

Alex Miller (Clojure team)02:08:05

My theory is that any title with a : or a - in it should be cut in half :)

lvh03:08:56

I have just submitted my proposal to the conj 🙂 I’m pretty excited; it’s a new venue for me

lvh03:08:32

(My proposal is about calling C code, so I don’t know how many people care — but I think it’s an excellent example of how Clojure’s strengths are really quite general and it turns out to do well in corners where you might not expect it)

bvulpes03:08:14

whoa c code?

lvh03:08:23

@alexmiller: alas I comically missed the SL CFP; my mom is visiting so I won't be able to make that :(

lvh03:08:11

@bvulpes glad to hear; my main concern is audience; it doesn't seem very popular right now

lvh03:08:26

Could be a chicken-egg problem :)

seancorfield05:08:19

Feedback on this: "Specifying the World (Singles) World Singles runs around 100 Internet dating sites, with over 40,000 lines of Clojure at the core. As part of a massive refactoring to support a new API-based version of our platform, we're focusing on our data model and the associated validation and data transformation rules. Find out how we're using clojure.spec across multiple application layers to engage our business team and clarify our rules, improve robustness and confidence, and simplify our code. I'll show how we use clojure.spec to support three different data models (input, domain, and persistence), to automate validation and testing, and to ease the transition between them."

ericnormand15:08:25

@seancorfield: I'm especially interested in testing using clojure.spec

ericnormand16:08:07

have you come up with good ways of tying clojure.spec into clojure.test?

eggsyntax16:08:48

@ericnormand: so far folks seem to mostly be treating it as a distinct test architecture, ie different lifecycle from clojure.test testing, because it may take much longer if you're pushing lots of generated data through your code.

eggsyntax16:08:27

That's just based on anecdotes, though, mostly in #C1B1BB2Q3

eggsyntax16:08:07

We're thinking about primarily having our CI server do it.

ericnormand16:08:08

I definitely used test.check with clojure.test

ericnormand16:08:15

it makes some sense, though

ericnormand16:08:27

with a CI server, you could have it continuously run the tests

ericnormand16:08:22

and log any failing cases somewhere

eggsyntax16:08:41

Yep, or just announce them in build notifications.

ericnormand16:08:02

well, the problem is you might find some after the build has been deployed

eggsyntax16:08:22

(but yeah, with drillable detail of course -- especially useful with shrinking 😄 )

eggsyntax16:08:55

Sure, yeah, depends on your branch/deployment choices I guess.

eggsyntax16:08:54

But I'd expect to have regular unit tests to catch common or critical failures, with the generated stuff primarily catching edge cases. You could also have separate configs such that some amount of generated data is pushed through your code pre-deployment, and then lots more on a less frequent basis to catch rare edge cases before users do.

eggsyntax16:08:16

I feel like best practices around spec are still very much in flux, though.

ericnormand16:08:20

that's why I was asking about how they do testing

ericnormand16:08:09

I think you're right about pushing a small # of generated data through before is a good idea

ericnormand16:08:25

and I had good experiences doing that with test.check

eggsyntax16:08:29

And I'd trust Sean's opinion way more than my own here, since he's been using it in prod for an extended amount of time, whereas our usage is still pretty experimental.

ericnormand16:08:04

so far, my testing with clojure.spec has been limited

ericnormand16:08:22

it has usually taken too long to run the tests

ericnormand16:08:36

I did not experience that with test.check

ericnormand16:08:43

usually because I was writing custom generators

seancorfield16:08:27

@ericnormand: we already use test.check with Expectations for property-based testing so this isn’t terribly different for us: we expect certain things of the summarized result of running check...

ericnormand16:08:19

and how do you print out the results of failures?

seancorfield16:08:22

We’re currently rebuilding our CI environments and planning to have a suite of generative tests that run there and run longer (bigger test size) that either don’t run as part of the DEV tests or only use small test sizes.

seancorfield16:08:12

@ericnormand: We use expect more-of and run the output of stest/check thru stest/abbrev-result — Expectations does a good job of pretty printing the :failures key if there are any.

seancorfield16:08:58

I may look at enhancing Expectations so that it can do it more clearly, but it’s pretty good already

seancorfield16:08:02

So… not much feedback in terms of changes to my title / abstract? Doesn’t look like @alexmiller is around for feedback...

ericnormand17:08:16

@seancorfield: maybe I'd put clojure.spec in the title

seancorfield17:08:03

"Specifying the World (Singles) with clojure.spec"?

seancorfield17:08:21

I think that loses some snappiness — and @alexmiller already hinted that titles with : and - are often too long and should be split / shortened. I think that applies to with a bit as well.

ericnormand17:08:51

> (clojure.)Specifying the World (Singles)

blueberry17:08:29

@ericnormand: That might contain too many parens, even for lispers 🙂

Alex Miller (Clojure team)18:08:51

Your last title is ok although it seems like you have a more specific focus on your abstract re layered data models that maybe could be the focus of the title

seancorfield18:08:40

"Specifying Layered Data Models with clojure.spec"?

ericnormand18:08:43

comments welcome: Testing stateful, concurrent, and async systems using test.check Generative testing is great for testing pure functions. But it is also used to test the behavior of stateful systems that change over time. They’re often designed for highly concurrent usage, such as databases and web services. And then sometimes your code is also asynchronous, such as in JavaScript. Learn several patterns for testing these more interesting situations. These find race conditions, corner cases, and inconsistencies across platforms.

ericnormand18:08:18

actually, please rip it apart

ericnormand18:08:32

would love to put it through the gauntlet

Alex Miller (Clojure team)18:08:11

so this is a good hook. the part people often don’t do as well is to tell the reviewers in the private comments what the “several patterns” are explicitly - that is, what are you actually going to talk about.

Alex Miller (Clojure team)18:08:53

you don’t need to do so here, but that’s what I (as a reviewer) want/need to see - because that’s actually more useful to us in understanding the talk

Alex Miller (Clojure team)18:08:08

(and believing you actually have specific things and didn’t just write the abstract :)

Alex Miller (Clojure team)18:08:18

if a proposal comes in early enough that I can ask that question, I will often go back to the submitter and do so, but most proposals come in in the last few days and it’s not possible to read and respond to them. In that case, reviewers have to make their best guess. And I guess conservatively. :)

ericnormand18:08:56

how much introduction to test.check will I need?

Alex Miller (Clojure team)18:08:08

in the abstract or in the actual talk?

ericnormand18:08:00

and is it still earlier enough in the cfp for the back and forth you just mentioned?

Alex Miller (Clojure team)18:08:19

yes re back and forth :)

Alex Miller (Clojure team)18:08:49

I think in the talk it depends what expectation you want to set for the attendees

Alex Miller (Clojure team)18:08:05

your abstract should make that clear

Alex Miller (Clojure team)18:08:25

the #1 cause of an attendee not liking a talk is because it was different than their expectation

Alex Miller (Clojure team)18:08:58

if you don’t state some level of assumed knowledge in the abstract then it’s on you to explain it (or not meet some people’s expectations)

Alex Miller (Clojure team)18:08:24

it’s totally fine though to say “assumes basic knowledge of generative testing or test.check” or something like that

Alex Miller (Clojure team)18:08:38

assuming we are multi-track for the conj, which I think we are (don’t remember)

Alex Miller (Clojure team)18:08:46

then it’s on me to schedule it right :)

Alex Miller (Clojure team)18:08:13

if it’s single track then I think a certain amount of explanation is required

ericnormand18:08:05

ok, thanks for this

ericnormand18:08:03

would you recommend putting in a link to some place where they can get the basic knowledge?

ericnormand18:08:12

like a prior clojure/west talk?

ericnormand18:08:24

oh, and now I'm filling out the form and it says maximum 5 sentences for the abstract

ericnormand18:08:46

would you rather me cut it down or do some runons? 🙂

seancorfield19:08:50

Once we’ve submitted a talk, do we get a chance to update it if we think of new stuff?

seancorfield19:08:43

Like expanding on the main points or whatever (your response to Eric above made me wonder if I needed to be more expansive in that section).

Alex Miller (Clojure team)19:08:36

@ericnormand: link/reference is fine. I’d treat what the form says as general guidance re length - yours looks fine.

Alex Miller (Clojure team)19:08:12

@seancorfield: if you want to send me a replacement I can update your entry, or you can also just resubmit. I usually run a pass before we do anything and delete old revs of a talk

Alex Miller (Clojure team)19:08:33

in either case, a heads up would be helpful

ericnormand19:08:25

ok, submitted!

lvh23:08:47

@ericnormand: Mostly just positive feedback: I have used test.check for exactly that purpose and it was pretty great 🙂