This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-04-07
Channels
- # admin-announcements (4)
- # aws (2)
- # beginners (25)
- # boot (116)
- # braid-chat (4)
- # cider (6)
- # cljsjs (4)
- # cljsrn (17)
- # clojure (196)
- # clojure-austin (23)
- # clojure-belgium (5)
- # clojure-dev (78)
- # clojure-dusseldorf (6)
- # clojure-italy (2)
- # clojure-japan (6)
- # clojure-poland (1)
- # clojure-russia (132)
- # clojure-sdn (1)
- # clojure-uk (26)
- # clojurescript (87)
- # code-reviews (3)
- # core-async (26)
- # cursive (8)
- # datomic (40)
- # devcards (8)
- # emacs (4)
- # funcool (32)
- # hoplon (30)
- # jaunt (34)
- # jobs (1)
- # lein-figwheel (1)
- # leiningen (5)
- # om (14)
- # onyx (8)
- # overtone (31)
- # parinfer (14)
- # proton (8)
- # protorepl (1)
- # re-frame (30)
- # reagent (10)
- # spacemacs (2)
- # untangled (107)
- # yada (3)
That direct link seems to be 404ing @angusiguess
@angusiguess: Running start-supervisor
throws “an invalid offset has been specified”. It would be nice to see your test, replacing simple-consumer
test-check data with a real channel, so we can test this setup with a stream.
@royalaid: Sorry about that: http://blog.goose.haus/2016/04/04/producers-and-error-handling.html
I'll polish things up a little bit more tonight, I have some more feedback to address.
No worries. I’m currently implementing an AWS data pipeline on core.async, and while we have a few nice abstractions, the error recovery is still a mystery. Your inputs are great.
Excellent! I'm slowly working up to trying to write a library to codify some of this.
It would be nice to see your example devoid of any assumptions on client/consumer. It might as well be any external Queue. not just kafka.
Wondering if there is a proscribed way to do unit testing of stuff involving channels?
testing is hard, but I’ve been using repl and plain old clj.test with a consumer/producer.
I am doing a similar thing, just grabbing all of the contents of a channel with (take-while some? (repeatedly #(poll! testing-chan)))
I've written a good few async tests. The small unit tests usually involve pumping individual signals through and checking the results.
I've been working a little bit with property based tests too but still working out how best to do that.
Using alts with a timeout and an assertion will fail the test rather than just hanging on blocking takes.
I'm going to talk about it in a bit but another thing you can do, if you have a stateful go-loop is to dump that state and assert on it.
But I like signals-in signals-out more.
@ghadi: we are talking about ~38 secs instead of 30 secs. I'm running on AWS micro and I think it's just not buff enough to handle the load
@angusiguess: you might want to opensource your async.test lib
@pri: I’ll have to think about how to structure things a little more but I can definitely talk about some of it too.
One of the other benefits of extracting these functions from our go-loop is that we can actually test the functions themselves too.
I’ve updated the blog post and the code to address some feedback, the producer should now recover properly and keep reading.