This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-03-31
Channels
- # admin-announcements (1)
- # beginners (16)
- # boot (25)
- # braid-chat (7)
- # cider (4)
- # cljsfiddle (6)
- # cljsrn (1)
- # clojure (256)
- # clojure-austin (4)
- # clojure-ireland (1)
- # clojure-poland (15)
- # clojure-russia (80)
- # clojure-uk (2)
- # clojurescript (30)
- # core-async (14)
- # core-typed (3)
- # cursive (35)
- # datomic (1)
- # editors (28)
- # hoplon (32)
- # immutant (1)
- # jobs (8)
- # jobs-discuss (1)
- # juxt (6)
- # leiningen (8)
- # liberator (7)
- # off-topic (16)
- # om (69)
- # onyx (38)
- # re-frame (10)
- # spacemacs (1)
- # untangled (117)
We don’t have any samples of that, but you could implement it pretty easily with triggers: e.g. https://github.com/onyx-platform/onyx-examples/blob/0.9.x/fixed-windows/src/fixed_windows/core.clj#L109
@otfrom @jasonbell I’m also thinking of building something that writes to S3 that way. I've started playing with Onyx just today so not sure if/when I can be of help
@jeroenvandijk: thanks for letting me (and @otfrom) know. I’ve just started looking today too.
Wild idea: use onyx to create an http load testing tool? Sounds realistic? Maybe something like http://engulf-project.org/ would simpler to implement with Onyx. Would be a nice example application that would be rather useful
@jeroenvandijk: Definitely could. I had some ideas about building a simulation testing suite with Onyx. Not enough hours in the day.
@michaeldrogalis: Cool Yeah I was mostly interested in the potential of the idea. I might try it at some point as it sounds pretty useful for us. Engulf is good, but still limited and is not maintained anymore.
But I should probably first finish learn-onyx before fantasizing further
@jeroenvandijk: Wise decision.
Should ack-segment (PipelineInput) ever receive nil as segment-id? Seems to happen with my input plugin when :onyx/pending-timeout is reached. Every pending segment just retries forever.
nope, it definitely shouldn’t. Which plugin are you seeing this in?
lucasbradstreet: sorry! the segment id is not nil, but the segment does not exist in pending-messages. This is a custom input-plugin. It seems like retried segments never get acked after pending-timeout is reached
It’s possible that they’ve already been retried by the time the ack-segment comes in, in which case you wouldn’t be able to dissoc it. You would ignore it in that case
That may not be your problem though
Is it possible that you’re acking slow enough that you main read-ch becomes overly full and then your retries blocked putting onto the read-ch?
Do they ever get read again from read-batch after a retry?
OK, sorry, I won’t be able to help much more today. It’s late and I’m off to sleep
(<!! (async/timeout 1000)) (>!! read-ch (t/input (random-uuid) (:message segment))) (swap! pending-messages dissoc segment-id)
off to sleep myself. I'll experiment more tomorrow and would love some tips on how to implement this properly
@ckarlsen: We have a new interface coming for input plugins next release. Should make writing your own I/O plugins waaay easier.
Will probably be out in in beta in ~8 weeks though, so worth it to write your own now since a good 80% of it will carry over.
We do have the SimpleInput plugin
True. That's already in 0.9.0