This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-07-28
Channels
- # admin-announcements (4)
- # beginners (11)
- # boot (148)
- # cider (74)
- # cljs-dev (31)
- # cljsrn (30)
- # clojure (55)
- # clojure-berlin (15)
- # clojure-greece (1)
- # clojure-japan (18)
- # clojure-poland (35)
- # clojure-russia (72)
- # clojure-spec (35)
- # clojure-uk (34)
- # clojurescript (134)
- # cursive (26)
- # datomic (42)
- # dirac (7)
- # editors-rus (1)
- # emacs (17)
- # hoplon (29)
- # jobs-rus (3)
- # juxt (1)
- # luminus (11)
- # off-topic (9)
- # om (66)
- # onyx (49)
- # pedestal (1)
- # perun (19)
- # proton (13)
- # protorepl (5)
- # re-frame (31)
- # reagent (13)
- # ring (2)
- # spacemacs (1)
- # specter (40)
- # spirituality-ethics (2)
- # test-check (41)
- # untangled (7)
- # yada (17)
I wish the user guide on the docs, is structured in logical way than just alphabetic order
I can’t help to think that this fake-s3 thing would be so much easier to write in Clojure (LOL), but looks useful!
(I was a Ruby fanboy once, for the record)
maybe it's possible to do on top of pithos (it might be "just" implementing a couple of protocols to do stuff in memory/disk)
@vijaykiran: sorry about that. We've been meaning to rearrange the docs in Jekyll
@mpenet: Sounds like a possibility, although the number of dependencies would scare me https://github.com/exoscale/pithos/blob/master/project.clj#L11
If anything, you could probably copy the test suite and build something light weight and know it is of a similar quality
(Ok back to work :))
@vijaykiran: Working on it, I dislike our current User Guide too. I've been working on compiling it into a one-page document with side-bar anchor links.
The more we grow, the more things that were once simple, like publishing docs, get more complex. 😛
How useful is Onyx on a single peer? It seems like a better way to handle running multiple "handlers" for core.async channels. Or is there something else more suited to this?
Onyx is a great way to model your computation through data!
@dominicm: some people use it for this, yes. It's a trade-off, since Onyx brings the full power and complexity of a tool that orchestrates a cluster. The flip side is that you get a very good computational model.
We are still running on only one peer. We chose onyx exactly because of these reasons
Correct. Also, if your input data source(s) are remote, you're already dealing with a distributed system, and so you can get fault handling for free.
This is very reassuring. I had dismissed Onyx as something for "later when scale comes" but I want an event based model, so I'll create serious bottlenecks if there's only one processor for it
Have a play around with learn-onyx and see if it fits your model. Might be a good fit.
Does a lifecycle before-task-start run before every segment that runs through that particular task? or does it run once as the job spins up?
it will run any time a peer starts operating on a task
so this could happen any time during a job e.g. if a new node starts up.
right, i wasn’t sure if it meant before a segment ran through it. I didn’t want my db connections to get restarted everytime a segment came through. lol
I figured, I just wanted to make sure you got the whole picture 🙂
It depends what you want to do
Triggers attach to tasks that have windows
@gardnervickers: I am setting a lifecycle before-task to setup some DB connections used by the trigger
No, your lifecycle runs before-task
and returns a map that looks like {:db-conn <db-conn>}
. That map is merged into the event
map. For any triggers that are attached to the same task your lifecycle is attached to, they will receive the event
map as an arg, which you can grab the :db-conn
out of.
so I am still confused what task would I attach the lifecycle to so the event ( with db conns ) is available in the trigger?
It would be whatever task you’re windowing on.
If you don’t need to do any processing, you can just make a dummy task
For the state you created in your lifecycle to be available to your trigger, both the lifecycle and trigger need to be attached to the same task.
Your trigger sync function will be supplied the event map
as an argument
@drewverlee: I meant to chime in earlier and say thanks for the PR on the dashboard. Were you able to connect to it?
i haven’t tried it yet, i’m currently struggling to understand why 3 separate datadog processes are running on this server. like who let the dogs out ahhaha 😭