This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2017-10-24
Channels
- # aws (7)
- # aws-lambda (3)
- # beginners (65)
- # boot (43)
- # cider (7)
- # cljs-dev (12)
- # cljsrn (15)
- # clojure (284)
- # clojure-austin (32)
- # clojure-brasil (4)
- # clojure-dusseldorf (4)
- # clojure-germany (1)
- # clojure-italy (40)
- # clojure-spec (21)
- # clojure-uk (69)
- # clojurescript (97)
- # core-async (11)
- # cursive (19)
- # data-science (1)
- # datascript (6)
- # datomic (30)
- # dirac (2)
- # emacs (4)
- # events (2)
- # fulcro (76)
- # graphql (38)
- # juxt (1)
- # lein-figwheel (1)
- # leiningen (6)
- # luminus (4)
- # lumo (13)
- # mount (4)
- # off-topic (24)
- # om (28)
- # onyx (32)
- # other-languages (1)
- # parinfer (40)
- # pedestal (1)
- # portkey (47)
- # re-frame (21)
- # reagent (4)
- # ring (4)
- # ring-swagger (3)
- # rum (1)
- # shadow-cljs (115)
- # spacemacs (5)
- # sql (14)
- # unrepl (1)
- # yada (3)
vvgkdiiuvlvhevsukklhbspsugudgukvkbtdlpdhlpvg
oops, when you hit the ol’ yubikey when rummaging around the back of your computer 😛
@michaeldrogalis the pool is now allocated directly in the constructor, e,g. https://github.com/onyx-platform/onyx-sql/blob/0.11.x/src/onyx/plugin/sql.clj#L174 however, i think @lucasbradstreet is right — we dont do any cleanup at the moment of this pool. if we want to do that (which we do), i think we want to keep those calls. shall i take a look at this instead ?
I think it’s ok to drop the lifecycles, but we should clean up the connections in here: https://github.com/onyx-platform/onyx-sql/blob/0.11.x/src/onyx/plugin/sql.clj#L191
Hey guys I know i have seen this mentioned before but I am running into the issue of
:cause "No response from MediaDriver within (ns):10000000000"
:data {:original-exception :io.aeron.exceptions.DriverTimeoutException}
@camechis Does that happen as soon as it boots up, or after a period? Either your media driver can't be connected to, or its getting starved for resources.
@lmergen Thanks 🙂
pretty sure its talking given that the dashboard is showing the correct number of peers and I can run some data through it successfully but it doesn’t stay up long even when no data is running through it ( idle basically )
What sort of resources are allocated to it? I think you said this also runs under K8s?
that is correct running on GKE. Good question on the actual resources it has as I am still new to K8, lol
I dont think no set limit translates to unlimited, so definitely have a look
@camechis hmm. Those timeouts are most likely after long GCs, but in your case it’s dying while idle
Yeah, I am a little unclear on whats causing it because it has been running now for an hour so. ( idle for the most part )
I meant the JVM gc rather than the onyx log GC
Mostly because GCs = no heartbeating to aeron = timeout
It depends on how much churn you have in your cluster. If it’s not much then I don’t recommend GC’ing at the moment, as I think it needs some more jepsen testing.
It seemed like gc only trims the logs and not the checkpoints? I can write my own checkpoint trimmer, are there any best-practices I should be aware of?
That’s correct. If you could contribute a checkpoint trimmer that’d be great :)
Whenever a coordinate is successfully written, all other checkpoints for a job are game to be trimmed https://github.com/onyx-platform/onyx/blob/0.11.x/src/onyx/log/zookeeper.clj#L693
The coordinate takes the form of {:epoch X :replica-version Y :tenancy T :job-id J}
You can essentially poll or watch the coordinate, and remove all checkpoints for all lesser epochs with an equivalent replica-version, or all epochs for lesser replica-versions (replica version + epoch form a vector clock)