Fork me on GitHub
#onyx
<
2016-02-24
>
greywolve07:02:26

what size machines do you guys recommend for ZK ?

greywolve07:02:17

assuming our work nodes are 4 core / 8 gb ram

greywolve07:02:25

would 2 core / 4 gb ram be enough?

greywolve07:02:40

for the 3 x ZK 3 x onyx node setup

lucasbradstreet07:02:09

I think those would be more than enough

lucasbradstreet09:02:07

They'll mostly sit idle, but I'm not sure how much lower you can safely push it

lsnape15:02:32

@lucasbradstreet: the groupid for lib-onyx is onyx-platform, but readme says onyxplatform. The groupid for onyx is onyxplatform. Assuming that the documentation is correct but the dep has been uploaded to clojars incorrectly?

lucasbradstreet15:02:16

Yes, it should be onyxplatform

lucasbradstreet15:02:41

I’ll fix it up and will get a release out for you

lucasbradstreet15:02:53

lib-onyx-0.8.12.0-20160224.152123-1 is up

lucasbradstreet15:02:57

on the group-id that’s in the readme

michaeldrogalis15:02:32

Whoops, my bad. T_T

lucasbradstreet15:02:59

I pushed onyx-amazon-sqs to no group-id 😞

lucasbradstreet15:02:07

One downside of immutable artifacts 😕

michaeldrogalis15:02:05

We can ask the Clojars folks to remove some of those bad deploys. Its not the first time simple_smile

lucasbradstreet15:02:28

Yeah, I’ve done it before

lucasbradstreet15:02:37

We’ll queue some up and batch them 😛

michaeldrogalis15:02:17

Decided to take the big plunge. Happy to announce that I'm supporting Onyx full time now! http://www.onyxplatform.org/jekyll/update/2016/02/24/Doubling-Down-on-Onyx.html

leonfs15:02:51

@michaeldrogalis: Congratulations indeed!!

michaeldrogalis15:02:17

Thanks @leonfs @lsnape! Really looking forward to working closer with companies that are dipping their toes into the water with Onyx. The extra time will be great.

nha15:02:38

Big news ! 😄

nha15:02:11

by non-jvm languages, you mean ClojureScript, right ?

michaeldrogalis15:02:20

@nha: We're likely to make a serious effort on Ruby or Python first, but JavaScript would be the 3rd. It mostly depends on who we're working with.

michaeldrogalis15:02:30

I'll release a roadmap in the next 2-3 weeks

nha15:02:40

Oh pretty cool simple_smile

lucasbradstreet15:02:42

I think the main thing is that we have an overall strategy

lucasbradstreet15:02:47

For 3rd party languages

nha16:02:54

I guess python makes a lot of sense in data intensive contexts (although not my thing).

michaeldrogalis16:02:54

It hits home for a lot of machine learning applications.

cddr16:02:41

Congratulations! I learnt a lot about the problem space in general from the amazing docs that you guys produced and your pointers on slack/gitter. Wish you luck in securing some lucrative contracts simple_smile

lucasbradstreet16:02:14

Thanks @cddr. I actually pointed a new client at integrity a few days back

cddr16:02:19

Oh yeah. I really need to put some work into that. I've been a bad maintainer

cddr16:02:05

Which part specifically? I feel in hindsight it maybe bundles too many disparate things together.

lucasbradstreet16:02:42

They are using Onyx + Datomic and were going to build their own meta-language for validation between the two and their web client

lucasbradstreet16:02:16

I’m not sure if they’ll end up using it, but I think they’ll probably at least use some of the ideas

mccraigmccraig19:02:33

i'm having a nasty problem in production atm - occasionally (monthly or so) the AWS cluster seems to fall to bits, and ZK and kafka go down for a while... whatever network hiccup is causing the problem goes away after a while, ZK recovers and mesos/kafka restarts the brokers - problem is onyx never recovers - the process seems to get wedged, and so far the only solution seems to be to submit a job to a new onyx-id with a new kafka consumer-id (a new onyx-id with the old consumer-id stays wedged)

mccraigmccraig19:02:12

@lucasbradstreet previously suggested i kill and restart the job when this happens... i couldn't do that because i hadn't wired up the kill api (i have now)... but since creating a new onyx-id still stays wedged i'm guessing the problem is elsewhere

mccraigmccraig19:02:26

dyu have any ideas for debugging this ?

mccraigmccraig19:02:46

and congratulations on the next phase of onyx !

michaeldrogalis19:02:23

@mccraigmccraig: Thanks! As far as your deployment, there's a lot there. It's hard to know where to start looking when AWS, Mesos, Kafka, Onyx, and your app are involved. I'd need to have a better idea of what the application itself is doing and what the configuration of the Kafka catalog entries are.

michaeldrogalis19:02:36

Will switch to PM for this one.

thomas20:02:29

wonderful news for the Onyx team!!! Congrats!!!

thomas20:02:56

And I really hope you (plural) will succeed.

michaeldrogalis20:02:25

@thomas: Thanks, we're looking forward to trying - that's for sure!

Drew Verlee21:02:48

@michaeldrogalis: I'm excited you have the ability to work on onyx full-time!!! 🎉

michaeldrogalis21:02:09

@drewverlee: Me too! Looking forward to building some awesome back end systems in the wild.