This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2015-10-30
Channels
- # admin-announcements (11)
- # announcements (1)
- # beginners (1)
- # boot (247)
- # cider (15)
- # clara (16)
- # cljs-dev (14)
- # clojure (118)
- # clojure-czech (10)
- # clojure-ecuador (1)
- # clojure-japan (1)
- # clojure-russia (22)
- # clojurescript (57)
- # data-science (12)
- # datomic (2)
- # devcards (1)
- # editors-rus (2)
- # emacs (1)
- # events (2)
- # funcool (1)
- # hoplon (37)
- # juxt (1)
- # ldnclj (2)
- # leiningen (1)
- # nginx (12)
- # off-topic (16)
- # om (361)
- # onyx (34)
- # re-frame (3)
- # spacemacs (9)
- # yada (43)
has anyone off loaded any of the processing of things in onyx to aws lambda? yet? or is that a crazy idea
@clojuregeek: They're more competing than complementary.
hmm ok
see i am still looking for excuse to use onyx 😉
You're a champ 😉
so the problem now i have withs 4 aws lambda functions, is some way to make sure things follow in order, how to see where i am in the process...
i thought, maybe I need a statemachine?
@clojuregeek: So, like, each piece of data, you want to track it as it moves through your processing pipeline - because presumably each stage takes a long time?
I haven't used Lambda, but I don't think it's a good fit for that - neither is Onyx. I mean, are we talking seconds or minutes+ for completion?
maybe useing kafka (kensises on aws) ... 4 queues
seconds
Oh - seconds is fine for both of these, then.
Since you're using AWS, maybe use Dynamo to track state as things move across the pipeline, keyed by some ID?
yes, i think surely i need a database ... the data can come from cloudtrail or maybe i have a collection of coludtrail yaml files that I want to use (as proof of concept for example)
What're the stages of your processing doing?
I'm reading in a cloudtrail log ... and if the log is someone changing tags on an instance .... i want to convert that change to TOSCA ... Then take that instance id and get "what are the tags right now" and make a new tosca document with those tags ... then run through a validation process to make sure it has the tags required for the business
TOSCA is a format to describe cloud orchestration http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.0/csd02/TOSCA-Simple-Profile-YAML-v1.0-csd02.html .... that my boss thinks is the future and wants to move towards
Hah. Yeah, Onyx would be excellent for doing that.
i didn't explain all that before?
I hear a lot of use cases every day. Sometimes I forget 😉
i may have been vague at some points
but now i'm not so much
Yeah, it sounds like you have it down at this point.
part of what converts cloudtrail to tosca we are going to open source soon
Cool. So yes, I'd pick either Lambda or Onyx here - mixing them will hurt.
so with oynx i'd have the pipeline that will do this ... with Lambda, i have to write something that will be the pipeline
I'm not really sure what Lambda offers you tbh.
its making it easy to deploy without figuring out how to do it 😉
just upload a jar
I see. You'll have to do a little more foot-work with Onyx. Same idea, though.
we've been trying to put output of this thing in sqs ... but then lambda can't pull from sqs.. only sns ...
its just alot of plumbing
Yeah, there's always a lotta that. 😧