This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-07-17
Channels
- # admin-announcements (1)
- # boot (24)
- # cider (8)
- # cljsjs (1)
- # cljsrn (5)
- # clojure (22)
- # clojure-greece (1)
- # clojure-italy (3)
- # clojure-russia (218)
- # clojure-spec (33)
- # clojurescript (51)
- # core-async (6)
- # cursive (1)
- # datomic (13)
- # defnpodcast (2)
- # funcool (2)
- # lein-figwheel (21)
- # onyx (41)
- # proto-repl (4)
- # protorepl (5)
- # reagent (4)
hi there! Can I use Onyx platform like distributed app server? I want put some functionality to Onyx which is traditionally run on app servers in web development. Can I read/write to DB from onyx functions or interact with some web services to process user request?
example: I want put user requests to http://atomix.io distributed queue an then read them form queue by onyx, process request in onyx, and finally put result to http://atomix.io
kafka looks a bit heavy for this tasks
@mike1452: > Can I read/write to DB from onyx functions or interact with some web services to process user request? Not sure what you mean by “onyx functions”, but it is the case that Onyx would be able to read from a DB. I assume it can also ingest http requests (though i haven’t personally tried this) I believe thats what this plugin allows for: https://github.com/onyx-platform/onyx-http. > I want put user requests to http://atomix.io distributed queue an then read them form queue by onyx, process request in onyx, and finally put result to http://atomix.io I can’t tell from that link what persistent storage atomix is using, so its hard for me to envision where the data that onyx processes would endup. Regardless, I believe the hurdle here is writing a plugin that acts as an intermediary between onyx and the storage. > kafka looks a bit heavy for this tasks Until very recently kafka didn’t have any native data processing capabilities.
^ but you probably want to have clarification from the core team on these points
Two questions both related to lifecycles
————————
1. when is the state of something like an atom (thats used in a lifecycle) set?
2. changing params at runtime
1. when is the state of something like an atom (thats used in a lifecycle) set?
----------
in this exercise https://github.com/onyx-platform/learn-onyx/blob/answers/src/workshop/challenge_4_2.clj
I changed
(def state (atom nil))
to (def state (atom 0))
in hopes of having to check for the nil value. When i ran the test, it still took on the nil value however. Sense the state is injected before the task starts, how is this possible?
2. changing params at runtime
———————————————
Is it possible to to dynamically change the paramaters of a task at runtime? I believe this is possible using lifecycles.
You have a running onyx job and you allow a client to send an argument to a running task. Example picture below hopefully will clarify and ambiguity.
@drewverlee: thanx for an answer. onyx-http looks like it is output plugin only, not for ingest. kafka is heavy in that sense that it is another moving part. atomix do the same job but it is embedded lib.
Writing a plugin for reading/writing from an Atomix backed queue would be very straight forward. Look to the Onyx bookkeeper plugin. You will have to manage the Atomix replica yourself, probably through lifecycles on the plugin.
@drewverlee: as for question 1, i would need to see an example, I can't make out what problem your seeing. As for 2, yea this is possible through the event map.
@mike1452: If you don't care whether or not Onyx can replay failed messages (which requires an input source that can tolerant a fault or provide replays), you can use Durable Queue. https://github.com/onyx-platform/onyx-durable-queue
We don't have a lot of options for "light" input sources that aren't replicated or fault tolerant because that's a pretty important moving part. It would be like getting rid of the steering wheel on a car.
@mike1452: Atomix will be able to provide the guarantees you’re after if you make it sync to disk on every write, and put all writes through the cluster leader. If you’re trying to use other datasources, or don’t care about message guarantees, it’s often easier to structure your reads as a lazy-seq and use https://github.com/onyx-platform/onyx-seq
Hi guys. Really simple question as I’m digging around the docs and don’t see anything obvious. If I lein uberjar
and I get a -SNAPSHOT.jar
and peer.jar
, which do I run from command line java -jar
etc and how. The deployment docs aren’t overly clear. Thanks.
@jasonbell: It's application-specific. Your peer jar should just launch your peers and load all relevant functions onto the classpath. Just confirming you found the same.
@michaeldrogalis: I ran it like this:
java -cp $(lein classpath):target/smstest-0.1.0-SNAPSHOT.jar -server smstest.core start-peers 10 -c resources/config.edn
If I’m doing it wrong now’s a good time to tell me as I’m rattling out a blog post to help others 🙂
uberjar will package your dependencies into a fat jar so you don’t need to get the classpath from lein.
@jasonbell: Thank you ❤️ ❤️
@gardnervickers: I get two jar files, the SNAPSHOT does have deps, and peer gives me a repl when I run it.
The peer jar should boot up, launch the virtual peers, and block. Are you basing it off the template?
that right
This is how we’re starting the peers w/ docker
If you’re not in docker, you can remove the stuff about calculating heap mem
@jasonbell: No prob, and thanks for writing up a post. That's great. ^^
Very nice
Thanks @jasonbell. Ill tweet it out tomorrow morning the main account so it gets more viewers 🙂
I can’t say it’s quality but if you believe it to be helpful to someone then please feel free.
Probably 🙂