hi there! Can I use Onyx platform like distributed app server? I want put some functionality to Onyx which is traditionally run on app servers in web development. Can I read/write to DB from onyx functions or interact with some web services to process user request?

example: I want put user requests to http://atomix.io distributed queue an then read them form queue by onyx, process request in onyx, and finally put result to http://atomix.io

kafka looks a bit heavy for this tasks

@mike1452: > Can I read/write to DB from onyx functions or interact with some web services to process user request? Not sure what you mean by “onyx functions”, but it is the case that Onyx would be able to read from a DB. I assume it can also ingest http requests (though i haven’t personally tried this) I believe thats what this plugin allows for: https://github.com/onyx-platform/onyx-http. > I want put user requests to http://atomix.io distributed queue an then read them form queue by onyx, process request in onyx, and finally put result to http://atomix.io I can’t tell from that link what persistent storage atomix is using, so its hard for me to envision where the data that onyx processes would endup. Regardless, I believe the hurdle here is writing a plugin that acts as an intermediary between onyx and the storage. > kafka looks a bit heavy for this tasks Until very recently kafka didn’t have any native data processing capabilities.

^ but you probably want to have clarification from the core team on these points

Two questions both related to lifecycles ———————— 1. when is the state of something like an atom (thats used in a lifecycle) set? 2. changing params at runtime 1. when is the state of something like an atom (thats used in a lifecycle) set? ---------- in this exercise https://github.com/onyx-platform/learn-onyx/blob/answers/src/workshop/challenge_4_2.clj I changed (def state (atom nil)) to (def state (atom 0)) in hopes of having to check for the nil value. When i ran the test, it still took on the nil value however. Sense the state is injected before the task starts, how is this possible? 2. changing params at runtime ——————————————— Is it possible to to dynamically change the paramaters of a task at runtime? I believe this is possible using lifecycles. You have a running onyx job and you allow a client to send an argument to a running task. Example picture below hopefully will clarify and ambiguity.

@drewverlee: thanx for an answer. onyx-http looks like it is output plugin only, not for ingest. kafka is heavy in that sense that it is another moving part. atomix do the same job but it is embedded lib.

Writing a plugin for reading/writing from an Atomix backed queue would be very straight forward. Look to the Onyx bookkeeper plugin. You will have to manage the Atomix replica yourself, probably through lifecycles on the plugin.

@drewverlee: as for question 1, i would need to see an example, I can't make out what problem your seeing. As for 2, yea this is possible through the event map.

@mike1452: If you don't care whether or not Onyx can replay failed messages (which requires an input source that can tolerant a fault or provide replays), you can use Durable Queue. https://github.com/onyx-platform/onyx-durable-queue

We don't have a lot of options for "light" input sources that aren't replicated or fault tolerant because that's a pretty important moving part. It would be like getting rid of the steering wheel on a car.

@mike1452: Atomix will be able to provide the guarantees you’re after if you make it sync to disk on every write, and put all writes through the cluster leader. If you’re trying to use other datasources, or don’t care about message guarantees, it’s often easier to structure your reads as a lazy-seq and use https://github.com/onyx-platform/onyx-seq

Hi guys. Really simple question as I’m digging around the docs and don’t see anything obvious. If I lein uberjar and I get a -SNAPSHOT.jar and peer.jar, which do I run from command line java -jar etc and how. The deployment docs aren’t overly clear. Thanks.

It’s okay, I got it.

@jasonbell: It's application-specific. Your peer jar should just launch your peers and load all relevant functions onto the classpath. Just confirming you found the same.

@michaeldrogalis: I ran it like this:

java -cp $(lein classpath):target/smstest-0.1.0-SNAPSHOT.jar -server smstest.core  start-peers 10 -c resources/config.edn

If I’m doing it wrong now’s a good time to tell me as I’m rattling out a blog post to help others 🙂

uberjar will package your dependencies into a fat jar so you don’t need to get the classpath from lein.

@jasonbell: Thank you ❤️ ❤️

@gardnervickers: I get two jar files, the SNAPSHOT does have deps, and peer gives me a repl when I run it.

The peer jar should boot up, launch the virtual peers, and block. Are you basing it off the template?

lein new onyx-template my-app < That’s how I created it.

Sorry

onyx-app

Not onyx-template

that right

https://github.com/onyx-platform/onyx-template/blob/0.9.x/src/leiningen/new/onyx_app/scripts/run_peer.sh

This is how we’re starting the peers w/ docker

Okay, thanks.

If you’re not in docker, you can remove the stuff about calculating heap mem

Yeah will do.

Thanks for the help, appreciated.

@jasonbell: No prob, and thanks for writing up a post. That's great. ^^

https://dataissexy.wordpress.com/2016/07/17/from-onyx-template-to-working-app-in-thirty-seconds-onyx-clojure/

Good luck guys, that’ll auto tweet via @jasonbelldata as well.

Very nice

Pleasure. Will post more as time goes on I’m sure.

Something a bit more structured and with some depth.

Thanks @jasonbell. Ill tweet it out tomorrow morning the main account so it gets more viewers 🙂

I can’t say it’s quality but if you believe it to be helpful to someone then please feel free.

Probably 🙂