This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-12-05
Channels
- # adventofcode (41)
- # bangalore-clj (4)
- # beginners (283)
- # boot (62)
- # clara (9)
- # cljsrn (3)
- # clojure (112)
- # clojure-brasil (1)
- # clojure-greece (1)
- # clojure-korea (6)
- # clojure-russia (99)
- # clojure-spec (29)
- # clojure-uk (12)
- # clojurescript (34)
- # clojurex (5)
- # core-logic (1)
- # cursive (31)
- # datomic (30)
- # devcards (5)
- # editors (19)
- # emacs (31)
- # events (5)
- # garden (4)
- # hoplon (137)
- # lein-figwheel (3)
- # luminus (4)
- # mount (7)
- # off-topic (7)
- # om (18)
- # om-next (3)
- # onyx (88)
- # proton (1)
- # protorepl (6)
- # re-frame (48)
- # reagent (15)
- # spacemacs (41)
- # testing (1)
- # untangled (2)
- # yada (18)
I want to submit an Onyx talk to Devoxx this year but I need to do a bit more collaboration with @jasonbell to build my experience. Current role doesn't relaly have a use case for Onyx unfortunately!
Is the onyx-local-rt runtime itself (whatever is returned from (api/init job)
etc) data only and can be in Om versioned state? (it means it will have time travel)
Hey guys, we are getting this error all of a sudden but not really sure what is causing this?
Exception in thread "main" clojure.lang.ExceptionInfo: empty String {:original-exception :java.lang.NumberFormatException}
at onyx.compression.nippy$fn__10918$fn__10919.invoke(nippy.clj:33)
at taoensso.nippy$read_custom_BANG_.invokeStatic(nippy.clj:1052)
at taoensso.nippy$read_custom_BANG_.invoke(nippy.clj:1049)
at taoensso.nippy$thaw_from_in_BANG_.invokeStatic(nippy.clj:1218)
at taoensso.nippy$thaw_from_in_BANG_.invoke(nippy.clj:1098)
at taoensso.nippy$thaw$thaw_data__10761.invoke(nippy.clj:1330)
at taoensso.nippy$thaw.invokeStatic(nippy.clj:1356)
at taoensso.nippy$thaw.invoke(nippy.clj:1279)
at onyx.compression.nippy$zookeeper_decompress.invokeStatic(nippy.clj:56)
at onyx.compression.nippy$zookeeper_decompress.invoke(nippy.clj:55)
at onyx.log.zookeeper$fn__16795$fn__16797$fn__16798.invoke(zookeeper.clj:564)
at onyx.log.zookeeper$clean_up_broken_connections.invokeStatic(zookeeper.clj:77)
at onyx.log.zookeeper$clean_up_broken_connections.invoke(zookeeper.clj:75)
at onyx.log.zookeeper$fn__16795$fn__16797.invoke(zookeeper.clj:561)
at onyx.monitoring.measurements$measure_latency.invokeStatic(measurements.clj:11)
at onyx.monitoring.measurements$measure_latency.invoke(measurements.clj:5)
at onyx.log.zookeeper$fn__16795.invokeStatic(zookeeper.clj:560)
at onyx.log.zookeeper$fn__16795.doInvoke(zookeeper.clj:558)
at clojure.lang.RestFn.invoke(RestFn.java:445)
at clojure.lang.MultiFn.invoke(MultiFn.java:238)
at onyx.test_helper$feedback_exception_BANG_.invokeStatic(test_helper.clj:24)
at onyx.test_helper$feedback_exception_BANG_.invoke(test_helper.clj:13)
at onyx.test_helper$feedback_exception_BANG_.invokeStatic(test_helper.clj:19)
at onyx.test_helper$feedback_exception_BANG_.invoke(test_helper.clj:13)
@camechis Is this happening in your peer or job submit process?
That’s an exception from your job being deserialized from Zookeeper into the job submit process.
there is an uncaught exception in our logs in a lifecycle that bubbled up. Just was buried in the logs so we missed it before
So far I've made a simple om.next app with onyx-local-rt, hard-coded job, textarea with env-summary, and buttons for commands and time travel: Reset, <-Revert, Tick->, Drain and Stop. Does that interest anyone? It's just the beginning and instead of the sample job from onyx-local-rt readme I'm gonna implement my own aggregations.
@yonatanel Almost! It has a few compiled functions inside of it.
@yonatanel That’s intriguing. 🙂 Curious to get a look when you’re ready, sounds brilliant.
@michaeldrogalis It works anyway. I don't actually need this to be data only, but at least not shared among instances
@yonatanel That’s super cool.
Heh. Is it a demo for them or something?
For instance if we don't understand why some data is the way it is, if command sourcing is in place I could just step through them.
Awesome. Definitely interesting in seeing it if/when you want to put it out there.
Killer. 🙂
That's not fun if you're inexperienced in cljs: referencing :cljs.core/identity
instead of clojure.core/identity
@yonatanel I actually didn’t know there was a difference. cljs == JS identity?
frankly i have no idea. I just needed an identity task to do aggregation on original segments
I think it behaves the same as clojure, but the fully qualified name is different and that's what onyx wants
Huh. I’ll keep my eyes out for it sometime.
Is there a well defined way for pulling data from APIs via HTTP into Onyx? I see an onyx-http
output plugin. What's the preferred way to do something similar for gathering input?
@stephenmhopper For the most part, Onyx is optimal for input streams that are durable and can be replayed verbatim. What’s the nature of the API you’re working with? Is it a REST API? Is there a sequence of calls you want to make across it?
That makes sense
I'll be polling endpoints periodically and want to process any new events from those endpoints
Sometimes it’s a good idea to stream in a sequence of API call descriptions through Kafka, and use a downstream tasks to invoke the actual call.
So I'm thinking of storing them to an intermediate storage location first like a DB of sorts
Kafka might work too
Sure, yeah that works just as well. You get the idea.
@yonatanel It should never be, no. You see it happening?
Yes. I have this:
(defn ^:export dump-aggregate [event window trigger opts state]
(println "dump-aggregate")
(println "state")
(pprint state)
(println "event")
(pprint event))
;...
:triggers
[{:trigger/window-id :
:trigger/on :onyx.triggers/segment
:trigger/threshold [1 :elements]
:trigger/refinement :onyx.refinements/accumulating
:trigger/sync ::dump-aggregate}]
And event is printed as nil all the timeHum. That’s a bug for sure.
Heh, thanks
Obviously I know nothing about this. Do you know how I get rid of this error?
Compiling "resources/public/js/compiled/onyx_cqrs_example.js" from ["src"]...
Failed to compile "resources/public/js/compiled/onyx_cqrs_example.js" in 2.047 seconds.
---- Could not Analyze resources/public/js/compiled/out/onyx/spec.cljc ----
No such namespace: clojure.future, could not locate clojure/future.cljs, clojure/future.cljc, or Closure namespace "clojure.future"
Oh, I'm using figwheel and onyx-local-rt probably isn't configured for that or clojurescript build. Maybe I can learn from onyx-blueprint
@yonatanel Bring in the dependency [clojure-future-spec “1.9.0-alpha14”]
. We flipped this to a dev dependency in core last release since we can’t use Clojure-1.9-alpha yet
Sorry, we forgot to document that I think
I’m itching to move the whole local-rt into core, but I want to see 1.9 come out properly first.
Could we do a manual release of onyx local rt? I think that'd solve the issue
It's on main line onyx now
Ill cut a release now
Odd, that dependency is already in local-rt
I seem to remember about being able to do Clojure version reader conditions just like we can do cljc vs clj
Any idea of that ever made it in?
It is but did we do a release?
I got in on circle and building but didn't add it to the release process
Ah. Yeah Ill cut it now
If i'm not mistaken, the wanted dependency should come earlier in the list, before onyx.
By list, you mean in :dependencies
?
I don't think that matter
Yeah I dont believe it does.
Either in deps or in require
[org.onyxplatform/onyx-local-rt “0.9.15.0”]
is out
I'm getting a nasty "Unable to resolve symbol: import in this context, compiling:(clojure/core_instant18.clj:11:1)". Not sure why though
@yonatanel, check out what I did in onyx-blueprint: https://github.com/colinhicks/onyx-blueprint/blob/master/src/clojure/future.cljs
this defines the clojure.future ns for ClojuresScript, only. The clojure-future-spec dep is Clojure-only.
@yonatanel As a result of upgrading?
Ahh, indeed. Right @colinhicks
Another reason I’m chomping at the bit for 1.9 😕
Yep. Over in CLJS land, spec is implemented in the latest, non-alpha builds. Makes things much nicer.
Perhaps, but also due to using latest clojure and cljs and om. Happens with this:
:dependencies [[org.clojure/clojure "1.9.0-alpha14"]
[org.clojure/clojurescript "1.9.293"]
[org.omcljs/om "1.0.0-alpha47"]
[org.onyxplatform/onyx-local-rt "0.9.15.0"]]
Doesn't happen with these defaults from lein new figwheel:
:dependencies [[org.clojure/clojure "1.8.0"]
[org.clojure/clojurescript "1.9.229"]
[org.clojure/core.async "0.2.391"
:exclusions [org.clojure/tools.reader]]
[org.onyxplatform/onyx-local-rt "0.9.15.0"]]
I think your latest error is a conflict between clojure 1.9.0-alpha14 and the clojure-future-spec dep in onyx-local-rt
@yonatanel You should be able to exclude clojure-future-spec and be okay
Talk about a dependency cage match
Yeah. And you might also have to stub-in the clojure.future ns as shown in my link above.
Yes, because now I get
Failed to compile "resources/public/js/compiled/onyx_cqrs_example.js" in 3.409 seconds.
---- Could not Analyze resources/public/js/compiled/out/onyx/spec.cljc ----
No such namespace: clojure.future, could not locate clojure/future.cljs, clojure/future.cljc, or Closure namespace "clojure.future"
try adding that in src/clojure/future.cljs
Bit of a crappy circular dependency problem that will go away when 1.9 is caught up on both sides of the clj and cljs house.
If this works, I think I can PR a change to local-rt that will avoid this future monkey dance
We seem to have corralled an adventurous bunch of devs over the years. ❤️
Ohhhh yeah. That’s what I’m talkin’ about. 😄