This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
- # ai (1)
- # beginners (190)
- # boot (24)
- # cider (43)
- # cljsjs (3)
- # cljsrn (29)
- # clojars (6)
- # clojure (310)
- # clojure-dev (6)
- # clojure-nl (6)
- # clojure-russia (11)
- # clojure-spec (66)
- # clojure-uk (95)
- # clojurescript (103)
- # clojurewerkz (2)
- # core-async (9)
- # cursive (4)
- # datomic (5)
- # hoplon (163)
- # lein-figwheel (52)
- # luminus (1)
- # off-topic (6)
- # om (6)
- # onyx (42)
- # perun (8)
- # re-frame (16)
- # reagent (10)
- # ring (7)
- # ring-swagger (1)
- # rum (1)
- # slack-help (2)
- # uncomplicate (1)
- # untangled (80)
Afternoon, I’ve just pushed a PR on the
onyx-twitter plugin for the tracking of tweets. Not sure what your process was in terms of versioning so I’ve just added -SNAPSHOT on my version until you’d seen it.
Hi, is any plan to use the new Datomic client library in the onyx-datomic plugin? does this move make sense to onyx rationale?
@nlessa Yeah, we should definitely upgrade to at least the newest version of the dependency we’re already using - no reason to stay behind. I didn’t get a chance to look at the new client library. Is it more or a less a mirror of what already existed with lighter footprint?
@jasonbell Thanks, looks good to me. Can you drop the SNAPSHOT bit and I’ll merge it? Our release infrastructure will bump the versions automatically.
@michaeldrogalis Yes, its like that .It seems that would be very useful to the scenario of several onyx peers using datomic as input/output. But I am not sure I am not missing some "but".
@nlessa No, just haven’t had time to bump it. If you exclude the dep and substitute your own, is it binary compatible?
I believe you still need a “peer server” operating in a different address space, which the Datomic client library connects to. The intent behind the client library is to provide a way to use datomic from short-lived processes (think AWS Lambda). Unless you’re scaling your Onyx peers very frequently, you’re likely to benefit from using the old Datomic peer library with it’s in-process cache.
Short and sweet but it does the job: https://dataissexy.wordpress.com/2016/12/28/filtering-the-streaming-twitter-api-with-onyx-plugin-clojure-twitter-data-firehose-onyx/
@jasonbell When you give the Twitter API a filter query, does it adjust its throughput to give you just as many tweets than if you didnt filter at all?
Well the flow predicate stuff means I can pipe out specific tracks into S3 bucket and so on.
Yeah, I was going to suggest that instead of using tracking, but if the throughput increases that’s better.
Just have to be careful with the number of tracks you do as there’s a limit on the URL length to the API, 200 chars I think.
What's considered the most idiomatic way to access output task results in onyx-local-rt considering there are no output plugins? Is it best to do something like (get-in env [:tasks :my-out-task :outputs]) or is it better to just do something like use the lifecycle to fit your use case (ex: inject a core.async chan to write output to)? Ideally, I'd want to push results rather than poll for them if I can avoid it.
get-in. Its just a data structure, so stick it in an atom and use a watch if you want to stream changes.
OK, I'm using get-in for now. I build my workflows dynamically so the extra complication is I then need to be sure to keep a map of which tasks are output tasks to grab their results accordingly. On the server, it's obviously much easier.
You have access to
(get-in env [:tasks …]), side-mapping shouldnt be needed if you dont mind recomputing that every time.
yeah that's why I'm keeping my own map because my task map can get big, so I don't want the overhead of recomputing
Sorry to bother, one more thing regarding onyx-local-rt. From the looks of the issues and scanning some of the code + branches on github, it looks like the triggers are broken for now? Is that true? Just realized I had a workflow with a window and timer trigger that's never getting called, but same workflow works in onyx.