Fork me on GitHub
#onyx
<
2016-10-24
>
michaeldrogalis00:10:30

@yonatanel Happy to adjust where we can if you think of a better way to phrase/place it.

michaeldrogalis15:10:00

@richiardiandrea @mfikes Replumb looks really good for an Onyx JS-Fiddle.. Which I suppose we need a name for . 🙂

michaeldrogalis15:10:33

So after playing around for a few hours, is my understanding correct that the library implementing cljs.js/eval is supposed to take care of *load-fn*?

mfikes15:10:50

@michaeldrogalis Yes. I’d recommend Replumb if you are looking for a conventional REPL experience and Klipse if you want live forms that are evaluated with their output placed in another place in the browser.

mfikes15:10:21

@michaeldrogalis Yes, but Replumb and Klipse both help implement *load-fn*

michaeldrogalis15:10:37

Klipse might do better, then. I had been taking care of live evaluation on my own. Even better if its off loaded.

richiardiandrea15:10:55

@michaeldrogalis thanks! mfikes is the master of names, we came up with Replumb together basically ;)

richiardiandrea15:10:19

Btw Klipse uses Replumb

mfikes15:10:12

Here is a good canonical example of what Klipse can look like: http://blog.klipse.tech/clojure/2016/03/30/destructuring.html

richiardiandrea15:10:13

I think viebel uses his own *load-fn* in there so that he can handle loading namspaces from GitHub

michaeldrogalis15:10:14

@richiardiandrea Nice. This doesn't seem like a hard task then, tbh.

michaeldrogalis15:10:35

@richiardiandrea Oh woah, how does that work?

michaeldrogalis15:10:59

@mfikes This seems right on the money. @lucasbradstreet Roughly what you were thinking too?

mfikes15:10:07

Yes… and he optimized the sh*t out of it so it can be fast

mfikes15:10:40

(In case, @michaeldrogalis you were considering going low-level for perf reasons, Klipse has strived to be fast.)

michaeldrogalis15:10:48

Brings a tear to my eye so I don't have to do that. 😛

richiardiandrea15:10:55

I am going to summon @viebel because I don't know the details, in theory it is a kind of a network get I guess

michaeldrogalis15:10:25

Hello @viebel! You've done some mighty cool stuff. 🙂

mfikes15:10:49

Here it is on github so you need not go digging for it: https://github.com/viebel/klipse

michaeldrogalis15:10:17

This is beautiful.

michaeldrogalis15:10:56

Yeah, as I was saying to @mfikes on Twitter, it'd be nice if we could lift everything in onyx-examples out of the repo so people can play with them without any set up.

Yehonathan Sharvit15:10:50

This is exactly the goal of klipse

michaeldrogalis15:10:31

Goal achieved, sir.

Yehonathan Sharvit15:10:51

Did you check codox klipse theme?

michaeldrogalis15:10:20

How have I never seen this before?! o_o

michaeldrogalis15:10:26

This is amazing!

Yehonathan Sharvit15:10:11

Do you have already a live demo snippet for onyx?

mfikes15:10:47

Just be aware that you can work around http://dev.clojure.org/jira/browse/CLJS-1831 by evaluating the two js* forms that are at the top of the picture in my tweet https://twitter.com/mfikes/status/789676348491071488

michaeldrogalis15:10:48

Yes. The dep below, plus the code snippet, work straight away in Cljs: https://github.com/onyx-platform/onyx-local-rt#usage

richiardiandrea15:10:51

Yeah it is cool stuff

michaeldrogalis15:10:23

@mfikes Cool, I had missed that the first time. I wasn't sure if that was to quiet the warning or something, the Jira post was helpful.

Yehonathan Sharvit16:10:55

@michaeldrogalis does the code work on klipse?

michaeldrogalis16:10:01

Since it requires an external dep, I assume not, but @richiardiandrea was mentioning that it can resolve things from GitHub?

mfikes16:10:48

@michaeldrogalis The js* forms are to work around a bug in order to get things to function; the warning is this one and can be turned off programmatically https://github.com/clojure/clojurescript/blob/f6a03a3d6eee64be417206d419b3017b09c5ddfd/src/main/clojure/cljs/analyzer.cljc#L385-L392

michaeldrogalis16:10:19

@mfikes Makes sense, this is super helpful

mfikes16:10:41

You can also apply my patch in the ticket, make your own local ClojureScript compiler, and work with that one.

michaeldrogalis16:10:42

Awesome, this all seems easy enough. Thanks to everyone's excellent work here to put together. 🙂

Yehonathan Sharvit16:10:53

@michaeldrogalis for external deps you need to use data-external-libs attribute as documented here https://github.com/viebel/klipse#clojure-only-1

michaeldrogalis16:10:47

Cool, going to need a little while before I can get it set up, but this seems to fit the bill nicely.

Yehonathan Sharvit16:10:52

See for instance how I’m using clojure.math.combinatorics in the source of this page http://blog.klipse.tech/clojure/2016/09/16/combinatorics-riddle.html

michaeldrogalis16:10:49

How exactly is the GitHub resolution working under the hood, out of curiosity?

Yehonathan Sharvit16:10:18

you provide an array of external-libs that are the url prefixes of your repository

Yehonathan Sharvit16:10:35

and my custom load-fn will look for the source files there

Yehonathan Sharvit16:10:37

for instance for onyx the external-libs will be:

, 

michaeldrogalis16:10:39

@viebel That's so cool. Really incredible work my friend.

Yehonathan Sharvit16:10:50

thank you so much!

Yehonathan Sharvit16:10:01

I can’t wait to se a demo of onyx with klipse

Yehonathan Sharvit16:10:31

I’m trying to make the code of the README work in klipse but there are issues

michaeldrogalis16:10:17

Close! The warning @mfikes mentioned is here: #error {:message "ERROR in file https://raw.githubusercontent.com/onyx-platform/onyx/0.9.x/src//onyx/static/default_vals.cljc", :data {:tag :cljs/analysis-error}, :cause #object[TypeError TypeError: Cannot read property 'default_vals' of undefined]}

michaeldrogalis16:10:23

I think we'll need to handle that on our own.

Yehonathan Sharvit16:10:05

So it can’t work for the moment?

michaeldrogalis16:10:57

I think either http://dev.clojure.org/jira/browse/CLJS-1831 needs to be resolved, or we sub in (js* "goog.provide('onyx.static$.default_vals');") and (js* "goog.provide('onyx.static$util');) somewhere.

michaeldrogalis16:10:24

Not Klipse's fault 🙂

Yehonathan Sharvit16:10:15

@mfikes how were you able to run onyx in Planck?

gardnervickers16:10:22

I think compilation munges onyx.static.default_vals to onyx.static$.default_vals because onyx.static.default-vals has a javascript reserved keyword in it. Not sure if that’s an issue here

mfikes16:10:54

@viebel See my tweet, it has all the details right in the picture: https://twitter.com/mfikes/status/789676348491071488

Yehonathan Sharvit16:10:38

Can you put the code here so I can copy/paste it?

Yehonathan Sharvit16:10:10

I see it jus above

mfikes16:10:18

@michaeldrogalis That error can be worked around with the js* forms (at least in Planck)

michaeldrogalis16:10:18

(js* "goog.provide('onyx.static$.default_vals');")
(js* "goog.provide('onyx.static$util');")

michaeldrogalis16:10:19

Things I never thought I'd be doing when I started a dist sys project - munging JS reserved words 😛

Yehonathan Sharvit16:10:16

Now, I’m encountering another error:

#error {:message "No such namespace: onyx.refinements, could not locate onyx/refinements.cljs, onyx/refinements.cljc, or Closure namespace \"onyx.refinements\"", :data {:tag :cljs/analysis-error}}

michaeldrogalis16:10:40

@viebel Does your tool let you specify the branch to resolve from? The work needed to get Onyx run on Cljs isnt in develop yet since I havent had enough reviewers, so its on https://github.com/onyx-platform/onyx/tree/cljc-triggers

michaeldrogalis16:10:03

If not, I can provisionally move it to master.

Yehonathan Sharvit16:10:15

OK will make external-libs pointing to cljc-triggers branch

michaeldrogalis16:10:33

Should do the trick. Thanks for helping us. 😃

michaeldrogalis16:10:00

Holy cow, it works

michaeldrogalis16:10:37

Soooo really all that's left to do is build up our examples to have Klipse links, and we're done.

Yehonathan Sharvit16:10:04

Not yet: For the moment, one has to run the following in the console

goog.provide('onyx.static$.util’);
goog.provide('onyx.static$.default_vals’);

Yehonathan Sharvit16:10:41

@mfikes do you know why prepending the js* stuff didn’t solve the problem?

Yehonathan Sharvit16:10:50

(js* "goog.provide('onyx.static$.default_vals');")
(js* "goog.provide('onyx.static$.util');")

(ns my.onyx
  (:require [onyx-local-rt.api :as api]))

Yehonathan Sharvit16:10:03

I had to run the goog.provide in the console!

Yehonathan Sharvit16:10:39

@michaeldrogalis instead of links to klipse repl, it would be much cooler to have a html page with embedded interactive code snippets

richiardiandrea16:10:50

I Love Open Source, good job guys

mfikes17:10:19

@viebel Is your goog.provide issue resolved?

michaeldrogalis17:10:39

@richiardiandrea Applause all around. 🙂

mfikes17:10:24

So, you can’t put forms prior to an ns form in a conventional namespace.

michaeldrogalis17:10:25

Ah, I didnt know that either.

Yehonathan Sharvit17:10:12

That’s not the problem

Yehonathan Sharvit17:10:42

I figured out that I can put forms prior to ns but the code will run only after the deps are resolved

Yehonathan Sharvit17:10:42

but that’s not a showstopper @michaeldrogalis

Yehonathan Sharvit17:10:34

In the html page with interactive snippets, you could add a

<script>
goog.provide('onyx.static$.util');
goog.provide('onyx.static$.default_vals’);
</script>

michaeldrogalis17:10:01

That's a fine solution. cc @colinhicks I think we have a few places where we can go with this.

michaeldrogalis17:10:13

Linking out to the repl might be a good first solution since its so easy.

Yehonathan Sharvit17:10:59

but how will you solve/workaround the goog.provide issue?

michaeldrogalis17:10:10

I suppose forking would be a suboptimal solution 🙂

michaeldrogalis17:10:38

Could do a hosted page with exactly one interactive snippet. 😛

Yehonathan Sharvit17:10:35

hosted page with one snippet - Sounds good to me!

michaeldrogalis17:10:18

Might take a shot at this tonight since its already close.

michaeldrogalis17:10:30

Will definitely ping everyone on Twitter when it's working. ^^

Yehonathan Sharvit17:10:27

It takes a bit of time to load - but it it is much less time (and effort) that cloning the repo and running it locally

michaeldrogalis17:10:07

Yeah, longer load time is normal for Onyx.

michaeldrogalis17:10:26

Can the output be pretty printed at the bottom?

Yehonathan Sharvit20:10:40

@michaeldrogalis working on pretty printing ...

michaeldrogalis20:10:13

@viebel Ah, okay. I didn't know if it was an out of the box option. I don't want to put you to work or anything, I can dig in on it later if you're busy.

Yehonathan Sharvit20:10:00

anyway it’s a good feature

Yehonathan Sharvit20:10:07

Happy to make it for you

colinhicks20:10:30

Hey, the Klipse approach is cool!

colinhicks20:10:46

I've been thinking: For the tutorials, it may not be necessary to compile the whole runtime in Klipse (or other browser-based REPL) ...

colinhicks20:10:52

That is, the runtime could already be compiled to JS, and available to the page. And we just ask the tutorial taker to define the job map, user-defined functions, etc.

michaeldrogalis20:10:19

Huh - yeah that's a solid idea @colinhicks.

Yehonathan Sharvit20:10:47

The problem is from a maintenance side

Yehonathan Sharvit20:10:08

every time you update onyx, you’ll have to remember to compile to js

michaeldrogalis20:10:52

We have a pretty extensive automated release process. Incidentally that piece isn't too bad for us.

michaeldrogalis20:10:03

Had to invest in a lot of infrastructure to manage all the subprojects of Onyx.

Yehonathan Sharvit20:10:42

You’ll also have to update the analysis cache files

Yehonathan Sharvit20:10:06

boostrapped cljs needs them for validating function calls etc...

Yehonathan Sharvit20:10:27

Pretty print is out!!!!

Yehonathan Sharvit20:10:53

You might need to empty your browser cache

Yehonathan Sharvit20:10:07

Make sure in the console that you get klipse version 3.3.0

michaeldrogalis20:10:41

Hmm, yeah that might not be so good then

michaeldrogalis20:10:08

Oh, the "empty cache" comment was unrelated to the above. Nevermind

michaeldrogalis20:10:31

Ahhhh beautiful!

michaeldrogalis20:10:39

@viebel gets today's gold star.

Yehonathan Sharvit20:10:23

I don’t find the appropriate emoji

Yehonathan Sharvit20:10:35

I just wanted to say: I’M TOUCHED!

Yehonathan Sharvit20:10:20

going to sleep now...

Yehonathan Sharvit20:10:34

It’s already 11:30 PM here

michaeldrogalis20:10:12

Sleep well, thanks for helping us. 🙂

michaeldrogalis20:10:00

Remove drain and stop and replacing with:

(api/tick)
    (api/tick)
    (api/tick)
    (api/tick)
    (api/tick)
    (api/tick)
    (api/tick)
    (api/tick)
    (api/tick)
    (api/tick)
    (api/tick)
Shows you show the segments move throughout the job.

michaeldrogalis20:10:13

Or whatever number of tick calls you want to see.

colinhicks20:10:56

which will be cool for a visualization

colinhicks20:10:52

it would be really cool to eventually have an interactive version of this https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-102

Yehonathan Sharvit20:10:49

Instead of the gist you could have an interactive snippet 😎

shaun-mahood21:10:40

So now that Onyx has the local runtime library, would it be reasonable to learn and use it for programs that would otherwise have no real use for it (for example, in a single page app or a program that will only ever run on 1 server)? I have no real need for it, so the amount of upfront configuration and learning has put me off learning it until now.

gardnervickers21:10:54

It's a wonderful programming model if you want a way to structure your computation to be easily scaled across multiple machines in the future.

shaun-mahood21:10:33

@gardnervickers: Are there any substantial benefits outside of scalability that you've found with it in use, or any types of problems that Onyx is really well suited for even if there is no possibility of ever scaling?

gardnervickers21:10:11

It forces you to think about where state lives inside your computation. That's really the big benefit. By acknowledging where state exists and is changed in your computation, distributing it becomes much much easier. So much so, Onyx can do it for you 😄

gardnervickers21:10:15

The model Onyx requires is a pretty good general purpose one, though weighing a different model versus Onyx's is highly dependent on what you're trying to do.

shaun-mahood21:10:41

@gardnervickers: Yeah, it seems very well thought out when looking at it from the outside - I'll probably try it at some point once more experienced users have had some more time to use the local runtime. Thanks for the help!

michaeldrogalis22:10:52

@shaun-mahood That is the idea, yes. Decouple the programming model from its runtime was a goal. It's good for any "flow"-type domain

shaun-mahood22:10:58

@michaeldrogalis: Good to know, thanks - I think I have a couple areas in my programs where it could clean things up. The talks have all been super interesting so it's great that the barrier to entry is so much lower.

michaeldrogalis23:10:50

@shaun-mahood We're quite pleased with that too. Enjoy the programming model. 🙂