I have opened two threads regarding jupyter kernels: one asking about the handling of many projects with the same kernel and the other regarding the best way to distribute the kernels. Personally, I do not believe there is a "nice" way to handle many projects with the same kernel; we will need to abuse environment variables here. Tomorrow I'll try to hack my way with environment variables and bash scripts. https://groups.google.com/forum/#!topic/jupyter/X7Fhs0C9vLs https://groups.google.com/forum/#!topic/jupyter/PvcNxwAcbLw

thanks @didiercrunch - I especially appreciate the documentation and information sharing with the group 😸

For those interested, CUDA GPU implementation of 50 recently introduced vectorized math functions has just landed into Neanderthal. Some are thousands of times faster than java.Math equivalents. https://github.com/uncomplicate/neanderthal/blob/master/src/clojure/uncomplicate/neanderthal/vect_math.clj

in the news: https://eng.uber.com/horovod/

Why is it always 'easy', and never simple?🙂

@blueberry you are a rock start! All your libraries are amazing. Continue like that.

@didiercrunch yes, thanks for working on clojupyter. I'm still digging into the code, and understanding the jupyter protocol, how graphics are rendered, and especially interactive graphics are accomplished. I'm keen to integrate clojurescript in the client somehow, but Jupyter supports multiple clients (possibly simultaneously on the same kernel) and some of them probably won't support javascript or even a DOM. Those are the questions I'm looking to answer first.

@pragsmike this is probably what you are looking for: https://github.com/roryk/clojupyter/blob/master/src/clojupyter/misc/display.clj

it is used as a middleware.

If the return value of a cell is a specific "record" then clojupyter send a message that is understood by jupyternotebook as a html/image blob

(latex too actually)

If you can use http://plot.ly, I suggest you to use.

For what I understand of http://plot.ly, you give it data and it returns you a url that represent the chart you want to plot. You can embed it in the jupyter notebook as html

I imagine it uses iframe for interactive charts

As for adding javascript in the jupyter notebook, that will be needed to use parinfer

I will again make a suggestion as to where you might better expend the cycles you have to spend. I think a more fruitful approach is to start with the reagent version of gorilla (that will give you a state of the art SPA starting point). Replace the plotting and charting stuff with Vega-Lite - which is 1. totally data driven, 2) written by the same people who did Vega (the Interactive Data Lab at University of Washington) and they clearly know what they are doing and have a lot of resources behind this, 3) is simple, declarative and you can use nothing more than CLJ/CLJS maps to drive it, 4) is already packaged on CLJSJS, 5) would fit nicely into gorilla from what I recall from hacking on gorilla plotting before. Then look into how Proto-Repl would fit into the client side (it is all JS, so should in theory work). Keep all the html, LaTex, Image, etc bits of gorilla.

Proto-Repl would give a real editor. The result could be one of the most potent notebook type things available. Of course this would not be python oriented or directly usable by python so, if that is a goal this might not be what you want.

The thing about Vega-Lite that is particularly apropos here, is that it is designed specifically for data science and statistical dynamic visualization and interaction

It is indeed a 'dumb'/misleading name, but it really is basically what is needed

vega-lite looks very good

Having been using it now for about a month, I can say it is extremely good and only going to get better based on comments from that group

You can even take their examples, run them through json->clj and use the resulting map to send back to client for display/testing/hacking. It just works

nevertheless, I like better jupyter than gorilla

It is more professional.

Sure, gorilla is no where near as developed. The idea above would be to make it really impressive

moreover now there is a jupyterlab project that is jupyter on steroid

@jsa-aerial I'll take your word for the choice of VL. What if we could use it in Jupyter client?

I am sure we could workout a way to make vegas work on jupyter

You need Jupyter client to work with JS

Yup

If you can do that, then sure, you can have VL in Jupyter.

that'd take some hacking, but also consensus in the jupyter community

But, basing something in a true reactive driven client, would yield all sorts of extra benefits.

I was hoping it would be straightforward to add some sort of rendering hook for a mime type in the jupyter client, where we could just run arbitrary javascript on a div

maybe it is in the new jupyterlab notebook

my knowledge of how jupyter works is near zero as yet

https://github.com/jupyterlab/jupyterlab

I propose to hack our way around with the current jupyter notebook architecture. By that I mean having an easy installation support for the kernel and some http://plot.ly capabilities. It cost nothing and we will learn things. Then, we will be able to make an informe decision to where to go. I believe it is important to leverage existent code and to deliver results rapidly

anyway @jsa-aerial you've answered one of my questions. now we just have to figure out how to carve out a space for it to live. I'm going to need to know how to work Jupyter for my job now anyway. Other folks at my company use Jupyter, and gorilla would not be well-received there.

I think it's likely that any work done to integrate VL in gorilla would easily drop-in to Jupyter, if we can carve out a space for JS to run in it.

Well, if you have to use Jupyter, you have to use Jupyter.

The BeakerX project (https://github.com/twosigma/beakerx) as similar goals as you are discussing, if nothing else may serve as a good JVM-based example. They have a pretty decent visualization library in Java and a working Clojure kernel, as well as the beginnings of a way to move data between kernels.

How about two parallel efforts: one to make gorilla awesome, and one to carve out a space inside Jupyter client where some of that awesomeness can run. A complication is that there isn't just one "jupyter client", but I'm willing to let the others get not-so-awesome rendering (like static PNGs) where the more capable one would get interactivity.

@dave.dixon thanks for that!

The issue with that is we end up splitting resources and getting less for more...

I wish I had the cycles, I really think the reagent gorilla + VL + Prot-Repl would be quite amazing

"Make gorilla awesome" is attractive. On the other hand, it may be duplicating a lot of work. I'd love to see a Jupyter kernel that 1) leveraged nREPL a la gorilla, and 2) achieved the BeakerX vision of sharing data between kernels, thus having cake and eating it too. Some of the Python viz libs in particular are fantastic and interactive.

Since you would be leveraging a lot of stuff already done, I'm not sure about the duplication. And VL is actually rather unique. I get the feeling people really haven't explored what it can do.

There are at least a couple of Jupyter-based plugins for atom. I gave a brief (unsuccessful) try to these when I was playing with BeakerX: https://atom.io/packages/jupyter-notebook and https://atom.io/packages/hydrogen

Another interesting project, sort of a "meta-kernel" that allows data sharing: https://vatlab.github.io/SoS/

ok, now I have to go learn about BeakerX. @jsa-aerial I hear you about splitting effort, but reaching the Jupyter community has a huge payoff.

Possibly - but don't expect that to make python folks want to try clojure. I thought the idea was to focus on clojurists first to 'keep' them from going elsewhere because they had no choice...