Clojurians Log v2

woohoo!

let me stare at it shortly

that would come from taking the derivative of a function with function arguments

Hmm.. i just do trial and error, so the minimum code that trigger this error is (((cube D) tanh) identity), is it?

And for example, replacing ((cube D) tanh) with e.g. sin won't trigger the error and show this graph 🙂

Not sure if this is scittle (sci) specific, but if ((cube D) tanh) works on clerk, there could be some bug?

very interesting, let me see if I messed this up in emmy

yep, looks like I missed this test case on my shift to reverse-mode AD

@reedho try it with v0.31.0 instead of the latest v0.32.0

try this instead

(render-to-app
 (fn []
   (->f (ev/with-let [!phase [0 0]]
          (let [shifted (emmy.viewer/with-params {:atom !phase :params [0]}
                          (fn [shift]
                            (fn [x]
                              (((cube D) tanh) (- x shift)))))]
            (emmy.mafs/mafs
             {:height 400}
             (emmy.mafs/cartesian)
             (emmy.mafs/of-x shifted)
             ))))))

the first example was trying to be clever with the generics

@reedho okay, I see what I did… looks like this worked in 0.31.0, but I tightened up the type bounds which stopped functions from coming in… for now use the snippet I had from above, and I will create an issue in Emmy

https://github.com/mentat-collective/emmy/issues/189

more tests needed but it’ll be an easy fix, I think: https://github.com/mentat-collective/emmy/pull/190

@reedho there’s no clerk dependency: https://github.com/mentat-collective/emmy-viewers/blob/main/deps.edn#L3-L10

(clerk-utils doesn’t have a clerk dependency)

the “viewers” code generates reagent fragments, and all you have to do is eval them with sci to get them to appear in the browser

here’s the portal viewer for example: https://github.com/mentat-collective/emmy-viewers/blob/5afd6de120391c668e8573204321ebe4442746dc/src/emmy/portal/reagent.cljs#L34-L67

and here’s emmy-viewers working in Maria via SCI: https://2.maria.cloud/gist/0405c3427c88326a181b307371f939bc

Oh that is very detailed hints, thank you so much, i will follow up soon.

But it turns out that in the example above, the Reagent component is actually not interactive. Makes sense.. The following does works:

I will add some functionality to recognize when things are of the emmy-viewers kind, and handle them correctly without bothering the user about that.

Now emmy-viewers are automatically inferred as a dedicated kind of visualization (`:kind/emmy-viewers`), and then handled accordingly, so that the user does not need to bother about the wrapping fn. https://scicloj.github.io/clay/clay_book.examples.html#emmy-viewers

that’s the way!!

final step @daslu is making sure we support the expansion… so, does

(mafs/of-x e/sin)

we should be able to get this working without the quote https://scicloj.github.io/clay/clay_book.examples.html#emmy-viewers

do you mean we should hide it by making it implicit? I think you're right. I'll have to think a bit about how to make things consistent across the two different situations, where the expression is either generated in the JVM or in the browser.

those forms inside the quote generate a reagent form, which we can then eval inside the viewer

running the forms that generate the quote should be totally fine outside of a quote

I bet it’s very close to working that way now

do you mean this quote?

This one is there because it is a JVM Clojure notebook, where we run some Clojurescript code (which calls emmy and emmy viewers, which generate those expressions). We need a way to express Clojurescript code in Clojure, and we do that using quoting. https://scicloj.github.io/clay/clay_book.examples.html#reagent

The second pathway would be running the emmy viewers code in the JVM Clojure. Then we will already get the relevant Clojure form, and will just need to pass it to the browser to be visualized. I'll look for a way to wrap this nicely.

yes

that’s what you should do

when you run that code on the JVM it will return a quoted form (which can be serialized to cljs and then eval-ed)

this is why the emmy-viewers code is cljc

yes. thanks! and then all that eval ceremony is not needed either

the way you could figured it out as a cljc API is so nice!

the eval is needed on the client

(since you have sent over a serialized reagent form)

and I used eval when it was all SCI as well

since running the emmy-viewers forms on the client side produces a quoted form as well… so, just pretend that that came over the wire instead of from the cell 🙂

exactly 🙂 🙏

@sritchie09 you were right! Here is how things look when using Emmy-Viewers in the JVM side. https://scicloj.github.io/clay/clay_book.examples.html#emmy-viewers

fabulous

Clay now takes care of the expand transparently, and it works nicely for the (mafs/of-x e/sin) example as @reedho demonstrated above. https://scicloj.github.io/clay/clay_book.emmy_viewers.html#usage There are additional problems to explore, and I will share them here gradually.

By the way, @reedho's work was used for a brief moment in the talk of Thomas Clark at the Conj last Friday. 🎉

Trying the Function Viewer tutorial:

(require '[emmy.env :as e :refer :all]
         '[emmy.mathbox.plot :as plot])

(defn my-fn [x]
  (+ -1
     (square (sin x))
     (square (cos (* 2 x)))))

(plot/of-x {:z my-fn :samples 256})

TypeError: this.options.klass is not a constructor
    at h.install (emmy-viewers15.js:1303:253)
    at m.__install (emmy-viewers15.js:1277:369)
    at emmy-viewers15.js:1276:482
    at u (emmy-viewers15.js:1273:229)
    at m.install (emmy-viewers15.js:1276:470)
    at m.init (emmy-viewers15.js:1275:298)
    at new m (emmy-viewers15.js:1275:246)
    at g.mathBox (emmy-viewers15.js:2730:129)
    at emmy-viewers15.js:2739:208
    at Id (reagent9.js:165:137)

yeah and maybe try with no/ simple optimizations, or with source maps

(but that won’t work until you get expand working)

since (plot/of-x {:z my-fn :samples 256}) won’t work “bare”

it needs a wrapping mathbox context, which it • carries around in a metadata slot for expand, • or you can wrap it up yourself

I did use expand (implicitly done by Clay now).

(defn expand
  "If `v` has a `fn?` (say, `f`) registered as Clerk viewer metadata, recurses
  with the expanded value `(f v)`. Else, returns `v` unchanged."
  [v]
  (let [xform (-> v meta :nextjournal.clerk/viewer)]
    (if (fn? xform)
      (expand (xform v))
      v)))

okay cool, then yeah a repro would be great!

Thanks!

Oh well, math stuffs i dont have enough knowledge, i will build scittle plugin with v 0.31 later to continue working out more examples. Thank you so much.

Hi. I rebuilt it now with v0.31.0 and tried the above:

(fn []
   [->f (ev/with-let [!phase [0 0]]
    (let [shifted (emmy.viewer/with-params {:atom !phase :params [0]}
                    (fn [shift]
                      (fn [x]
                        (((e/cube e/D) e/tanh) (- x shift)))))]
      (emmy.mafs/mafs
       {:height 400}
       (emmy.mafs/cartesian)
       (emmy.mafs/of-x shifted)
       (emmy.mafs/movable-point
        {:atom !phase :constrain "horizontal"})
       (emmy.mafs/inequality
        {:y {:<= shifted :> e/cos} :color :blue}))))])

Thanks for the thorough debugging, @reedho & @sritchie09.

@reedho here is a Pull Request if you wish: https://github.com/reedho/scittle/pull/2

Thank you so much

@daslu, I believe the part removed strange characters on the plugin build is not necessary. Instead of remove the strange (unicode) chars from js build output we put <meta charset="utf-8"> in host html. I've just follow up with this comment on the PR.

Oh, you are right! I removed that part now.

PR now merged. 👍

hey all, pinging on this

hey Sam, thanks! on Saturday I should be finally free of some tasks and be able to read and learn and experiment

Hi, I have been trying related things to figure out how to make skittle run emmy viewer. It seems obvious that this can be done based on what has been done in clerk-utils. Is this the right way? @sritchie09

Manually call expand is work as expected like this:

(render-to-app
 (->f (emmy.viewer/with-let [!phase [0 0]]
        (let [shifted (emmy.viewer/with-params {:atom !phase :params [0]}
                        (fn [shift]
                          (((cube D) tanh) (- identity shift))))]
          (ev/expand
           (emmy.mafs/of-x shifted))
          ))))

But it won't without expand. If i remember, expansion was called automatically on clerk?

Yes. Probably we can learn from Maria as Sam has proposed: > https://github.com/mhuebert/maria/blob/988168c/editor2/src/main/maria/editor/extensions/emmy.cljs Personally I haven't got to learn precisely what expand does. I can look into it, probably not today.

iirc if there is a specific metadata entry on the form holding an expansion function, you call that on the form before you pass it to the viewer

I’m also interested in getting a custom build of scittle up and running with datascript. I saw that this was already one of the plug-ins provided, and I was able to get it working within the main scittle repo by tweaking the bb.edn file in plugins/demo. What I’d like to do is just grab the js files generated from the custom build and have those locally in my project and include them using script tags, but my attempts at getting this working with a basic bb.edn (using http/serve to set up the assets + nrepl to get a repl port going) haven’t succeeded. Is there any special setup you do when putting js artifacts on the cdn?

@cantwell.nc I don't do anything special, just publish to npm and then access via CDN. There is a project called scittle-hoplon which is published to npm: https://www.jsdelivr.com/package/npm/scittle-hoplon perhap this helps you

I always have trouble finding the original github repo (cc @mynomoto)

oh duh, it's even in this conversation. @mynomoto told me that he just built scittle using the original repo

Not totally sure what I was doing wrong in the first place (maybe mixing js files, like @mynomoto mentioned?), but I got this working! here's a repo with the steps I took in case anyone else would like to do this: https://github.com/cantwellnc/data-scittle

congrats

the install! function here should be all you need to copy any existing scittle examples; https://github.com/mentat-collective/emmy-viewers/blob/main/src/emmy/viewer/sci.cljs#L35

All right, i will try some testing. Thanks.

For now i have created a working emmy plugin in here: https://github.com/reedho/scittle/tree/my_playground/plugins/emmy

@reedho nice! emmy-viewers should be basically the same, yeah?

I thought so, but i still confused that if EV will also bring "clerk" which is not desirable in my opinion for now

Probably better i should just do trial and error build later today

cc @clark.thomaswilliam (This is the topic thread where @reedho is doing the incredible work of bringing Emmy Viewers to more environments. Probably we should start a new thread soon where we can share usage examples, etc.)

@daslu do you all want to pair quickly?

I feel like we can get this working pretty fast

Hi! I'll check 🙏

here’s the portal viewer for example: https://github.com/mentat-collective/emmy-viewers/blob/5afd6de120391c668e8573204321ebe4442746dc/src/emmy/portal/reagent.cljs#L34-L67

(rpc/call 'emmy.viewer/expand v)

After some brutal tinkering, i found a way, we can show some graph now

https://github.com/reedho/scittle/blob/my_playground/plugins/demo_emmy/resources/public/index.html#L22-L62 Not sure if this is actually correct 🙂

Wonderful, works for me now too. So looking into the way Portal handles values (linked above), some parts of the process were unnecessary in this case (`expand`, etc). Right?

> the install! function here should be all you need to copy any existing scittle examples; https://github.com/mentat-collective/emmy-viewers/blob/main/src/emmy/viewer/sci.cljs#L35 As this instruction, the poc of EV scittle plugin implementation here: https://github.com/reedho/scittle/blob/my_playground/plugins/emmy_viewers/src/scittle/emmy_viewers.cljs#L40-L69

what's the removing of metadata about?

Any thing i do is just trial and error until its work 🙂

what error was reported when you didn't do this?

I guest expand is needed somehow if the graphs have inner graph or some thing like that

> what error was reported when you didn't do this? Wait, let me just check it

It still works without error

Some notes i collect while building the ev plugins: Error if requiring emmy.viwer.sci

The required namespace "nextjournal.clerk.viewer" is not available, it was required by "emmy/clerk.cljc".

The required JS dependency "process" is not available, it was required by "node_modules/jsxgraph/distrib/jsxgraphcore.js".

Dependency Trace:
        scittle/emmy_viewers.cljs
        jsxgraph/sci.cljs
        jsxgraph/core.cljs
        node_modules/jsxgraph/distrib/jsxgraphcore.js

Also the production build output of scittle.emmy-viewers.js has invalid regex error like so:

Uncaught SyntaxError: Invalid regular expression: /[̀-ͯ]+$/: Range out of order in character class
    at RegExp (<anonymous>)
    at scittle.emmy-viewers.js:3234:319
    at scittle.emmy-viewers.js:3313:326
    at scittle.emmy-viewers.js:2793:109
    at shadow$provide.<computed> (scittle.emmy-viewers.js:2793:210)
    at Ur (scittle.js:770:224)
    at $ (scittle.js:770:458)
    at scittle.emmy-viewers.js:5267:352
    at scittle.emmy-viewers.js:5592:4

I fixed the out js manually to trial and error the index.html demo

@borkdude metadata removal was because sub-pieces have rendering functions attached to them as metadata

and sci tried to serialize that and failed, so I stripped it all so only the data structures remained

@borkdude this is so if you provided, say, a bare plot without a wrapping scene it could supply its own wrapping scene

will write more soon

expand is necessary if you want to show, e.g., (mafs/of-x {:y sin})

that doesn’t actually make sense alone, so expand wraps it in the components that make it all work

thanks. and if I understand correctly, in the current implementation this requires a connected backend to work -- is that right?

no, that was just a hack because portal didn’t have expand available on its side in SCI

but it did have rpc calls

(maria works with no connected backend @daslu)

makes sense, thanks

the constraint in portal (this is coming back now!) is that portal couldn’t compile a fresh SCI bundle here, but COULD eval some files with sci

(not sure whether I understand the portal constraint, but maybe it is not so important to understand at the moment. what I remember is that standalone portal, with no backend, does not have the ability to add js dependencies.)

so probably a reasonable plan for scittle-based static HTML would be in two stages: • support thins that do not require expand, • learn from https://www.maria.cloud/ how to cover the cases who do need expand Looks like the relevant Maria code is here: https://github.com/mhuebert/maria/blob/988168c/editor2/src/main/maria/editor/extensions/emmy.cljs

only this part

(when-let [m (meta x)]
        (cond (= emmy.mafs/default-viewer (:nextjournal.clerk/viewer m))
              (show/reagent-eval opts (emmy.viewer/expand x))

              (:portal.viewer/reagent? m)
              (show/reagent-eval opts x))))))

that will handle all of the emmy-viewers stuff

nice, thanks

@daslu every emmy-viewers form is a block of sci-able reagent code, so the first step is to just print the data structure

then get expand working

then get eval working

in what cases will eval be needed?

@daslu I think if you try to get the data structure to print it will be much more clear how this is all wired together

for example - if I have a function that produces ['some.ns/SomeComponent {:x 0 :y 10}] what do you do with that?

you have to call eval, right?

if emmy-viewers was client side ONLY then I could have built it so that it doesn’t include the quote above, and then eval would not be necessary

I see, right, thanks.

but because it’s cljc I have to include the quote, and then • a clj consumer builds the quoted thing, sends it over the wire, and the client calls eval • but if you’re in maria or scittle and fully client side, well, we just call eval right away since we’re already on the client

we just call eval once over the whole code, right?

this is just what scittle does on any code.

or do you need to eval some subforms in some cases?

I think you may be right…

the inner eval might be required to strip off the quotes

but I don’t have a good sense, you might be right

thanks. anyway, we can print the forms as you suggested and see what make sense.

if not you’ll see the extra quotes and then boom, the solution will be obvious

okay! keep sharing, I can help bump this along (especially if you share code)

> One way to do it is creating a fork of Scittle There is a better option, create a scittle plugin and then re-compile scittle with that plugin enabled: https://github.com/babashka/scittle/tree/main/plugins/demo

Thanks! 👀

@mynomoto has created the scittle hoplon project, and is using this setup

let me look if I can find the source

at least the build is available here: https://www.jsdelivr.com/package/npm/scittle-hoplon

maybe it's not public. but at least this is how you can do it too.

The hoplon plugin is on scittle itself: https://github.com/babashka/scittle/tree/main/plugins/hoplon We use it on the playground: https://hoplon.io/playground.html

oh yes, of course :)

do you have the code where you build it?

in CI or so

I only build it once so it was on my machine. I think I changed the scittle build itself to add the hoplon plugin. One thing to notice is that you cannot combine js generated from different builds, you need to use all the outputs from the same build.

Maybe we can change the scripts a bit to include plugins via an env var or so

anyway, let me know if you have any questions about this @daslu

That is very helpful, many thanks @borkdude & @mynomoto.

this is fantastic!!!

@daslu I’ve been making lots of maria sketches to help folks with emmy-viewers, it’s been great: https://2.maria.cloud/gist/9376a3d9e0408de7d5cdb8d5dddbd37b

Yes, I saw! Fantastic

@daslu if you end up getting some code up and share a link I can send a PR too

Sure, great 🙏

Hello all, sorry its been a month btw. I have just able to test and experiment with this emmy viewer plugin implementation: https://github.com/reedho/scittle There is note on top of the readme where to look the plugin code. Simple mafs and mathbox can be rendered as expected, i put some example online here https://ev-demo.datafy.id/

While continue working with more complex example like shown in my snippet below:

(render-to-app
 (fn []
   (->f (ev/with-let [!phase [0 0]]
          (let [shifted (emmy.viewer/with-params {:atom !phase :params [0]}
                          (fn [shift]
                            (((cube D) tanh) (- identity shift))))]
            (emmy.mafs/mafs
             {:height 400}
             (emmy.mafs/cartesian)
             (emmy.mafs/of-x shifted)
             ))))))

I guest it has something to do with ->f but still no luck tinkering to make (emmy.mafs/of-x shifted) to work