Clojurians Log v2

Wow, lots of things happened while I was away.

I can confirm that commit 336167c fixed my initial issue. Thank you so much!

it’s not optimized yet so currently works for small things only

I should add diffing, currently we send the whole new state on a change

Are you by any chance also using hiccup around this data? It’s also a common thing with the hiccup library since the html macro tends to explode the code

The data itself isn’t in Hiccup syntax. I made sure of this by using pr-str to convert the edn data to a giant string, and Clerk didn’t manage to swap! it. At the moment I’m working around this by redefining clerk.webserver/ws-handlers and clerk.render/atom-changed, and it’s working. The giant string is going through. It seems that the solution is just to avoid having the giant value be nested in a fn or eval form.

can you paste a small repro?

Absolutely, but I need some time, if you don’t mind. I’m still in the middle of work.

sure, no rush, thank you

@mkvlr Thanks for the wait. This notebook should produce the following error lines:

Caused by: Syntax error compiling fn* at (0:0).
...
Caused by: java.lang.IndexOutOfBoundsException: Method code too large!
...

I can reproduce this. I tried to change the form from (fn [_] data) to (let [data ...] (fn [_] data)) which didn't help. just eval with a large blob of data seems to behave like this. I tried with sci.core/eval-form which did work for a large blob of data

but perhaps introducing a new operation :reset! instead of :swap! would also help, where you don't have to push the data through eval

This at least fixes this particular case for me: https://github.com/nextjournal/clerk/pull/710

My local workaround right now is exactly the same as your PR. I haven’t tried sci-core/eval-form though. I might try it later.

sci.core you need to add this library which I did with (clojure.repl.deps/add-lib 'borkdude/sci) in the REPL

@nam_nguyenhoai you can use clerk with :git/sha 39da39689570d8efb841ec5849761a8529f1f6a3 in your own project to test the PR, without building your own custom CLJS

@mkvlr I’m so sorry, I’ve been working on something else and just returned to my Clerk project today. There seems to be an issue with Clerk’s rerendering (see my latest message in #clerk).

https://github.com/nextjournal/clerk/commit/748f73e4958ac5131889196ff81635b3f83a90f5 so working with large ⚛️ sync atoms should me much faster now.

https://github.com/nextjournal/clerk/pull/722 should speed things up by a lot

@nam_nguyenhoai would be great if you could take it for a spin and let me know if it works for you and if you notice any other issue with it

I worked on this during my transatlantic flight, I pushed two things to main: • fixing it for large values https://github.com/nextjournal/clerk/commit/336167c858d9640710092f564ac7a76cb2834c1e (same change both of you did, sorry for the overlap) • sending editscript diffs instead of the whole value for atom changes (clerk was already doing this for the other direction for notebooks) https://github.com/nextjournal/clerk/commit/233541b5bcc3e6dc1ac7d565a41333bd07f12b07

this works now but the recompute performance is still pretty poor

fixing this is a bit more involved, I’ve started on https://github.com/nextjournal/clerk/compare/delay-viewer-eval?expand=1

we ran into issues caused by the eval on read a number of times, right @borkdude? Though overnight I had the idea that we could also delay evaluation by implementing viewer-eval on top of viewer-fn (invoke the viewer fn does the eval).

doing eval when reading always felt wrong to me

but so far we got away with it

in the change above I’m doing it explicitly in a postwalk step

and moving away from the defprotocol things

not sure that this is good though

maybe the defprotocol stuff is good but we should use it a bit differently (like not eval on read)

what's the problem you're trying to solve exactly here?

https://github.com/nextjournal/clerk/blob/08cebfe8ea2159466c27e56f9225b8851ac5f88e/notebooks/delay_viewer_eval.md

ah yes

(defn eval-cljs
  ([form] (eval-cljs {} form))
  ([viewer-opts form]
   ;; because ViewerEval's are evaluated at read time we can no longer
   ;; check after read if there was any in the doc. Thus we set the
   ;; `:nextjournal.clerk/remount` attribute to a hash of the code (so
   ;; it changes when the code changes and shows up in the doc patch.
   ;; TODO: simplify, maybe by applying Clerk's analysis to the cljs
   ;; part as well
   (with-viewer (-> viewer-eval-viewer
                    (update :transform-fn comp maybe-rewrite-cljs-form-for-cherry)
                    (assoc :nextjournal.clerk/remount (hash-sha1 form)))
     viewer-opts
     (->viewer-eval form))))

this is also a related problem

yes, it seems better to have a separate eval phase

or perhaps a delayed eval like your PR is called

do you have a preference for a custom type like we have now vs something simpler (tagged data)?

i thought I simplified things by moving to tagged data [tag opts form]

seems like an unimportant detail to me

but not sure anymore

that helps, thanks

dang, why didn’t I think of the fact that eval could be delayed leaving stuff mostly as is yesterday

because you're on a plane? ;)

that’s the downside of having too much time on the plane, right

or having jetlag? ;)

at least you killed some time :)

so if we do this:

(defn ->viewer-fn [form]
  (map->ViewerFn {:form form
                  #?@(:cljs [:f (delay (eval form))])}))

we should still be able to bind *eval* to a cherry evaluator when we invoke the fn, is that correct?

yes, but note that theoretically any form could also return a delay so you need to distinguish between your delay and a returned delay perhaps

yes

will deref on invoke

cool

oh right, yes, (delay (eval '(delay ...)) will work out ok

just not deref twice

yes

hmm, seems simple enough but isn’t working. https://github.com/nextjournal/clerk/commit/0be230bcfe965bf19fe20b1277f37db877008144 Do you see anything wrong?

I hate these failure modes

I'd have to run that locally, but I'm afk soon

can look Monday with some more focus

cool, I’ll write if I get futher until then

pro tip: don't do a demo with an experimental PR

found it 🙃

https://github.com/nextjournal/clerk/commit/b11926b2813f6a5d997af2024b686407a5a3bcfb

👍

I remember having this error using an old(er) version of cider.el once. Perhaps it's just a matter of upgrading

Also the repo you're pointing at should probably not be used as a reference, it's 3 months old and just a test at the time. Things have changed since then

The way you should connect: Add :render-nrepl to serve! > - :render-nrepl opt into starting a render nREPL server into Clerk's render environment running in the browser, pass {} to start it on the default port or pass a custom :port number. And then from emacs (and I'm having trouble finding this in the docs @mkvlr btw, but perhaps this is because it's just how it's done by default):

cider-connect-cljs
select port
select repl type: nbb
sesman-link-with-buffer

If you still have problems doing it that way, please create a step by step repro and I'll have a look next Monday

I am using CIDER 1.16.0, which seems to be the latest stable version. Before trying it with the cloned repo, I did set up a minimal Clerk project using :render-nrepl in serve! and cider-connect-cljs with nbb as you mentioned and got the same error. Actually, already in the browser console (while Emacs freezes), I get

Unhandled clojure.lang.ExceptionInfo
Clerk encountered the following error attempting to read an incoming message: No reader function for tag object
{:message 
"
"
{:type :nrepl, :msg {:prefix-rewriting "false", :ns "user", :err "unknown-op", :"
0 more…"}

Update: after removing the Cider extension clj-refactor.el, it works just fine with no errors! I guess I’ll leave it uninstalled for now, haven’t used it much anyways.

Good to know

I was sad to see it was only three minutes 😆 More Clerk content makes more happy.