Fork me on GitHub
#clerk
<
2023-12-09
>
Carsten Behring16:12:51

When Clerk is rendering an large object coming from libpython-clj it does not behave well and is hanging, sometimes throwing strange exceptions. These classes are very dynamic, defining some number of interfaces: clojure.core.protocols.Datafiable clojure.lang.IObj java.lang.Iterable libpython_clj2.python.protocols.PBridgeToJVM libpython_clj2.python.protocols.PBridgeToPython libpython_clj2.python.protocols.PCopyToJVM libpython_clj2.python.protocols.PCopyToPython libpython_clj2.python.protocols.PPyAttr libpython_clj2.python.protocols.PPyDir libpython_clj2.python.protocols.PPyItem libpython_clj2.python.protocols.PPythonType libpython_clj2.python.protocols.PyCall tech.v3.datatype.ffi.PToPointer What I find curious is, that rendering such object is not the same as rendering (str ..) of such object (which behaves well). So what does Clerk do to render it (if not call str on it) ?

mkvlr19:12:09

is it seqable?

mkvlr19:12:35

clerk traverses sequable? collections, see https://book.clerk.vision/#presentation

mkvlr19:12:38

> Here, we're giving it a set with 1, 2, 3 in it. In its generalized form, present is a function that does a depth-first traversal of a given tree, starting at the root node. It will select a viewer for this root node, and unless told otherwise, descend further down the tree to present its child nodes.

Carsten Behring21:12:26

But it does not render anything, see screenshot.

mkvlr14:12:32

would need a proper repro to say anything

Carsten Behring17:12:16

I made one here: https://github.com/behrica/clerkPythonIssue As it uses libpython-clj you need to get that set-up. The repo above contains a .devcontainer config, which can create a suitable setup on keypress in Docker. (using VSCode or devpod)

Carsten Behring17:12:45

Open in "Codespaces" would work out-of-the box as well. (has free tier)

Andrea10:12:10

Quickly tried and couldn’t get a much “saner” print representation of the dataset over nREPL either. If the call py/->jvm is the way to go to have a decent one and you want it to be the default for pyobject values you could use a viewer like:

(def pyclj-viewer
  {:pred (comp #{:pyobject} type)
   :transform-fn (clerk/update-val py/->jvm)})

(clerk/add-viewers! [pyclj-viewer])
If you leave the object as is instead, I think Clerk falls back to just EDN printing which sends a huge string to the client making it unresponsive

👍 1
Andrea10:12:10

> What I find curious is, that rendering such object is not the same as rendering (str ..) a string is (should 🙂) always be printable in Clerk because we apply some pagination which prevent the above issues

Carsten Behring14:12:18

I was coming from the "java world", so I made this probably wrong assumption, that Clerk should at a certain moment call "toString" when getting a "unknown Java type". This would be reasonable to do in Java, as a lot of Java classes implement "toString" reasonably well. In my concrete case, the "unknown object" is as well sequable?, which probably triggers a certain viewer. But then something goes wrong, as the items of the objects are "strings" (long strings). -> which should trigger the eliding of the string and cut it and prevent crash But this does not happen. So maybe there is something to improve on Clerk side ?

Andrea15:12:57

Ok, I guess seqable? is not enough, we assume an object to be sequential? to recursively present its components (don’t recall why atm). If you call

(viewer/viewer-for (clerk/get-default-viewers)
                     (py/py. datasets-module fetch_20newsgroups)) 
you see the pyclj object actually dispatches to the fallback viewer, which essentially does a pr-str and sends the whole EDN data to the client. Btw (seq (py/py. datasets-module fetch_20newsgroups)) only returns the keys ("data" "filenames" "target_names" "target" "DESCR")

Carsten Behring21:12:57

yes, this "object" is weird. In libpython-clj the python object wrappers implement certain clojure related java classes to behave "as much as possible" as Clojure types

Carsten Behring21:12:31

>> you see the pyclj object actually dispatches to the fallback viewer, which essentially does a pr-str and sends the whole EDN data to the client.

Carsten Behring21:12:47

any reason that no "string shortening" gets done in this case, as it gets done for "String" ?

Carsten Behring21:12:35

These 2 as well behave well:

hlship22:12:22

I can't imagine a much better experience for Advent of Code than Clerk + Clojure.

🖤 4
🙏 2
1