This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
When Clerk is rendering an large object coming from libpython-clj
it does not behave well and is hanging, sometimes throwing strange exceptions.
These classes are very dynamic, defining some number of interfaces:
clojure.core.protocols.Datafiable
clojure.lang.IObj
java.lang.Iterable
libpython_clj2.python.protocols.PBridgeToJVM
libpython_clj2.python.protocols.PBridgeToPython
libpython_clj2.python.protocols.PCopyToJVM
libpython_clj2.python.protocols.PCopyToPython
libpython_clj2.python.protocols.PPyAttr
libpython_clj2.python.protocols.PPyDir
libpython_clj2.python.protocols.PPyItem
libpython_clj2.python.protocols.PPythonType
libpython_clj2.python.protocols.PyCall
tech.v3.datatype.ffi.PToPointer
What I find curious is, that rendering such object is not the same as rendering (str ..)
of such object (which behaves well).
So what does Clerk do to render it (if not call str on it) ?
clerk traverses sequable? collections, see https://book.clerk.vision/#presentation
> Here, we're giving it a set with 1, 2, 3 in it. In its generalized form, present is a function that does a depth-first traversal of a given tree, starting at the root node. It will select a viewer for this root node, and unless told otherwise, descend further down the tree to present its child nodes.
But it does not render anything, see screenshot.
I made one here:
https://github.com/behrica/clerkPythonIssue
As it uses libpython-clj
you need to get that set-up.
The repo above contains a .devcontainer
config,
which can create a suitable setup on keypress in Docker.
(using VSCode or devpod
)
Open in "Codespaces" would work out-of-the box as well. (has free tier)
Quickly tried and couldn’t get a much “saner” print representation of the dataset over nREPL either.
If the call py/->jvm
is the way to go to have a decent one and you want it to be the default for pyobject values you could use a viewer like:
(def pyclj-viewer
{:pred (comp #{:pyobject} type)
:transform-fn (clerk/update-val py/->jvm)})
(clerk/add-viewers! [pyclj-viewer])
If you leave the object as is instead, I think Clerk falls back to just EDN printing which sends a huge string to the client making it unresponsive> What I find curious is, that rendering such object is not the same as rendering (str ..)
a string is (should 🙂) always be printable in Clerk because we apply some pagination which prevent the above issues
I was coming from the "java world", so I made this probably wrong assumption, that Clerk should at a certain moment call "toString" when getting a "unknown Java type". This would be reasonable to do in Java, as a lot of Java classes implement "toString" reasonably well. In my concrete case, the "unknown object" is as well sequable?, which probably triggers a certain viewer. But then something goes wrong, as the items of the objects are "strings" (long strings). -> which should trigger the eliding of the string and cut it and prevent crash But this does not happen. So maybe there is something to improve on Clerk side ?
Ok, I guess seqable?
is not enough, we assume an object to be sequential?
to recursively present its components (don’t recall why atm). If you call
(viewer/viewer-for (clerk/get-default-viewers)
(py/py. datasets-module fetch_20newsgroups))
you see the pyclj object actually dispatches to the fallback viewer, which essentially does a pr-str
and sends the whole EDN data to the client.
Btw (seq (py/py. datasets-module fetch_20newsgroups))
only returns the keys ("data" "filenames" "target_names" "target" "DESCR")
yes, this "object" is weird. In libpython-clj
the python object wrappers implement certain clojure related java classes to behave "as much as possible" as Clojure types
>> you see the pyclj object actually dispatches to the fallback viewer, which essentially does a pr-str
and sends the whole EDN data to the client.
any reason that no "string shortening" gets done in this case, as it gets done for "String" ?
These 2 as well behave well: