clerk

2023-12-09T16:50:51.746349Z

When Clerk is rendering an large object coming from libpython-clj it does not behave well and is hanging, sometimes throwing strange exceptions. These classes are very dynamic, defining some number of interfaces: clojure.core.protocols.Datafiable clojure.lang.IObj java.lang.Iterable libpython_clj2.python.protocols.PBridgeToJVM libpython_clj2.python.protocols.PBridgeToPython libpython_clj2.python.protocols.PCopyToJVM libpython_clj2.python.protocols.PCopyToPython libpython_clj2.python.protocols.PPyAttr libpython_clj2.python.protocols.PPyDir libpython_clj2.python.protocols.PPyItem libpython_clj2.python.protocols.PPythonType libpython_clj2.python.protocols.PyCall tech.v3.datatype.ffi.PToPointer What I find curious is, that rendering such object is not the same as rendering (str ..) of such object (which behaves well). So what does Clerk do to render it (if not call str on it) ?

mkvlr 2023-12-10T14:23:32.276609Z

would need a proper repro to say anything

2023-12-10T17:06:16.969239Z

I made one here: https://github.com/behrica/clerkPythonIssue As it uses libpython-clj you need to get that set-up. The repo above contains a .devcontainer config, which can create a suitable setup on keypress in Docker. (using VSCode or devpod)

2023-12-10T17:07:45.657589Z

Open in "Codespaces" would work out-of-the box as well. (has free tier)

mkvlr 2023-12-09T19:26:09.374379Z

is it seqable?

mkvlr 2023-12-09T19:28:35.977779Z

clerk traverses sequable? collections, see https://book.clerk.vision/#presentation

mkvlr 2023-12-09T19:29:38.241539Z

> Here, we're giving it a set with 1, 2, 3 in it. In its generalized form, present is a function that does a depth-first traversal of a given tree, starting at the root node. It will select a viewer for this root node, and unless told otherwise, descend further down the tree to present its child nodes.

2023-12-09T21:42:26.793569Z

But it does not render anything, see screenshot.

Andrea 2023-12-11T10:39:10.676699Z

Quickly tried and couldn’t get a much “saner” print representation of the dataset over nREPL either. If the call py/->jvm is the way to go to have a decent one and you want it to be the default for pyobject values you could use a viewer like:

(def pyclj-viewer
  {:pred (comp #{:pyobject} type)
   :transform-fn (clerk/update-val py/->jvm)})

(clerk/add-viewers! [pyclj-viewer])
If you leave the object as is instead, I think Clerk falls back to just EDN printing which sends a huge string to the client making it unresponsive

👍 1
Andrea 2023-12-11T10:44:10.919289Z

> What I find curious is, that rendering such object is not the same as rendering (str ..) a string is (should 🙂) always be printable in Clerk because we apply some pagination which prevent the above issues

2023-12-11T14:47:18.531999Z

I was coming from the "java world", so I made this probably wrong assumption, that Clerk should at a certain moment call "toString" when getting a "unknown Java type". This would be reasonable to do in Java, as a lot of Java classes implement "toString" reasonably well. In my concrete case, the "unknown object" is as well sequable?, which probably triggers a certain viewer. But then something goes wrong, as the items of the objects are "strings" (long strings). -> which should trigger the eliding of the string and cut it and prevent crash But this does not happen. So maybe there is something to improve on Clerk side ?

Andrea 2023-12-11T15:19:57.532219Z

Ok, I guess seqable? is not enough, we assume an object to be sequential? to recursively present its components (don’t recall why atm). If you call

(viewer/viewer-for (clerk/get-default-viewers)
                     (py/py. datasets-module fetch_20newsgroups)) 
you see the pyclj object actually dispatches to the fallback viewer, which essentially does a pr-str and sends the whole EDN data to the client. Btw (seq (py/py. datasets-module fetch_20newsgroups)) only returns the keys ("data" "filenames" "target_names" "target" "DESCR")

2023-12-11T21:03:57.028929Z

yes, this "object" is weird. In libpython-clj the python object wrappers implement certain clojure related java classes to behave "as much as possible" as Clojure types

2023-12-11T21:04:31.718759Z

>> you see the pyclj object actually dispatches to the fallback viewer, which essentially does a pr-str and sends the whole EDN data to the client.

2023-12-11T21:05:47.066549Z

any reason that no "string shortening" gets done in this case, as it gets done for "String" ?

2023-12-11T21:28:35.334429Z

These 2 as well behave well:

hlship 2023-12-09T22:05:22.989319Z

I can't imagine a much better experience for Advent of Code than Clerk + Clojure.

🖤 4
➕ 1
🙏 2