Fork me on GitHub

Hi there seem to be issues with the file-watch capability - depending on editor. if I use vim there are errors on backup files ending in ~ but even when reconfigured - there are invalid path exception errors being created.


getting there...


if I use vim - no luck xed - no luck nano - works ok...


with logs enabled, I see exceptions of the form 2023-02-01 14:02:07 DEBUG DirectoryWatcher:372 - DirectoryWatcher got an exception while watching! java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: /home/xxx/clerk-demo/notebooks/data_science.clj/? at java.base/sun.nio.fs.UnixPath.encode( ...

Sam Ritchie03:02:01

Ah, it’s not ignoring the temp files I guess? Works with emacs too…

Sam Ritchie03:02:27

Sorry, you said that!


The exception is raised by the watcher itself - so quite low level. maybe due to files the are quickly removed.


I looked closer at the DirectoryWatcher - the exception is thrown here according to the default listener this is just logged so should not cause errors. In the logs following this the watcher skips a ENTRY_CREATE event and then creates an ENTRY_MODIFY Event - I am not sure why this is skipped by clerk for update. log details and stack traces FYI 2023-02-01 15:58:48 DEBUG DirectoryWatcher:301 - ENTRY_DELETE [/home/.../clerk-demo/notebooks/data_science.clj] 2023-02-01 15:58:48 DEBUG DirectoryWatcher:372 - DirectoryWatcher got an exception while watching! java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: /home/.../clerk-demo/notebooks/data_science.clj/? at java.base/sun.nio.fs.UnixPath.encode( at java.base/sun.nio.fs.UnixPath.<init>( at java.base/sun.nio.fs.UnixFileSystem.getPath( at java.base/java.nio.file.Path.of( at java.base/java.nio.file.Paths.get( at io.methvin.watcher.PathUtils.subMap( at io.methvin.watcher.DirectoryWatcher.runEventLoop( at io.methvin.watcher.DirectoryWatcher.lambda$watchAsync$1( at java.base/java.util.concurrent.CompletableFuture$ at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.exec( at java.base/java.util.concurrent.ForkJoinTask.doExec( at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec( at java.base/java.util.concurrent.ForkJoinPool.scan( at java.base/java.util.concurrent.ForkJoinPool.runWorker( at java.base/ 2023-02-01 15:58:48 DEBUG DirectoryWatcher:301 - ENTRY_CREATE [/home/.../clerk-demo/notebooks/data_science.clj] 2023-02-01 15:58:48 DEBUG DirectoryWatcher:474 - Skipping create event for path [/home/.../clerk-demo/notebooks/data_science.clj]. Path already hashed. 2023-02-01 15:58:48 DEBUG DirectoryWatcher:301 - ENTRY_MODIFY [/home/.../clerk-demo/notebooks/data_science.clj]


I tracked this down - the problem is in the detail of the way that an editor writes files. in short - if using neovim then: :set backupcopy=yes see here for more details....


Dear Clerk, I love you so gosh darn much. If I had my way, you would be a system service that (clerk/serve!) 's my entire filesystem at all times. Thank you for being you. With <3, Matt

🖤 6

Discovering what #pathom does by visualizing its “resolving” by using Clerk. I'm using it on my day-to-day job, it's still a very recent experiment, but I already see that the onboarding for people who never heard about Pathom is way easier (and it also helps me to reason about what's happening when I call an Pathom attribute and how I could improve it).;cid=C87NB2CFN

❤️ 8
🖤 4

Is there a way to refresh the main entry on Clerk Garden? My published notebook seems to be stuck at an older version if I go to , newer version has already been created (


hmm, that sounds like a bug, right @U0303ET1R2T?


@U0FT7SRLP Did you build the newer version by manually going to or through the form on The form goes to and appends a query parameter ?update=1 , which makes that link update to point to the latest build. If you simply build a separate commit by directly visiting the stable url with the commit sha, the mutable url won't get updated.


@U0303ET1R2T Yeah I changed the commit manually the second time. And then I tried from the normal url and that pointed to the old one


Ah great! Thank you both


The original reasoning behind this was, that you could use the mutable url as a way to share your notebook with the world and you want updates to that be controlled. But the current UX is not great and we'll probably rework it (and document it better).


Yeah makes sense


The only “bug” would be that I tried again through the form. But I’m guessing Garden saw the cache and thought it didn’t have to update anything?


Oh, if you try again through the form it should update. That's indeed a bug. I'll look into it.


Thank you 🙏


Btw, really cool I can publish a db like this. Didn’t think about this before


@U0303ET1R2T I was just trying the update=1 and it seems that it only works with and not with => the extra slash


Oh, thanks for the catch!

👍 2

started working on showing the exec status / progress in the notebook. I like how I can add a specific viewer for this map and then there’s not a lot of boilerplate. 😸

💯 12
❤️ 10
👍 4

Hey @sritchie09. Follow-up from the;cid=C035GRLJEP8 trying to get @nivo/line setup as a viewer in Clerk using the latest greatest custom CLJS functionality in clerk-utils. I set up a barebones project and attempted to get it working piece by piece. I wasn’t successful but here’s the report so far 🧵


I set up this barebones project with only io.github.nextjournal/clerk and org.mentat/clerk-utils in the deps.edn


I created a Clojure namespace, jaydeesimon.notebook, to act as a notebook and a CLJS namespace, jaydeesimon.viewer, that will eventually house the nivo/line component for the viewer but for now, just has a dummy function that I’m hoping I can see will show up in a viewer via SCI


The first time I ran (mentat-clerk/serve! {:cljs-namespaces ['jaydeesimon.viewer]}), it generated a package.json, added some dependencies, invoked shadow-cljs and tried to build but it failed with an error

Sam Ritchie17:02:49

I was thinking about this, I think there’s an extra dependency I need to add, let me help out in a few when I’m back at my computer!!

Sam Ritchie17:02:01

We’ll get this dialed and I’ll use it to write the guide today


> I was thinking about this, I think there’s an extra dependency I need to add, let me help out in a few when I’m back at my computer!! Sounds good. Take your time. I've got some info I'll put down on this thread. Just ignore it til you have time

❤️ 2

This was the first error

The required JS dependency "react-dom/client" is not available, it was required by "nextjournal/clerk/static_app.cljs".


I added react-dom to the package.json, tried again and it failed looking for another JS dependency. I realized it was looking for all of the dependencies in Clerk’s package.json so I ended up adding all of the to my package.json


I was able to get past it looking for all of the JS dependencies by adding it to the package.json but now the build is failing with this message


The required namespace "reagent.ratom" is not available, it was required by "nextjournal/clerk/render/hooks.cljs".


Seems like it can’t find reagent. I tried adding reagent to the deps.edn and that didn’t work. I also created a shadow-cljs.edn and added it as a dependency there but also no luck


It seems like I’ll have a ways to go after the build part gets resolved but at the moment, I’m stuck


Here’s a to start it up and the package.json that was initialized by clerk-utils

Sam Ritchie17:02:47

Making sure its SHA matches the Clerk dependency

Sam Ritchie17:02:23

You should be able to delete the shadow-cljs.edn, it’ll be ignored

Sam Ritchie17:02:09

I THINK that should do it, but I’ll roll my sleeves up and make sure once I get home… I’m so glad you’re trying this, it’s important that this custom JS thing end up feeling as easy as possible @U098UL4QP


Ok yup that did it. I io.github.nextjournal/clerk.render as a dependency and it built and now works!


I'm not 100% sure how this kind of dependency works but I'll come back to that

  {:git/url ""
   :git/sha "d387037aa45303c32e311b62580b6f85e0b72919"
   :deps/root "render"}

Sam Ritchie17:02:41

It’s a dependency that Clerk stuck in a subfolder of the repo

Sam Ritchie17:02:32

So the “render” folder in the clerk repo holds a deps.edn that specifies extra stuff , that’s what this is pointing at


Ok gotcha I see it now

Sam Ritchie17:02:34

Next challenge is the SCI customization


Right. I want to get my jaydeesimon.viewer/add var available in SCI so I can invoke it from the viewer


Ok I'll work on that. You sent an example of how to make the namespace available in SCI yesterday. Gonna look for that. Will let you know how it goes

Sam Ritchie17:02:27

Basically do this but swap your namespace in for jsxgraph.core

Sam Ritchie17:02:06

And if this namespace in your project requires the namespace with “add”, you only need to pass this sci customizing namespace to serve!

Sam Ritchie17:02:26

And it will pull in anything else you use via the require

Sam Ritchie17:02:06

With live reloading of the JS . You probably have to restart the page if you add new functions but then on reload they will be available for the viewers


How/where does or should the SCI customization code happen? In the JSXGraph project, I think the jsxgraph.clerk-ui namespace is being and it's building it. Should I make my own shadow-cljs.edn and follow this example?

Sam Ritchie17:02:12

With the clerk-utils code all you need to do is make a cljs file analogous to the one above and stick it in src or dev,

Sam Ritchie17:02:18

And then pass it’s namespace symbol via :cljs-namespaces

👍 2
Sam Ritchie17:02:22

@U098UL4QP okay, back at the computer, so I can type properly formatted code 🙂

👍 2

Works! Amazing!


@sritchie09 I gotta go afk for a bit but this is great. Next step for me is to add the nivo/line dependencies, get it to bundle it up, add it to sci and then see if I can make a viewer from that

Sam Ritchie18:02:12

@U098UL4QP nice! and you don’t even need to have the viewer namespace in there; it will get pulled in automatically, since you’ve required it in jaydeesimon/sci.cljs

👍 2

Sounds good. I'll be back. Thanks for your help! Let me know what I can do to help with the guide

Sam Ritchie18:02:13

@U098UL4QP all you should have to do is npm install @nivo/line , then you’re back on the happy path of your repro the other day

🙌 2
Sam Ritchie18:02:23

@U04V15CAJ do you have thoughts on the cleanest thing to add to this SCI customization namespace, if @U098UL4QP wants to use a js library? I’m assuming create-ns won’t work when passed a js namespace…

Sam Ritchie18:02:38

I think we solved something like this, I can’t track down which project it was in


you should use the :classes option for JS libraries.

Sam Ritchie18:02:17

so (:require ["@nivo/line" :as nline]), then :classes {'nivo nline} , something like that?


It depends on what clerk does in its :load-fn


see the example I linked


I guess if you (set! clerk.sci/libname->class (assoc clerk.sci/libname->class "@whatever/your-lib" the-lib) then it would work

Sam Ritchie18:02:21

ah, I did this once and it worked:

(:require ["three" :as three])

(def three-ns
  (-> (into {}
            (map (fn [[k v]] [(symbol k) v]))
            (.entries js/Object three))))

 {:namespaces {'three three-ns}})


it works, but it's not recommended

Sam Ritchie18:02:11

it does have the advantage of not requiring that I control the load-fn


This would be the only change

Sam Ritchie18:02:02

awesome, I’ll try that out and note it for the docs

Sam Ritchie18:02:59

in @U098UL4QP’s case if he wants to convert the components like

(def line-ns
  (-> (into {}
            (map (fn [[k v]]
                   [(symbol k) (r/adapt-react-class v)]))
            (.entries js/Object line))))
, processing each value, would you recommend this style?


I just told you that I don't recommend turning JS libs into "cljs namespaces" ;)

Sam Ritchie18:02:10

haha no but read the example

Sam Ritchie18:02:28

this is taking everything inside the namespace and calling r/adapt-react-class on it

Sam Ritchie18:02:46

so I end up with Reagent components for every component inside some JS namespace

Sam Ritchie18:02:10

(in my head this is different, and something I would manually do if I were wrapping a React component library)


Do what you need to do in user space


I don't have enough context to say if this is a good idea in general

Sam Ritchie18:02:02

(def line-ns
  {'component1 (r/adapt-react-class line/component1)
   'component2 (r/adapt-react-class line/component2)})

Sam Ritchie18:02:05

this would be the alternative


why do you need to make a namespace out of this?

Sam Ritchie18:02:31

so folks can use those components in their clerk viewers

Sam Ritchie18:02:46

like [line/component1 {:k "v"} …]


I would just expose the JS lib and let people do what they normally do in reagent, but that's just my preference

Sam Ritchie18:02:55

It would be the same except with a :> as the first vector entry …


yes, which seems good enough to me

Sam Ritchie18:02:21

As long as I can document how to do it super easily for someone, that’s my top concern writing these Clerk guides

Sam Ritchie18:02:47

And you solved that above


I suppose you can defer those folks to the reagent docs, you don't have to protect them too much by wrapping stuff, that often leads to more surprises


but of course it all depends :)

👍 2

That's just my take, there is no single best answer I think

Sam Ritchie18:02:02

Sometimes I wrap if I’m mixing in other reagent components that I’ve added, so I don’t want the confusion of :>

Sam Ritchie18:02:20

I did it for the mathbox primitives so I could add docstrings

Sam Ritchie18:02:01

Im a pathological component wrapper I guess, full of justifications


As long as you also expose the raw stuff, so users have the flexibility, I think it's good. Sometimes a wrapper hides stuff by mistake, which is mostly what I'm concerned about


Looking forward to your final solution :)

Sam Ritchie17:02:23

@U04V15CAJ this is weird,

------ WARNING #1 - :undeclared-var --------------------------------------------
 File: /Users/sritchie/code/clj/clerk-utils/dev/clerk_utils/sci_extensions.cljs:14:7
  11 |   (sci/copy-ns clerk-utils.custom
  12 |                (sci/create-ns 'clerk-utils.custom)))
  13 | 
  14 | (set! cs/libname->class
 Use of undeclared Var nextjournal.clerk.sci-env/libname->class
  15 |       (assoc cs/libname->class
  16 |              "react" react))
  17 | 
  18 | ;; This next form mutates SCI's default environment, merging in your custom code

Sam Ritchie17:02:35

but it’s definitely there…


maybe try the fully qualified name?


I can't see the full context just from this snippet

Sam Ritchie17:02:10

sorry, yes, I’m trying to figure out if I have some wonky version of clerk, will report shortly

Sam Ritchie17:02:27

Here is the context… when I call the following ina cljs namespace:

(prn (ns-publics 'nextjournal.clerk.sci-env))
I see
{mount #'nextjournal.clerk.sci-env/mount,
 code-namespace #'nextjournal.clerk.sci-env/code-namespace,
 hooks-namespace #'nextjournal.clerk.sci-env/hooks-namespace,
 parser-namespace #'nextjournal.clerk.sci-env/parser-namespace,
 onmessage #'nextjournal.clerk.sci-env/onmessage,
 doc-url #'nextjournal.clerk.sci-env/doc-url,
 !edamame-opts #'nextjournal.clerk.sci-env/!edamame-opts,
 render-namespace #'nextjournal.clerk.sci-env/render-namespace,
 initial-sci-opts #'nextjournal.clerk.sci-env/initial-sci-opts,
 viewer-namespace #'nextjournal.clerk.sci-env/viewer-namespace,
 ->viewer-eval-with-error #'nextjournal.clerk.sci-env/->viewer-eval-with-error,
 set-state #'nextjournal.clerk.sci-env/set-state,
 ->viewer-fn-with-error #'nextjournal.clerk.sci-env/->viewer-fn-with-error,
 read-string #'nextjournal.clerk.sci-env/read-string,
 eval-form #'nextjournal.clerk.sci-env/eval-form}
which does NOT match the cljs code I see for the dependency hash I’m using:

Sam Ritchie17:02:26

when I search the Clerk codebase for parser-namespace , for example, I don’t see anything at all?? what a mystery


maybe try clojure -X:deps tree :aliases '[...]' (and fill in your aliases there too) and see what version of clerk you got?

Sam Ritchie17:02:36

weird, it doesn’t seem to show git dependencies: EDIT sorry still working…

Sam Ritchie17:02:15

io.github.nextjournal/clerk 4180ed3

Sam Ritchie17:02:20

that’s what I had expected…

Sam Ritchie17:02:50

I’m looking in the .gitlibs directory at the clerk dep, and I see the code I expect, but not the publics from that link

Sam Ritchie17:02:05

facepalm oh… I must have updated deps.edn after starting cider long ago…

Sam Ritchie17:02:18

let’s see… sorry folks


🎉 @nivo/line is successfully rendering! Here's my It was as easy as adding the @nivo/line npm deps and then adding ResponsiveLine to the :classes SCI context

🙌 4
Sam Ritchie01:02:56

Woohoo, that’s excellent news!!

Sam Ritchie01:02:19

That looks like a great library, looks good in the notebook too


I haven't tried building the static site yet but will try that out later. Thanks again for the help!

Sam Ritchie14:02:44

When you do you won’t be able to open index.html directly, you’ll need to run a web server so the site doesn’t complain about the relative JS link… copy the command from the clerk-utils package.json

Sam Ritchie14:02:03

Npm run serve I think? I will add this to the guide


@sritchie09 If your guide already uses babashka, you can also use:


or you can add it to deps.edn and run with clojure -X

❤️ 2
Sam Ritchie14:02:31

Thank you! This is getting better and better

Sam Ritchie14:02:07

If I can replace the gh-pages package with babashka I can ditch the package.json completely


What's the gh-pages package?

Sam Ritchie18:02:46

@U04V15CAJ sorry, back at the comp. I imagine this is easy to recreate; it’s an npm package that I’ve been using to accomplish the task of “move public/build (or whatever output-dir you choose) to the gh-pages branch and commit”

Sam Ritchie18:02:33

which has the side effect of triggering a GitHub Pages Publish

Sam Ritchie18:02:07

@U04V15CAJ I was GOING to say: “for, I think one thing we need is the ability to set the Cache-Control header to max-age=0, since the shadow-cljs build produces main.js and the browser caches it and won’t pick up changes….” But I forgot I’d gone the CAS route, so the path changes every time

Sam Ritchie18:02:33

boom, works great

Sam Ritchie18:02:35

me too, but sometimes when I’m working on a notebook I like to publish locally too without merging a PR (since I only have it run on changes to main)


I don't merge any PRs with that setup

Sam Ritchie18:02:13

but you do have to push to main right

Sam Ritchie18:02:35

which is definitely the right way to do it, that’s what I have set up too

Sam Ritchie18:02:18

but when I am first working on these libraries and trying to share default stuff sometimes I run bb release-gh-pages to get my local branch etc pushed up


anyway, committing to gh-pages is scriptable

👍 2