Clojurians Log v2

What would a hook give you over "listing those namespaces explicitly as :unresolved-namespace :exclude"

Not sure how you envision such a hook

the hook would check if the namespace exists

how?

by trying to find it on the classpath for example

hooks don't run in a normal JVM, they run in a sandbox (SCI)

but even if you could find the namespace on the classpath, it's not a guarantee that the namespace is already loaded

this is what require is for ;)

in our case we can’t require it because it would be a circular dep

and we know it’s loaded

to avoid duplication you could make an ns-group for both clojure.core/comment and nextjournal.clerk/example and then configure it only once with config-in-call

or just disable the linter wholesale in the "comment" things

didn’t know about ns-group, nice that solves the having to write it down three times

I don't see how calling (require 'the-namespace) in a comment form would cause a circular dep btw

have you ever considered to bring dynamism to the config so I could generate the list of namespaces?

I have considered it but so far I've resisted it due to the complexity it brings. You could maybe programmatically generate the .clj-kondo/config.edn as an alternative

ah, so I could require it inside the comment, but not at the ns level

yes

this works at least for clojure.core/comment, not sure about nextjournal.clerk/example but perhaps it works when you do :lint-as clojure.core/comment . If that doesn't work, I could probably fix that

what does the hooks sci sandbox expose btw? Is there access to the filesystem?

also thought about generating the config, will probably give that a try

there is no access to the file system. there is however ns-analysis which gives you access to analysis of already linted namespaces from the cache. but there isn't a hook that is fired on unresolved namespaces or so

why not first try the ns-groups thing

and then the generation option as a second alternative

I could have a look at this on monday btw, if you haven't solved it by then

yes, ns groups first makes sense

still think it’s worthwhile to reduce the friction with writing these rich comments

but well possible that it’s not worth the complexity

nice that the hooks are properly sandboxed. Where can I read more about how that works? On https://github.com/clj-kondo/clj-kondo/blob/master/doc/hooks.md it says > The namespaces clojure.core, clojure.set and clojure.string are also available. but I guess slurp etc is not?

correct

after that blog the :macroexpand option appeared to make this stuff a little easier to use at the cost of precision of locations

should have been more precise, I meant how the sandboxing works with regards to slurp. Is https://github.com/clj-kondo/clj-kondo/blob/972f5f87579b1cd12d1bce0a8d0d3fdeb897271e/src/clj_kondo/impl/hooks.clj the right place to look?

yes. it's just a regular SCI config. not sure what you would want to slurp though, but I haven't fully thought through the consequences of adding slurp and if this is something that (e.g. third party, exported) hooks should be able to do. also note that clj-kondo isn't always started from the root of the project directory which complicates the usage of slurp as well.

I meant that I don’t see where slurp is excluded

well, slurp isn't added in SCI by default since by default it behaves like a safe-to-use sandbox

ah ok, makes sense

if you're looking for something like "does this namespace 'exist'" in the sense that it's something clj-kondo has seen before, you can use ns-analysis in hooks

but currently there isn't a hook related to unresolved-namespace

this wasn’t related to my problem anymore (there ns-group and then maybe generate the config makes sense), was just curious

curiosity is always good :)

speaking of safety: :termination-safe was removed because of the performance downsides & implementation complexity (e.g. https://github.com/babashka/sci/issues/348), right?

wondering if sci could cooperate and check if the thread is interrupted in its main interpreter loop

it was removed because SCI doesn't really control the whole virtual machine it runs on which is something you need for this

btw the linked example takes 140ms on my machine now, down from the reported 6s, nice job!

on Clojure IRC they are now using SCI to have a safe eval via the IRC channel. They control the timeout by just starting a new bb process (which comes with sci.core built-in) and killing the process after n seconds

(def t
  (Thread. (fn []
             (time
              (sci/eval-form
               (sci/init {})
               '((fn a [n] (if (#{0 1} n) 1 (+ (a (- n 2)) (a (- n 1)))))  40))))))

(.start t)
(.interrupt t)

(.getState t)

would there be a place in sci where we could check for (.isInterrupted (Thread/currentThread)) while running the recursion?

or is that vm internal?

I'm sure it could, but this would still not solve cases like (str/join "," (range))

in other words, you'd need to instrument all core and other built-in stuff as well. this is why java agents exist

to do it at the bytecode level

not saying it would solve all problems with termination

but might be worthwhile to check for interrupted where sci can to make it more cooperative when running in a thread

I have a working impl https://github.com/babashka/sci/compare/fn-interrupt?expand=1. You want an issue with the problem statement rather than have it be part of the PR description, right?

I don't even know if I want this WITH a problem statement. So maybe a github discussion would be the best place to start

I don't want to give users the impression that SCI will always behave a certain way reliably with respect to this. Just fixing it here maybe give some partial solution but I'm not convinced that it will be sufficient

this is not about giving a guarantee

it’s about fixing a problem that you can’t interrupt a long-running recursive function

can you do that in normal Clojure?

I don’t think so

being able to do that is a nice thing about an interpreter

in clojure you have to check that in user code

but if you can make sci more cooperate better in a multithreaded setting and it doesn’t come with a heavy implementation burden or performance impact (I couldn’t observe one in my very limited testing) that’s a win imo

using sci to eval user code with limited compute time (say do what you want in 100ms) is a pretty valid use case I think

and recursive functions are a way common way by which you can shoot yourself in the foot

and this solves this problem

I think of this similarly to a performance optimization: it makes my system more responsive to the interrupt command and frees up resources faster.

I've once been on a project where I've written a macro which injects that kind of code in all kinds of places to make it more robust, but eventually it was never robust enough. I'm not going to pollute the codebase with any instrumentation code unless it's a complete solution not just for some cases

Maybe the CIDER approach with the agent is a better solution

for development at least

not sure why you’re arguing for a solution that needs to solve all problems related to interrupting now

I’m not claiming this solution is that

I’d prefer sci to handle some of the cases and I think function calls are a big chunk, loop recur is probably another.

think it’s a nice tradeoff with the interpreter, you give up some performance by running further away from the metal but gain some control