Another question: would it be possible to build a stepping debugger on top of SCI? I.e. is there a way to tell it to evaluate a single form at a time?

@wombawomba do you mean top level forms, or also function arguments etc?

I mean also function arguments etc — basically I'd like something like (eval-1 state '(+ 1 (inc 1))) => [state' '(+ 1 2)]

...or optionally something that doesn't modify the form, but keeps some sort of execution state to track which parts of the form have been evaluated

@wombawomba what would be your use case for this? it's not supported by SCI currently. it would require changes inside SCI and these changes are going to affect performance since you need to make checks like "am I debugging? then do this, else that".

Basically I'm building something like a programmable cloud — think AWS(-light) with a simple set of primitives on top of which you can code your own infrastructure — and I'd like to provide a user-friendly development environment on top of it. If it's possible, being able to execute code one step at a time and looking at the results would be a really useful addition to this development environment.

(FWIW, being able to pause/resume jobs is valuable for HA in an environment like this — since it means I could migrate running code between hosts — which is why I want that capability)

I'd imagine it should be possible to build a macro that'd transform an input form to insert whatever debugging checks would be necessary, to avoid a performance hit for non-debugging scenarios

One way to do this would be to rewrite the form in continuation-passing style — perhaps via something like https://github.com/positronic-solutions/pulley.cps#what-is-pulleycps

For a CI-like environment I think executing top-level expressions: like, do this, see the effect, then do that, should be a good start. Kind of a like a notebook where you can see intermediate results.

Yeah, I already have a REPL for situations like that, but there are scenarios where stepping debuggers can be a useful complement

FWIW such debuggers are pretty popular in other languages; for instance the PDB variants in Python (e.g. https://github.com/pdbpp/pdbpp)

(CIDER has somthing similar: https://docs.cider.mx/cider/debugging/debugger.html)

I haven't looked into the SCI internals too closely, but since AFAICT it's basically a Clojure interpreter written in Clojure, implementing stepping should be comparatively straightforward

@wombawomba In Clojure I hardly ever use a debugger. I use tools like println, tap, or def. If my tool would not support a debugger I as a user would probably write it in such a way that I could inspect intermediate results by breaking things down into smaller steps myself. Implementing that in (a fork of?) SCI would be a non-trivial amount of work. Having a macro that generates everything twice could work. But still a lot of work. If this is a commercial undertaking I'd be willing to put paid time into it, perhaps as a fork of SCI. Having said that, there is also a low cost way of getting almost the same with a user-land macro. E.g this break point tool works with babashka (based on SCI) as is: https://github.com/technomancy/limit-break

Another approach is to instrument the expression before running it through SCI (which is similar to what CIDER does). This can happen outside of SCI without any changes.

Yeah I was just wondering if that could be a useful way forward here. If I could do continuation passing across SCI calls then I should be able to just build this myself on top of SCI.

@wombawomba What I mean is this:

(def ctx (sci/init {}))
(sci/eval-form ctx
               '(defmacro debug []
                  (let [ks (keys &env)]
                    `(do
                       (println "locals:" (zipmap (quote [~@ks]) [~@ks]))
                       (println "Press return to continue")
                       (read-line)
                       nil))))

(sci/binding [sci/out *out* sci/in *in*]
  (sci/eval-string* ctx "
(defn foo [x y z]
  (debug)
  (+ x y z))

(foo 1 2 3)"))
;; =>
;; locals: {x 1, y 2, z 3}
;; Press return to continue
"6"

You can write debug, you can get some information, the users can inspect whatever. And then you can continue evaluation as always.

This debug function/macro can do whatever you want, read things from disk, open an ssh connection, evaluate more expressions to inspect the state of your CI, etc.

@afoltzm I tried that library you just posted here: https://github.com/swannodette/delimc and it worked with babashka (and hence it will work with SCI)

Alright, thanks :)