Demo of the “futurize current eval” command (here mapped to ^Z) I pushed hours ago:

[:prompt {:cmd false, clojure.core/*ns* <#C4C63FWP5|unrepl>/ns user, clojure.core/*warn-on-reflection* nil}]
(do (Thread/sleep 10000) 42)

^Z
[:eval <#C4C63FWP5|unrepl>/object [#unrepl.java/class java.util.concurrent.FutureTask "0x68f1b17f" "java.util.concurrent.FutureTask@68f1b17f"] 4]

(deref *1)
[:eval 42 5]

@cgrand cool, but just like the linux equiv I usually open another tab and let the other one run undisturbed

since that way I will be "notified" when the command finishes, opposed to having to check for it

serial execution and all 😉

yup I have a heated inner argument going on 🙂

hehe

Sending (future …) (or a/go or a/thread) (as a user) is always a possibility. However we don’t too that too often. It would be cumbersome to do that constantly. Here the intent is really “fix my blunder” (same spirit as : interrupt) not generic bg-eval.

well that assumes the blunder can be fixed, not everything is interruptible

So re:sequential exec I believe that the REPL should not add concurrency of its own, all concurrency should happen at the lang level (or put in other words: requested by the user)

agreed

to be honest even the interruptible eval is too much for me

re:interruptible in imy impl I use the dreaded Thread/stop which is more potent

https://docs.oracle.com/javase/8/docs/technotes/guides/concurrency/threadPrimitiveDeprecation.html

I perfectly understand and that’s why no command is mandatory.

and also deprecated 😉

it has been deprecated for a long time

yeah given the stance of backwards compatibility for the JVM it will probably never be removed

still I never use deprecated things for new code I write

just feels wrong

but back to the subject, I think interrupt is a good thing

but letting the REPL interrupt itself implies that everything must be evaled in its own thread

indeed

instead of using the thread of the loop itself

if you open another connection and somehow could access the other loop (hint shadow.repl) you could interrupt via the other connection

so 2 threads must pretend to be 1

granted that is a lot harder on the user than ^Z

be the 2 threads might probably be required anyways

otherwise you can't really interrupt something when the user disconnects from the REPL while something is running

well you could let that just finish

https://github.com/cgrand/unrepl/blob/master/src/unrepl/repl.clj#L178-L183

that propbably needs to check for EOF

might not always want to interrupt on EOF though

From a spec point of view it means going from a spec that assumes the existence of the connection to a spec that talks about setting up connections. Your modus operandi for interruption would broadens the scope of the spec.

sockets die all the time after all

yes, everything shadow.repl related assumes that the server was started in a shadow.repl aware fashion

it does not assume that there is a plain socket REPL we are going to upgrade

setting the desired behavior on disconnect would be a good candidate for a command

with a pushbackreader you can unread eof

Speaking of EOF when stdin is connected to a term and you hit ^D (not escaped, handled by the term) then .read returns -1 but subsequent reads work.

>> It follows that some tooling needs (e.g. autocompletion) may be better serviced by a separate connection which may not be a REPL (but may have started as a REPL upgraded to something else).

amen to that 🙂

not convinced that starting from a REPL and upgrading is the best course of action

but it certainly is one way to go

@thheller that’s why I used MAY and not SHOULD

Here is a more unusual command: :unrepl.jvm/enable-sideloader

[:unrepl/hello {:commands {:interrupt <#C4C63FWP5|unrepl>/raw \u0003, :exit <#C4C63FWP5|unrepl>/raw \u0004, :background-current-eval <#C4C63FWP5|unrepl>/raw \u001a, :set-source <#C4C63FWP5|unrepl>/raw [\u0010 <#C4C63FWP5|unrepl>/edn (set-file-line-col <#C4C63FWP5|unrepl>/param :unrepl/sourcename <#C4C63FWP5|unrepl>/param :unrepl/line <#C4C63FWP5|unrepl>/param :unrepl/column)], :unrepl.jvm/enable-sideloader <#C4C63FWP5|unrepl>/raw "\u0010(enable-sideloader)"}}]
[:prompt {:cmd false, clojure.core/*ns* <#C4C63FWP5|unrepl>/ns user, clojure.core/*warn-on-reflection* nil}]

unknown.Clazz
[:exception #error {:cause "unknown.Clazz", :via [{:type <#C4C63FWP5|unrepl>.java/class clojure.lang.Compiler$CompilerException, :message "java.lang.ClassNotFoundException: unknown.Clazz, compiling:(unrepl-session:1:165)”,
[.....] 
 <#C4C63FWP5|unrepl>/... {:get <#C4C63FWP5|unrepl>/raw "\u0010(... G__228)"}]} 1]
[:prompt {:cmd false, clojure.core/*ns* <#C4C63FWP5|unrepl>/ns user, clojure.core/*warn-on-reflection* nil}]

^P(enable-sideloader)
[:command :sideloader-enabled 2]
[:prompt {:cmd false, clojure.core/*ns* <#C4C63FWP5|unrepl>/ns user, clojure.core/*warn-on-reflection* nil}]

unknown.Clazz
[:unrepl.jvm/find-class “unknown.Clazz”]

nil
[:exception ... 3]

why does that need to be a command?

why not just eval (unrepl.jvm/find-class 'unknown.Clazz)?

but that seems like something tooling vs REPL. is that something the user would ever type by hand?

The example is a bit too dry: it’s something that happens during eval

if the compiler (or the code itself) tries to load a new class

not sure what your goals are here?

connecting to a process and extending its classpath

(require ‘namespace.available.on.the.client.but.not.the.target) would work

even if not top-level but a transitive dep etc.

ah missed that part .. the server is asking the client for a class that the server cannot find

yes, in the middle of compilation or execution

hmm but how would that work if you run into something that actually requires a bunch of classes?

fail the first one, request it, retry, fail the next one, request that, retry, ....

Well it would be transparent to the user.

I don't know ... I'm not convinced that bootstrapping the entire thing over a REPL is a good idea in the first place

now shipping basically an entire additional classpath over it

seems like that is asking for trouble

how would you handle version conflicts? the server has guava v15 but the client expects guava v21 or so

The classloader is local to the repl session

I assume its goal is to let tools ship "extensions" without adding that extension to the classpath of the server runtime?

It could but I was rather thinking about user code.

but the user code should be on the classpath of the runtime no?

On a remote JVM because you are trying to figure out something happening in one environment but not locally? "Open a ssh tunnel for the repl and code like it’s local" <- that’s the intent

I don't get it

the assumption is that you connected to a socket, whether thats local or remote makes no difference

the server has a classpath

but you want to load things there that don't exist

yes and I believe it’s more convenient to have them shipped on demand by the repl client than scp-ing them on the server class path (and remembering to clean of after)

yes I see that might be useful

but you could just send (unrepl.jvm/load-this-jar-please "cool.jar" <jar-contents>)?

and build a jar every time you modify the file?

(unrepl.jvm/load-this-class-please ...)

and transitive deps?

solved by the tool

and sending multiple load requests in order

trying to think of something I would want to load this way

the tool could even just process ClassNotFoundException and try to add them

so you’d rather reimplement the deps resolution mechanism on the tool side (with the risk of getting it slightly wrong or even totally wrong if the user code relies on reflection, explicit calls to class loader) rather than rely on the real thing.

but then you would respawn the evaluation until no ClassNotFoundException is thrown

(and hope none gets trapped/wrapped)

how would you do that differently?

ah the classloader can block

It’s about 20 loc: 10 for a stub ClassLoader in Java (which delegates findClass to a fn) and 10 for the fn

hmm yeah thats better than ClassNotFoundExceptions

'enable-sideloader
                     (fn []
                       (var-set clojure.lang.Compiler/LOADER
                         (unrepl.ClassLoader. (var-get clojure.lang.Compiler/LOADER)
                           (fn [name]
                             (when @unrepl
                               (write [:unrepl.jvm/find-class name])
                               (with-bindings {in-eval false}
                                 (when-some [base64-encodedclass (read)]
                                   (base64-decode base64-encodedclass)))))))
                       :sideloader-enabled)

but how would you handle version conflicts?

you can't just inject random classes and potentially pass those into user code

the server runtime has MyClass.someFn(a,b,c) loaded but the code you injected uses MyClass.someFn(a)?

Then your code throws

It’s for troubleshooting/dev, not for bulletproof hotpatching of a production server

I remain skeptical, seems to me like it would be way too easy to shoot yourself in the foot

its a cool idea though

The good thing is that no command is core to the protocol

This sideloader implementation is most certainly a bad idea. • The inversion of control between server and client may happens while the client as partially sent a form so the state of the PushbackReader may be dirty, which means more logic to track that. • if the repl has been upgraded it doesn’t work anymore.

All saner implementations seem to imply a repl connection and a request/response connection.

not only side-loading suffers from partial input

what else?

(some-long-running-task) (something-else) then ^Z while first one is running

doesn't background the first one

at least from looking at it the (.read *in*) would still have (something-else) left to read before getting to the ^Z

You conflate ui and protocol

do I? I meant that I entered (some-long-running-task) (something-else) and it went over the wire, then I realize that it takes a long time and want to bg it

not assuming any UI for the moment

since its a streaming protocol, the ^Z will not be processed until after (some-long-running-task) finishes?

you only peek at the next char in the loop and unread it

The client should not send an incomplete form.

it didn't, the user was just impatient

ie. I frequently load-file in-ns

so I have queued two things

can't bg or interrupt the load-file anymore

it sucks when load-file fails because in-ns still goes through, but every repl has that issue

How ok but should the client send the second form before having acknowledged processing of the previous one?

well that is the question .. is it streaming or RPC?

Why queue on the server (in the connection buffer) rather than on the client? Honest question

it implies that the client knows how to separate the text the user is typing into tokens

it can probably do that yes given that it upgraded the REPL in the first place

It reminds of two commands I wanted to explore: :enable-echo (to know how the reader cut the input and matching eval result with a chunk of input)

The other command is :reset-reader

And reset would effectively happens anywhere.

Thinking of things like inf clojure and neoterm, which send the current file/selection to a terminal naively, I can see the potential for confusion.

@dominicm are EDN out messages even reasonably parseable in vim script?

@cgrand no, but with python, no problem. Vim has good calling out to python, ruby and lua. Neovim can work with anything. Go has a good story around it for example.