This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2017-03-14
Channels
- # arachne (5)
- # architecture (2)
- # bangalore-clj (7)
- # beginners (96)
- # boot (34)
- # braveandtrue (1)
- # cider (12)
- # cljs-dev (38)
- # cljsrn (59)
- # clojure (326)
- # clojure-dev (35)
- # clojure-greece (1)
- # clojure-italy (6)
- # clojure-russia (47)
- # clojure-spec (16)
- # clojure-uk (25)
- # clojurescript (136)
- # core-async (18)
- # cursive (18)
- # datascript (2)
- # datomic (28)
- # dirac (6)
- # emacs (4)
- # garden (3)
- # hoplon (28)
- # instaparse (1)
- # jobs (4)
- # juxt (1)
- # lein-figwheel (10)
- # liberator (1)
- # mount (3)
- # off-topic (39)
- # om (16)
- # om-next (1)
- # onyx (15)
- # pedestal (9)
- # proton (1)
- # random (1)
- # re-frame (48)
- # reagent (8)
- # ring-swagger (4)
- # rum (3)
- # specter (5)
- # sql (3)
- # unrepl (273)
- # untangled (27)
- # vim (4)
- # yada (7)
@cgrand I still have trouble wrapping my brain around the intended upgrade process with CLJS in mind
well eval
is split in compile and run and only the run part happens in the JS runtime (self-hosted CLJS notwithstanding)
but you need to override read
(for interrupt etc) as well which happens on the JVM side
starting another REPL while inside CLJS is kind of inlikely since it would start read
on the JS side which doesn't exist
Things can work if the CLJS REPL starts out with the protocol in mind, but upgrading it now as done is CLJ is really hard
only way I see so far would be to force the user to call (unrepl.cljs/repl the-actual-repl-fn)
so it can inject the hooks it needs
how that would transfer the necessary stuff for print/eval to the CLJS side I don't know though
for extra kicks the JS runtime also doesn't have a blocking read
from *in*
or even an *in*
when in a browser
we could probably fake that somehow by forwarding the *in*
of the JVM to JS but that just makes my head hurt
the JS platform just has so many differences to the JVM that many things just can't be translated 1:1
It’s really the hybrid model and the current crop of repls giving access only to the CLJS side (no defmacro at the repl, right?)
short term I believe it’s easier to provide an unrepl
replacement to repl/repl
compatible with its backends.
About yesterday issues, they are not similar: • the classloading stuff suffered from concurrency between msgs sent by the server and the client • interruption/backgrounding is ok as long as the client knows when a form is complete.
but if the client is assumed to wait for the result before sending more isn't that RPC?
I don’t believe so (RPC) but I don’t want to play a definitions game, waiting is a meh solution
tagged literals are not complex as long as you preserve them, aliases, autoresolution, autogensyms, syntax quote, eval-read etc. are the pain
Hmmm that’s true we don’t need a full blown reader we just need to tell when a form is complete (and to cross finger to not disagree with the server – eg a new syntax in Clojure 1.10 )
only later to realize that I couldn't send #inst "2017"
as it treated it as two forms
@kotarak some clients will do it anyways though to provide syntax highlighting or auto-complete for the REPL input
That doesnt mean it's actually reading. There is no reading in Vim for syntax highlighting. There were bugs in lein because it didn't have necesaary custom readers available.
So what I have in mind is to have one (1) escape sequence (preferrably just one char). From an impl point of view, at the Reader level (big R: java Reader, before the lisp reader) the escape sequence is detected once detected further characters are sent to an edn reader, once 1 edn form is read.
it means that the input (from the user) is still free form and commands can happen anywhere in the middle.
Didn't you say that eval runs un separate thread? Why isn't the command side-channeling possible then?
I still think all this is much simpler in the shadow.repl model where you'd send the interrupt command over another connection
generally opening a new connection is underrated, trying to make everything work over one connection is really hard
independent connections are simple, dependent connections a bit less. (I’m trying to draw a line in the sand)
I understand @theller like the user sends "(long-command) (next-command)". Eval of first runs. User sends stop/background command. Unrepl applies it to runnung command. Since the eval is on different thread, unrepl can still respond.
@kotarak currently unrepl only peeks at the next char in the stream. which isn't the command so it won't handle the command
@kotarak, read may be altered by previous evals so (except if you accept unbound buffering) you can’t accept ^C anywhere
my mental model was “one form after the other” with ^C
being a sort of form (clojure reader happily returns a symbol for ^C
btw)
@kotarak the issue is (long-running)|(+ 2 2)^C
where |
is the current stream position (after read
and while in eval
)
@kotarak how does a term work when you have a process (computing not reading) and a stdin which is ....(1 million input chars)^C
where does the megabyte go? (ok one megabyte is not big)
I think the point is: there is stdin and the terminal which accepts the ^C from the user.
No. It's not. The big-batch is entirely up to the client. It goes to the stdin queue. The command can be executed immediately.
I mean: the user can run arbitrary code so preventing her/him from hogging memory is useless
if you backpressure further commands won’t be sent so the end result is the same : command is ignored (except that back pressure allows the client to inform the user
but a command may be stuck in a client-side buffer so the client doesn’t know it’s ignored.
so unrepl reads constantly and looks for commands and pipes everything into the other *in*
while unrepl reads from the real *in*
and writes everything except commands to the PipedWriter
Let’s put names on things: socket-in
is all the client send and we want to route things either to regular-input-out
or to commands-out
, ok?
unrepl reads from socket-in
always and creates unrepl-in
, binds that to *in*
so the REPL will read from that
An expression is evaluating (trying to find a SHA256 collision ;-)), so no ones read from regular-input-in
@theller but eval is busy so no reading occurs on unrepl-in, at some point (if no unbound buffer), the “routing” loop is going to block because writing to unrepl-in
blocks
yeah it is unlikely the user will type megabytes of data before trying to interrupt something
still seems very term-ish to me but that might be cool if you actually use a terminal and not a tool
for a tool I'm fairly sure the second connection is easier, all you need to know is the ID of the connection you want to interrupt
@kotarak from a VIM perspective: the user wants to interrupt the REPL, so I assume you have a :repl-interrupt
command for that, or just map ^C
to that command
I could. I'd rather reuse the existing connection, though. But in general: yes. It would work.
btw what I'm really after is this: https://github.com/Microsoft/language-server-protocol/blob/master/protocol.md
I want the the editor to tell me didSave
so I can skip watching the filesystem in my CLJS build process
the REPL just becomes a different part of the system that just has to solve REPL things
I'm trying to explain why way of thinking (and why I'm pushing for the second connection)
> the REPL just becomes a different part of the system that just has to solve REPL things
For tooling it's better to have your own server loop. I did that back then in VimClojure. It worked well.
does it detect that the user started (cljs.repl.node/repl)
and provide complete/doc lookup for CLJS?
assuming the user did that in the REPL and not through some other command you created in VimClojure
?
As Rich said: > REPL stands for something - Read, Eval, Print, Loop. > It does not stand for - Eval RPC Server/Window.
For tooling I want to evaluate a certain set of commands with defined input. And get a certain output back.
I have the impression that the REPL is the only thing you are allowed to use and must use it for everything
I don't have an issue with bencode
or even JSON-RPC
really they work for what they are supposed to be
sure I would prefer EDN
but it doesn't matter really. once you implement the protocol you never see it again
I'm still confused. I will just silently work on vimpire and see where I get. Unrepl could give vim a real repl with stdin reading. And without python.
{"op": "doc-lookup", "sym": "condp"} vs. Printf("(doc-lookup %s)", hopefully-proper-escaped-string)
I just think a repl is poor counter point for a protocol that is not user interaction.
@cgrand I do agree however that CLJS could get away with RPC eval just fine as it can never read
anyways
also CLJS might actually complete a read
(JVM-side) but not be able to eval
when no browser is connected to a browser REPL
re-re-re-re-reading Rich on REPLs >>> we should be precise about what we call REPLs. While framing and RPC orientation might make things easier for someone who just wants to implement an eval window, it makes the resulting service strictly less powerful than a REPL. Let's look at some of that power: 'read' is a protocol. It works on character streams, one of the most widely available transports. It uses delimiters (spaces, [], (), {} etc) to separate forms/messages. Readers implement this protocol and return data upon encountering complete forms. read is shipped with Clojure and multiple implementations are available. It is available for use in programs, including programs you invoke in a REPL. A substantial subset of evaluable+read-able forms are supported by edn, which has even wider support. The reader format is extensible. So, when you get read, you are getting a lot. When embedded in a REPL, even more is implied by read. There must be some source (stream) of things read. That source is the same source that should be used should the code being evaluated call 'read'. If that source is a terminal with a human, they can continue their interaction in place. 'eval' implies the full power of Clojure. Take a form as data, evaluate it, return a result as data. 'print' takes the returned data and prints it on an output stream as characters. It is true that not all things might print read-ably, which can pose challenges for programs sitting on the other end of a REPL that would like to call 'read' on its output. But we have improved Clojure in this area - now the default print-method prints something that is read-able (with an appropriate tag handler installed). Another area where many REPLs presume a human consumer and print un-read-ably is when exceptions occur. This socket REPL will default to printing exceptions read-ably, but you can switch to a human oriented formatter of your choice. Again, as with read, when embedded in a REPL, even more is implied by print. There must be a destination (stream) of things printed. That destination is the same destination that should be used should the code being evaluated call print. If that source is a terminal with a human, they can leave their reading focus in place and see all output targeted towards them. 'loop' - do it again. Note however that a REPL will not attempt to read/print while evaluating. The stream binding and nestability is important. It means that you can, in the code you evaluate in your REPL, launch a nested interpreter, or Eliza-like thing, or a text-adventure etc, and those things will get input from the same source and print to the same target. This is what someone who knows what REPLs are expects and demands. You can, if you desire, even launch an eval RPC server in a REPL, but not vice-versa. REPLs work over the most primitive connections, terminals, and can be pipe, rerouted, etc with all tools. They accomplish a lot over a single connection, which will often be all you can get. But, we now have nice editors/IDEs often running on the same machine, with the ability to use sockets. Multiplexing their needs with those of a human REPL consumer over the same connection is going to make things bad for one or both of them. With the socket REPL, the recommendation is that tools will open more than one connection, and use a dedicated channel for most editor/IDE operations that might want to interact with the running process. With the improved read-ability of REPL output, I think an edn-based REPL makes a fine RPC server for program-to-program communication, especially since the set of 'verbs' supported is bounded only by what you can evaluate, and the types of things you might discover is open. These are the intentions of this feature.
> But, we now have nice editors/IDEs often running on the same machine, with the ability to use sockets. Multiplexing their needs with those of a human REPL consumer over the same connection is going to make things bad for one or both of them.
A takeaway is that a repl is also about providing its very own building blocks to the user. That’s where a CLJS repl fails. That’s where unrepl fails by not providing access to a function for tagged printing
my approach so far is "cap the output at X size" and if it goes over that don't print the result but print "Eval successful but result value too large too print."
so from a tools perspective I would prefer if I could eval
but don't print
sometimes
the REPL solves that in a sense already by (def x (produce-something-large))
and just printing #'user/x
FWIW I have this issue in shadow-build where it would print the entire CLJS compiler-env after completing the build
so all my fns that are invoked via the REPL looked like (defn foo [] (do-the-work) :done)