Fork me on GitHub
#unrepl
<
2017-03-10
>
dominicm10:03:21

Something that's extremely useful for tooling is message ids. So that I can "track" the result of my command. I can't seem to see that in the current discussion though. Is there an alternative?

cgrand10:03:58

@dominicm if I understand correctly, it’s the 3rd component of messages (”group id”) that you are looking for. No?

dominicm10:03:24

Possibly, I will admit to not having looked closely. Just watching your examples with [:out "foo"] for example

dominicm10:03:05

It looks like it is, yeah!

cgrand10:03:04

@dominicm impl & spec were coevolved but sometimes one or the other was ahead. Some past examples may slightly differ now.

dominicm10:03:43

Are there any rules/guarantees about group ids? (I'm totally going to keep trying to mix the concepts from nrepl into the design :p)

cgrand11:03:53

none they are even optional, it’s best-effort group ids 🙂

cgrand11:03:00

tell me what properties you’d like

dominicm11:03:02

@cgrand I was just wondering if there was any uniqueness required like nrepl had. I've never fully understood that aspect though.

cgrand11:03:15

well if you always use :fortytwo as your group id, you may get angry users

dominicm11:03:50

I was about to ask about "completion" or "finished" events. But thinking about it, they don't necessarily make sense either, at least in the core.

dominicm11:03:01

They do for slow, streaming operations, which do have an actual completion

cgrand11:03:41

completion like in “done” not like in “autocompletion”?

dominicm11:03:08

Yes, completion as in done, sorry.

cgrand11:03:06

well you either get [:eval result group-id]or [:command result group-id] or [:exception e group-id]

dominicm11:03:20

In the nREPL, which is a slightly different model, you would get something like: In-> {:op :eval :code "(do (println :hello) 10)" :id 1} Out-> {:out ":hello" :id 1} {:value 10 :id 1} {:status ["done"] :id 1} When you got a :status key, you'd know nothing more would happen for your result.

cgrand11:03:42

but it’s not a guarantee that you won’t get more :outs from this group-id (as therads may have been spawned)

dominicm11:03:10

The nrepl was somewhat confusing about that point. I've seen a few confusing situations by that.

dominicm11:03:43

I think for unrepl the only question mark I have, and I think auto-completion might be a good example of this, or maybe cljr-refactors "find-usage". Say you have an operation you know is slow, but produces multiple independent values. Perhaps the result of something like:

(take 1000 (iterate (fn [x] (Thread/sleep 1000 (inc x))) 10))
It would be nice to be able to get:
[:somekey 1 :fourtytwo]
[:somekey 2 :fourtytwo]
…
[:somekey 1000 :fourtytwo]
[:finished :fourtytwo]
I'm not sure the key should definitely be separate. Maybe it's fine to have a sentinel value you watch for (e.g. [:somekey ::finished :fourtytwo]). :somekey would be similar to :eval, but would have a multiple-occurence as opposed to single.

cgrand11:03:12

Hello and prompt

[:unrepl/hello {:commands {:interrupt <#C4C63FWP5|unrepl>/raw \u0003, :exit <#C4C63FWP5|unrepl>/raw \u0004, :set-source <#C4C63FWP5|unrepl>/raw [\u0010 <#C4C63FWP5|unrepl>/edn (set-file-line-col <#C4C63FWP5|unrepl>/param :unrepl/sourcename <#C4C63FWP5|unrepl>/param :unrepl/line <#C4C63FWP5|unrepl>/param :unrepl/col)]}}]
[:prompt {clojure.core/*ns* <#C4C63FWP5|unrepl>/ns user, clojure.core/*warn-on-reflection* nil}]
Form sent + return value
(future (while true (println "Hi Dominic!") (Thread/sleep 2000)))
[:eval <#C4C63FWP5|unrepl>/object [#unrepl.java/class clojure.core$future_call$reify__6962 "0x5d5f10b2" {:unrepl.ref/status :pending, :unrepl.ref/val nil}] 1]
Outs keep going:
[:out "Hi Dominic!" 1]
[:out "\n" 1]
[:prompt {clojure.core/*ns* <#C4C63FWP5|unrepl>/ns user, clojure.core/*warn-on-reflection* nil}]
[:out "Hi Dominic!" 1]
[:out "\n" 1]
[:out "Hi Dominic!" 1]
[:out "\n" 1]

dominicm11:03:44

Oh, I understand prompt now, that's a neat way of implementing sessions…

cgrand11:03:05

1 session == 1 connection

cgrand11:03:30

About (take 1000 (iterate (fn [x] (Thread/sleep 1000 (inc x))) 10)): • if it’s a user eval, it will block the repl (until completion or interruption — if supported), an extension may add a :extension-name/long-bg-eval command but plain eval blocks • if it’s part of some command (eg :auto-complete then the command author may choose to emit whatever kind of messages s/he desires (please namespace them) and client implementations must disregard not-understood msgs.

dominicm11:03:46

Okay, so you think the "done" should be implemented by the author of the command. Okay, interesting. I think that's better generally.

dominicm11:03:24

I have a feeling that the nrepl "sorta" blocks evals, it uses a queing system iirc.

dominicm11:03:44

But yeah, I like this. Makes more sense now I've picked your brain a little 🙂. I'll be watching closely.

cgrand11:03:50

yeah but while eval is going you can still have autocompletion (with nrepl)

dominicm11:03:19

Ah, so you can't have an eval during autocompletion currently? Interesting.

cgrand11:03:10

if you want to have them going in parallel you either need to have two connections OR multiplex over a single one. Both are out of scope of unrepl core.

cgrand11:03:41

an unrepl is first and foremost a repl

dominicm12:03:25

From a tooling perspective, do you ever want to reuse a connection? Also, if I have 3 clients on a single connection, do they all get broadcasted the :eval/:out etc.?

cgrand12:03:06

@dominicm you want reuse if the connection is painfully setup to traverse networks

cgrand12:03:41

and how can you have 3 clients on the same connection?

pesterhazy12:03:31

works. nice!

dominicm12:03:55

@cgrand I mean, it's a socket. So, couldn't multiple people be connected to that socket & do (unrepl.repl/start)

cgrand12:03:45

@dominicm from the server point of view each client (even if connected to the same server socket) is a different connection. They are isolated

dominicm12:03:30

@cgrand I see. I guess that increases the need for async commands.

pesterhazy12:03:23

the idea I think is to open multiple connections if you want to do things concurrently @dominicm

pesterhazy12:03:53

a client could have a separate "tooling connection" for autocompletion etc

dominicm12:03:14

@pesterhazy You couldn't have a separate connection for cider's output buffer though

pesterhazy12:03:17

or even a connection pool if you need to do >1 tooling task at the same time

dominicm12:03:37

That needs to suck up from everywhere.

pesterhazy12:03:01

CIDER's main buffer would be the primary connection

dominicm12:03:31

It requires that all connections go through a single interface so it can be monitored. Means that the global *out* feature of CIDER wouldn't work when combined with my-bin-foo which connects to unrepl and does some stuff

pesterhazy12:03:24

I'm not familiar with the global *out* feature

dominicm12:03:00

@pesterhazy wrap-out middleware in cider-nrepl

pesterhazy12:03:54

unrepl does give you *out* back in EDN form

dominicm12:03:30

@pesterhazy wrap-out sends *out* to all client sessions

pesterhazy12:03:11

ah if somone printlns in some other thread, that gets sent to emacs, right?

pesterhazy12:03:49

hm I never found that terribly useful as I always connect to an external JVM process (not launched by emacs)

pesterhazy12:03:02

but I realize a lot of people use the jack-in feature

pesterhazy12:03:19

hm but if emacs spawns a boot process or whatever, can't it just redirect its stdout to a buffer?

dominicm12:03:59

I'd prefer it if the process didn't need to be started by a host.

dominicm12:03:07

This will be used on remote systems

pesterhazy12:03:03

you can always start a separate repl connection that listens to the process's stdout and writes it to the buffer

pesterhazy12:03:21

I always felt cider does too much by default - features like "global out" could be implemented as opt-in modules

pesterhazy12:03:20

even using jack-in should be an additional convenience over cider-connect, rather than the simpler default of connecting to an existing process

dominicm12:03:04

> opt-in modules What does this mean?

pesterhazy13:03:21

I meant a modular feature which can be turned on if the user decides

dominicm13:03:34

as part of unrepl?

cgrand13:03:02

of cider I guess

dominicm13:03:30

but, how would cider implement it?

benedek13:03:04

reality is ppl already struggle to make difference between cider and clj-refactor

benedek13:03:28

they expect all the refactorings there because they installed cider

cgrand13:03:27

so your point is that reality push to make cider more and more battery included?

dominicm13:03:29

I've noticed that too. CIDER is sort of the ultimate in culmination of editor tooling.

benedek14:03:14

yeah i guess. this is my point

cgrand14:03:06

Ok I may be going out on a limb but here is an idea for CLJS/JVM repls: when sending a snippet to eval from a file it’s easy to know if it’s for clj, for cljs or both (cljc). Directly at the repl, consider (by default) that the user is typing some cljc and evals twice: clj then cljs (and eval cljs even if clj throws an exception). Now one has to make the output intelligible (not swamped in stacktraces) and clearly tagged as clj or cljs. What do you think?

dominicm15:03:27

@cgrand If cljc is eval'd twice, will there be multiple [:eval 10] responses?

cgrand15:03:16

maybe something like

(type {})
[unrepl.cljs/entangle {:clj 12 :cljs 11} 11]
[:eval “clojure.lang.PersistentArrayMap” 12]
[:eval “cljs.core/PersistentArrayMap” 11]

cgrand15:03:31

but would the end result be useable?

dominicm15:03:34

Like this ^^ Definitely

dominicm15:03:00

At least from a consumption perspective

cgrand17:03:04

Having mapped :interrupt to ^C I now want to map ^Z to :background-current-eval which would transform the current eval in a future and returns it.

dominicm17:03:45

> and returns it The future?

dominicm17:03:12

Ah okay, I thought it might be the async [:background-eval 10 :fourtytwo]

richiardiandrea17:03:53

awesome, I would like to get this repl that can read the future asynchronously 😄

richiardiandrea17:03:48

but if the delay is big enough, then are we still getting the future? :thinking_face: 😹

dominicm17:03:16

I do wonder how much value there is over (future my-stuff)?

cgrand17:03:30

None it's more like "sh*t it's taking too long, let's background it"

dominicm18:03:31

Take the current evaluation?

dominicm18:03:52

After I've hit enter?

dominicm18:03:52

I can't even fathom how that's done 😛

bhauman22:03:21

I am really digging these ideas and this code. I recently spent a good bit of time making a much better piggieback/interruptible-eval for CLJS. So this code is very very helpful. Much thanks.

bhauman22:03:08

The REPL topic is at the forefront of my thinking right now.

bhauman22:03:29

So obviously tooling calls can go over a second connection.

bhauman22:03:06

Is there a mechanism for extention?

cgrand09:03:23

The protocol itself is obviously open: add your own namespaced commands, messages and encodings. However, if that was the question, I haven't thought enough about a command to extend implementation at runtime. Anyway it would just be one command and, as such, not core to the protocol (which is: cleanly multiplexed out, self discovery of commands)

bhauman23:03:52

The idea is that a cljs repl would just take over the *in* stream and that it we could fairly easily create/launch a cljs output-multiplexing unrepl

bhauman23:03:14

CLJS is such bi-furcated universe that tooling normally needs a set of services that work in conjunction with the repl and that would benefit from using the repl as a means of communicating with the js env client ...

bhauman23:03:24

* editor sends a file-changed message/command/clj-fn-call

bhauman23:03:01

* this call in turn triggers a cljs compiler

bhauman23:03:55

* the result of this compilation sends a report of namespaces changed/to-load to the browser

bhauman23:03:50

* using the repl as a means of communication in this last step is a nice to have

bhauman23:03:37

In addition to this it is also interesting to consider that the repl-eval-env/js-runtime is also a ui and could possibly want access to a stack of cljs services that reside on the "server-side" in the CLJ environment.

bhauman23:03:37

Walking through a model of this,

bhauman23:03:46

in an unrepl session start a figwheel server process and call commands against it. start/stop/compiler. The websocket and compile-env that the process creates will be needed to create a cljs repl.

bhauman23:03:32

Open another clojure.main repl upgrade it to the cljs-unrepl with a figwheel-based repl-env by referencing a var that was set in the previous window.

bhauman23:03:10

Now we have a situation where we can simply control the figwheel tools in one session and have a CLJS repl in the other.

bhauman23:03:52

The thing is that it would be nice for all environments(clj/cljs/self-host-cljs) to converge on a base set of functionality which is what the unadorned stream based repl does.

bhauman23:03:26

Duplicating unrepl functionality for cljs wouldn't be too difficult and it would meet this end.

bhauman23:03:02

But tooling environments/needs/services start diverging very quickly, so it would be nice to have a set of standard commands (i.e. completion) that tooling authors can rely on across implementations.

bhauman23:03:38

oh boy sorry I'm just dumping all this here without ... and it's definitely getting into the off topic range