Fork me on GitHub
#unrepl
<
2017-03-09
>
cgrand08:03:58

@cfleming thanks for logging

cgrand08:03:12

Answers OoO to your questions: 5. No. EDN is a subset of Clojure code and Clojure code can only reprsents is a subset of values (and a value may contain a cycle). Thats’ why I have a handful of tagged literals to fill the gap.

cgrand08:03:24

4. I don’t want to restrict to the sole scenario of upgrading a plain repl.

cgrand08:03:44

3. There was no official socket repl either. EDN can be backported as a lib.

cgrand09:03:14

2. yes it’s an abstraction over repl impls (a single lang may have more than one) to keep details about what to eval to do what from crippling in client code.

cgrand09:03:41

1. The input being free form means that eval is what you get by default. In my impl the :unrepl/hello msg advertises the support of :interrupt, :set-source and :exit.

cgrand09:03:54

:set-source is described (in the msg, not in the spec) as sending ^P followed by (set-set-file-line-col …):

[:unrepl/hello 
 {:commands
  {:interrupt <#C4C63FWP5|unrepl>/raw \u0003,
   :exit <#C4C63FWP5|unrepl>/raw \u0004,
   :set-source <#C4C63FWP5|unrepl>/raw [\u0010
                            <#C4C63FWP5|unrepl>/edn (set-file-line-col
                                          <#C4C63FWP5|unrepl>/param :unrepl/sourcename
                                          <#C4C63FWP5|unrepl>/param :unrepl/line
                                          <#C4C63FWP5|unrepl>/param :unrepl/col)]}}]

cgrand09:03:22

so a client can trigger a command iff it understands the encoding and all the params

cgrand09:03:49

The hello message may also contain some id stating the nature of the repl (”CLJS”).

cgrand09:03:05

Re: SQL, did you consider the user upgrading to an unrepl SQL terminal?

cfleming09:03:47

@cgrand Sure, I understand how the set-source command would be run, but I don’t understand how it actually works. For example, when a user sends a form from their editor to the REPL, if I don’t give that code the correct source location stack traces etc will be wrong. That’s why I do the wrapping that @thheller was complaining about - to be fair, it’s pretty awful.

cfleming09:03:09

So rather than a set-source, what’s really needed IMO is an eval-at-pos

cfleming09:03:35

Or eval-with-pos, perhaps. i.e. it needs the source position and the code to be evaled.

cgrand09:03:48

what’s not ok with issueing set-source and then sending the form to eval?

cfleming09:03:47

Sure, that’s a possibility, just trying to clarify that that is how it works. It feels somewhat icky, since you’re swapping essentially a functional interface for a stateful one, but that could work.

cgrand09:03:21

a repl is a stateful thing

cfleming09:03:04

Sure, but trying to minimise the dependency on state in potentially racy ways is generally a good goal.

cfleming09:03:26

But I think that’s probably ok, just trying to clarify in my mind what you’re envisioning.

cgrand09:03:54

I don’t envision concurrency over one connection

cgrand09:03:19

eg no call to doc while an eval is running

cgrand09:03:48

you would need two connections for that (or multiplexing not covered by unrepl)

cgrand09:03:17

the only concurency I have is to support :interrupt and (as all commands) is optional

cfleming09:03:37

So for something like execution interruption, you would use a command like… right, :interrupt would stop the last executed command?

cgrand09:03:07

would stop the current running eval

cgrand09:03:53

(I should check if, in my impl, it could stop the current running command)

cgrand09:03:07

(that’s a good question on the behavior to spec)

cfleming09:03:33

The SQL thing is really an IntelliJ issue I have to deal with. But even something like Emacs will have to deal with it too, i.e. the user would probably want to switch major/minor modes depending on what they’re wanting to send.

cgrand09:03:48

currently commands are not interruptible

cfleming09:03:50

But it illustrates the problem that the client can’t automatically know what the server is expecting.

cfleming09:03:10

Right. nREPL uses Thread.stop() IIRC, since interruption is probably unreliable.

cfleming09:03:27

Which means you need to start evals on a new thread

cgrand09:03:32

I use Thread.stop() too

cgrand09:03:04

commands are not interruptible but it’s an impl detail

cfleming09:03:41

What do you think about the possibility of the REPL advertising in its hello message a content type, or something similar?

cgrand09:03:53

should they be? by the same comand?

cfleming09:03:58

So the server could advertise to the client what it was expecting to receive.

cfleming09:03:24

I think so. Commands are just a parameterised evaluation.

cgrand09:03:00

it’s a good idea prompted by your questions (see https://clojurians.slack.com/archives/unrepl/p1489050829293403)

cfleming09:03:19

Ah, I missed that - I think that would be very useful.

cfleming09:03:30

For client side tooling.

cgrand09:03:33

I don’t know how interrupt appears in Cursive UI

cfleming09:03:04

It’s a button in the REPL toolbar.

cfleming09:03:19

And I have an open issue to make it bindable to a keystroke.

cgrand09:03:34

but if it’s the same interrupt for user-eval and command a user may interrupt a command (or you have to keep track of what is running: eval or command)

cgrand09:03:13

while interrupting a command may be something you want to do because it’s takiing too long

cfleming09:03:28

What will happen if a client sends multiple commands? The server will have to read one ahead for interruption to work, I guess.

cgrand09:03:38

So: • one command which interrupts only eval • one command which interrupts anyithing • two commands, one for eval, one for commands

cfleming09:03:51

Since by definition it will have to read the interrupt command while something is still evaluating.

cfleming09:03:26

Interrupt seems kind of special, now that I think about it.

cgrand09:03:41

like I said interrupt is the only case of “reading while eval” that I plan to spec

cfleming09:03:35

So the exec loop will read a command, execute it on another thread to allow interruption, then read the following command and execute it if it’s an interruption, otherwise queue it?

cgrand09:03:37

and because the result of read may depend on the repl state, reading ahead is not great

cgrand09:03:57

currently I use only one char to interrupt (`^C`)

cfleming09:03:32

Seems like having interrupt not require form parsing is a good idea.

cfleming09:03:13

Apart from that case I think I’d rather have the commands have easily-inspectable textual representations, but I think that is a good exception.

cgrand09:03:16

Hmmm one can imagine :secret-ns/stop for interrupt: it works too because you only need a wider pushback window and you don’t need to use read

cfleming10:03:16

Right, any hardcoded string would be fine.

cgrand10:03:17

However (secret/stop) would not work because it has unbounded representations (eg ( secret/stop ))

cgrand10:03:13

How a repl would advertise its nature? which kind of information?

cfleming10:03:18

I’m not sure - what immediately springs to mind is a content type, but perhaps just a keyword would do.

cfleming10:03:49

I don’t even know if content types exist for program code apart from text/clojure or something.

cgrand10:03:50

Well there’s the lang but also the platform

cgrand10:03:31

cljs - jvm -browser cljs self-host v8

cfleming10:03:43

True, and for CLJS you might want text/clojurescript;env=node

cfleming10:03:21

It’s tricky - imagine the change-a-repl-to-cljs case.

cgrand10:03:23

why a contenttype string rather than a map?

cfleming10:03:35

No real reason, a map is probably a better idea.

cgrand10:03:05

I’m all ears

cfleming10:03:34

I connect to a socket REPL, but at that point I haven’t received anything - I still have to guess what is on the other end.

cgrand10:03:22

plain repl?

cfleming10:03:25

I send my unrepl code for Clojure after guessing, and I get a [:hello {:lang :clj :env :jvm}] but it’s mostly redundant since at that point I know what it is anyway.

cfleming10:03:12

Then I send some code to start a CLJS REPL, and I’m still in the same situation - I’m faced with an [:upgrade nil] and a socket, but I don’t know what’s coming.

cfleming10:03:32

At that point I probably do, since I started the CLJS REPL, but if the user has done it manually I don’t.

cfleming10:03:51

It seems like the REPL type notification comes after it would be useful.

cgrand10:03:21

Only if you consider that unrepl repls won’t become more supported

cgrand10:03:36

what if there’s a unrepl cljs repl?

cfleming10:03:44

Well, no - I have to know what the remote REPL type is in order to send my unrepl init code, no?

cgrand10:03:12

so when the user upgrades you get [:bye nil]followed by [:hello …cljs…]

cfleming10:03:14

I can’t send cljs-unrepl init code to a Clojure REPL, or an SQL one.

cfleming10:03:55

Assuming the user runs something which starts a CLJS unrepl directly. If it starts a CLJS socket REPL I won’t get that.

cgrand10:03:20

I mean yes it won’t get that

cgrand10:03:37

bleh I always sucked at double negations

cfleming10:03:47

Hehe, I got you

cfleming10:03:29

It seems to me that the content advertisement comes too late to be useful in many (not all) cases.

cfleming10:03:22

In the case that the user is in one type of unrepl (say CLJ), and runs code to start another sort of unrepl (say CLJS) right away, then it’s useful.

cfleming10:03:52

But you still can’t discover from a raw socket REPL what the other side is expecting. Writing it down like that, it seems obvious.

cgrand10:03:00

Bootstrapping a plain repl to an unrepl one is just that a bootstrapping solution. I’d like more repls to support unrepl from the start. In this case (larger ecosystem) content advertisement will be useful.

cfleming10:03:16

Yes, I agree with that.

cgrand10:03:42

In the mean time you have to rely on clutches (a magic “sniff platform and send upgrade blob” button)

cfleming10:03:31

Yes, or the user explicitly telling you what to do (convert CLJ REPL to unrepl, convert CLJS REPL to unrepl, etc)

cfleming10:03:53

I suspect there will always be a certain amount of user interaction here, and the user will have to have some idea of what’s actually going on.

cfleming10:03:05

Which it would be nice to avoid, but is probably unavoidable.

cfleming10:03:51

Ok, I need to go to bed - it’s late here and I have an early start tomorrow.

cfleming10:03:11

I’ll read up on further discussion when I get in tomorrow.

cfleming10:03:34

In general, this is starting to feel like a nice solution, though.

cgrand10:03:04

ok & thanks, good night!

cgrand13:03:59

Clojure 1.9.0-alpha14
user=> (require '[unrepl.repl])
nil
user=> (unrepl.repl/start)
[:unrepl/hello {:commands {:interrupt <#C4C63FWP5|unrepl>/raw \u0003, :exit <#C4C63FWP5|unrepl>/raw \u0004, :set-source <#C4C63FWP5|unrepl>/raw [\u0010 <#C4C63FWP5|unrepl>/edn (set-file-line-col <#C4C63FWP5|unrepl>/param :unrepl/sourcename <#C4C63FWP5|unrepl>/param :unrepl/line <#C4C63FWP5|unrepl>/param :unrepl/col)]}}][:prompt {clojure.core/*warn-on-reflection* false, clojure.core/*ns* <#C4C63FWP5|unrepl>/ns user}]
Create an image
(java.awt.image.BufferedImage. 100 200 java.awt.image.BufferedImage/TYPE_INT_RGB)
[:eval <#C4C63FWP5|unrepl>/object [#unrepl.java/class java.awt.image.BufferedImage "0x366ef90e" {:width 100, :height 200, :attachment <#C4C63FWP5|unrepl>/mime {#unrepl/... {:get <#C4C63FWP5|unrepl>/raw "\u0010(... G__219)"} <#C4C63FWP5|unrepl>/... nil}}] 1][:prompt {clojure.core/*warn-on-reflection* false, clojure.core/*ns* <#C4C63FWP5|unrepl>/ns user}]
print-level was set to low, ask for the elided part
^P(... G__219)
[:command ([:content-type “image/png”] [:get <#C4C63FWP5|unrepl>/raw "\u0010(file G__218)”)) 2][:prompt {clojure.core/*warn-on-reflection* false, clojure.core/*ns* <#C4C63FWP5|unrepl>/ns user}]
Ask for the content (it’s a :get in a #unrepl/mime):
^P(file G__218)
[:command "iVBORw0KGgoAAAANSUhEUgAAAGQAAADICAYAAAAePETBAAABKElEQVR42u3RIQEAAAjAsPcvDSUQiIkX+KpGrzIBiIAAERAgAgJEQIAIiIAAERAgAgJEQIAIiIAAERAgAgJEQIAIiIAAERAgAgJEQIAIiIAAERAgAgJEQIAIiIAAERAgAgJEQIAIiIAAERAgAgJEQIAIiIAAERAgAgJEQIAIiIAAERAgAgJEQIAIiAwAIiBABASIgAARECACIiBABASIgAARECACIiBABASIgAARECACIiBABASIgAARECACIiBABASIgAARECACIiBABASIgAARECACIiBABASIgAARECACIiBABASIgAARECACIiBABASIgAARECACIgOACAgQAQEiIEAEBIgJQAQEiIAAERAgAgJEQAQEiIAAERAgAgJEQAQEiIAA0V0L0zPWZOcDDp8AAAAASUVORK5CYII=" 3][:prompt {clojure.core/*warn-on-reflection* false, clojure.core/*ns* <#C4C63FWP5|unrepl>/ns user}]
base64 of a black png

bhauman14:03:45

Just want to thank @cfleming for inviting me and say that I've read the whole conversation including the linked discussions.

bhauman14:03:13

I haven't delved deep into the proposed solution yet.

bhauman14:03:33

But some things are starting to surface

bhauman14:03:37

One thing to note, is that a major problem is trying to treat 4 explicit streams as 2 this seems to be causing a lot of pain.

bhauman14:03:08

If we reified them as the 4 streams they are, a lot of problems may disappear?

cgrand14:03:32

What are you 4 streams? (I see 5)

bhauman14:03:00

I was just about to say 5

bhauman14:03:30

There are also explicit warnings in CLJS

bhauman14:03:37

Which makes 6

cgrand14:03:45

cmd-in/eval-in/result (shoud we split it?)/err/out

cgrand14:03:55

in fact way more

cgrand14:03:46

for example launch (future (while true (Thread/sleep 1000) (println “Hi Bruce”)))

cgrand14:03:54

and a couple others

cgrand14:03:10

are those futures all writing to the same out?

bhauman14:03:38

In a sense yes they are-ish

bhauman14:03:58

But I think it's worth thinking about it this way

bhauman14:03:30

If you reify streams of output

bhauman14:03:55

You can reify other captured streams

bhauman14:03:22

But it can possibly solve the multiplexing problems

bhauman14:03:48

Reify by creating a socket in this case

bhauman14:03:39

Or multiplexing over a socket

cgrand14:03:32

Excerpt from my outputstream:

[:out "Hi Bruce" 1][:out "\n" 1][:out "Hi Christophe" 2][:out "\n" 2][:out "Hi Bruce" 1][:out "\n" 1][:out "Hi Christophe" 2][:out "\n" 2][:out "Hi Bruce" 1][:out "\n" 1][:out "Hi Christophe" 2][:out "\n" 2][:out "Hi Bruce" 1][:out "\n" 1][:out "Hi Christophe" 2][:out "\n" 2][:out "Hi Bruce" 1][:out "\n" 1][:out "Hi Christophe" 2][:out "\n" 2][:out "Hi Bruce" 1][:out "\n" 1][:out "Hi Christophe" 2][:out "\n" 2][:out "Hi Bruce" 1][:out "\n" 1][:out "Hi Christophe" 2][:out "\n" 2][:out "Hi Bruce" 1][:out "\n" 1][:out "Hi Christophe" 2][:out "\n" 2][:out "Hi Bruce" 1][:out "\n" 1][:out "Hi Christophe" 2][:out "\n" 2][:out "Hi Bruce" 1][:out "\n" 1]

cgrand14:03:39

reified and multiplexed, no?

cgrand15:03:25

Some important updates to the spec: • section “Filling the gap” https://github.com/cgrand/unrepl#filling-the-gap • section “Message templates” https://github.com/cgrand/unrepl#message-templates

cgrand15:03:39

So I consider the core of unrepl complete.

pesterhazy15:03:12

I'm playing with the repo using

rlwrap java -cp $HOME/.m2/repository/org/clojure/clojure/1.9.0-alpha14/clojure-1.9.0-alpha14.jar:src clojure.main -i src/unrepl/repl.clj -e '(unrepl.repl/start)'

pesterhazy16:03:54

1. could unrepl send "\n"s after each message? that would make interacting as a human easier

cgrand16:03:43

each message or only after :prompt?

pesterhazy16:03:49

why not after each message?

cgrand16:03:29

each message then! I pushed the change

kotarak08:03:07

cgrand: nice. That makes it even vim usable. One transfer mode is line based.

cgrand09:03:03

How do you parse the line after that?

kotarak12:03:42

Painfully if EDN. Easily if JSON. But transfer with channels can already be troublesome. sigh

pesterhazy16:03:08

I noticed that I get a Java window, I image because you import java.awt.Image

pesterhazy16:03:09

I'd tend to say that rich results like images should be opt-in to avoid such surprising side-effects

pesterhazy16:03:04

"exit" doesn't seem to work ("Type exit to exit back to the original repl.")

cgrand16:03:18

java -Djava.awt.headless=true

cgrand16:03:29

exit is not mine 🙂

pesterhazy16:03:56

I mentioned exit because it's mentioned in the readme

pesterhazy16:03:55

playing with unrepl is fun, really liking what I'm seeing already

cgrand16:03:11

Will fix the README.

pesterhazy16:03:55

is there a way to enter the ^C char from a terminal?

cgrand16:03:24

^V is the escape sequence to prevent a terminal from

pesterhazy16:03:35

works well (not with rlwrap though)

cgrand17:03:17

Did you try ^q^c?

cgrand17:03:24

(Emacs-style)

cgrand17:03:56

CLJS specific command: set source type (clj cljc or cljs)

pesterhazy18:03:13

^Q^C also doesn't work - seems like an issue with rlwrap

cgrand18:03:50

It was a blind guess (I'm afk it's the parenting rush hours) cause readline is emacsy

richiardiandrea19:03:45

I have been following for a while now, good job folks 😉

thheller19:03:12

fun little anecdote why unrepl would be useful

thheller19:03:15

[[:fs-updates:fs-updates  [[{{:dir:dir  #object[#object[java.io.File java.io.File 0x3f7d50df 0x3f7d50df ""//UUsseerrss//zziilleennccee//ccooddee//sshhaaddooww--ddeevvttoooollss//ssrrcc//mmaaiin"n"], :dir-type :classpath, :name ]", s:dir-type h:classpath, a:name d"oswh/addeovwt/odoelvst/osoelrsv/esre/rwvoerrk/ewro/rikmeprl/.icmlpj"l, .:ext c"lcj", l:ext j"", c:file l#object[java.io.File j"0x772a31be , ":file /#object[java.io.File Use0x772a31be r"s//Uzsielresn/czei/lceondcee//schoaddeo/ws-hdaedvotwo-odlesv/tsorocl/sm/asirnc//smhaaidno/ws/hdaedvotwo/odlesv/tsoeorlvse/rs/ewrovrekre/rw/oirmkpelr./cilmj"]p, l:event .:modc}l]j"], :event :mod}]]]

thheller19:03:46

two go loops receiving the same message and doing a prn for debugging purposes 😉

thheller19:03:46

and that is over nrepl as well, threads are fun ...

cgrand19:03:52

Note to self: look into bindings conveyance for core async