Fork me on GitHub
#unrepl
<
2017-06-07
>
plexus10:06:11

> I understand the concern but after some earlier iterations I came to the conclusion that the input stream must be left untouched. @cgrand could you say a bit more about why you came to this conclusion?

cgrand10:06:39

I considered two approaches: full framing (eg {:id id :code "code"} and interspersed “metadata” (adding stuff between forms).

cgrand10:06:08

interspersed metadata require the client to perfectly parse user input which is a strong requirement (and makes it brittle towards language change).

cgrand10:06:08

framing makes further upgrades difficult (transitioning from framed to streamed and (potentially) framed again) especially when some further input is already buffered. furthermore framing encourages to multiplex usages of a single connection, which adds complexity to the server. Plus in no time you are reinventing nrepl-sans-bencode 🙂

dominicm10:06:57

I've made that comment. EDN over socket as a new transport for nrepl could bring some of the benefits on unrepl.

dominicm11:06:19

But nrepl is fairly tied to bencode, and everything is tied to the limitations of the 4 data types that are there.

plexus11:06:39

thanks @cgrand, that makes sense

plexus11:06:02

yeah, I think those are reasonable arguments.

dominicm12:06:51

the problem still exists somewhat from what I can tell though, right?

dominicm12:06:16

as a curious idea, could unrepl provide a form that you can put code into? & that would essentially be (unrepl/with-id)?

dominicm12:06:30

e.g. (unrepl/with-id 1 (+ 1 1))

pesterhazy12:06:08

In unravel I just send [my-eval-id (form-to-eval)]

pesterhazy12:06:31

low-tech solution to track replies

cgrand12:06:02

*1? exceptions etc?

cgrand12:06:41

Is offset management that hard? (is my proposed solution too cumbersome?)

pesterhazy12:06:13

I feel I'm missing the context

cgrand12:06:43

@pesterhazy it may be something you have missed

cgrand12:06:15

now unrepl sends :echo messages right after read but before eval

cgrand12:06:55

these messages contains the offset (from the start of the unrepl session) and length of the form being evaluated

pesterhazy12:06:16

offset being the number of evals executed so far?

cgrand12:06:27

number of input characters

cgrand12:06:55

UTF-16 code units with normalized newlines to be precise 🙂

cgrand12:06:07

(aka don’t send CRLF)

pesterhazy12:06:30

that's for when I send (+ 1 2) (+ 3 4), that is, I'll get multiple :echos?

pesterhazy12:06:09

I think that's a useful feature

cgrand12:06:30

> (+ 1 2)(map inc (range 5))(str "hello" " " "world")
< [:echo {:from [1 1], :to [1 8], :offset 0, :len 7} 1]
< [:started-eval {:actions {:interrupt (unrepl.repl/interrupt! :session335 1), :background (unrepl.repl/background! :session335 1)}} 1]
< [:eval 3 1]
< [:echo {:from [1 8], :to [1 27], :offset 7, :len 19} 2]
< [:started-eval {:actions {:interrupt (unrepl.repl/interrupt! :session335 2), :background (unrepl.repl/background! :session335 2)}} 2]
< [:eval (1 2 3 4 5) 2]
< [:echo {:from [1 27], :to [2 1], :offset 26, :len 26} 3]
< [:started-eval {:actions {:interrupt (unrepl.repl/interrupt! :session335 3), :background (unrepl.repl/background! :session335 3)}} 3]
< [:eval "hello world" 3]
[:prompt {:file "unrepl-session", :line 2, clojure.core/*warn-on-reflection* false, clojure.core/*ns* <#C4C63FWP5|unrepl>/ns user}]

pesterhazy12:06:00

seems great for showing exactly where an error is

pesterhazy12:06:15

in the terminal I could show a little arrow e.g.

pesterhazy12:06:55

My problem was a different one. I was to correlate doc replies, i.e. I want to send something like (send-to-tooling-cx '(doc clojure.core/filter) {:callback show-result})

pesterhazy12:06:29

for that, I need to keep track of evaluations, But using a simple vector with an eval-id works well

pesterhazy12:06:46

Am I right in assuming that :echo wouldn't help for this case?

cgrand12:06:05

I think you are wrong in this case

pesterhazy12:06:22

I mean I get an echo number back

pesterhazy12:06:36

but then I would still have to correlate the echo response with what I sent

pesterhazy12:06:57

and in pathological cases, I might get an echo late, or not at all

cgrand12:06:26

when you send (doc cojure.core/filter) you know the offset in your outgoing tooling stream, so when you get the echo you can obtain the eval id

pesterhazy12:06:27

how do I know the offset? I just count how man bytes I've sent so far?

cgrand12:06:27

:echo occurs after read and before eval

cgrand12:06:45

couting chars would be better but yes

dominicm12:06:17

Oh, didn't realise echo gave back id. I thought it had, I was confused this came up again. Yeah, only "issue" is multiplexing.

dominicm12:06:27

which is a non-feature

cgrand12:06:05

so there’s no pathological case I think (I’d rather be proven wrong now than in 6 months)

pesterhazy12:06:13

hm I still prefer my simple-minded solution - it only requires me to keep track of one correlation, not two

pesterhazy12:06:37

anyway there's no downsides to allowing both approaches, right?

cgrand12:06:19

I’m not going to prevent you from evaluating what you want

pesterhazy12:06:07

I appreciate it 🙂

cgrand12:06:15

send-to-tooling-cx is local only or not yet written?

cgrand13:06:23

@pesterhazy did you consider processing messages in a slightly different way? Currently you dispatch on connection+tag first. I would dispatch on the group-id first.

pesterhazy13:06:36

sorry, I don't follow. What would the group-id be?

cgrand13:06:57

the third component of a message (the eval id if you wish)

pesterhazy13:06:38

what would I do with the group-id?

cgrand13:06:01

I’m thinking about the tooling connection

pesterhazy13:06:03

I mean that would be part of using :echo I guess - keeping associations of group-ids and callbacks

pesterhazy13:06:25

and outbound messages and group-ids

cgrand13:06:43

Hmm trying to sort out my ideas...

cgrand13:06:12

Let forget the dispatch order for now.

cgrand13:06:34

But about :echo: make send! takes a callback, the callback is called when :echo is received, and it sets the real callback in your callbacks atom.

cgrand13:06:44

my concern about dispatch order is that this “real” callback may be stateful (or have an explicit passed (reduce-style) state) and used to process all the messages (`:eval`, :outetc.) for one form

pesterhazy14:06:36

right that'll work

pesterhazy14:06:33

haven't run into issues with stateful callbacks yet

pesterhazy14:06:09

my plan was to worry about that when I start seeing related issues

pesterhazy14:06:27

do you have a concrete example of such an issue?

cgrand14:06:28

It's not much of an issue it's just that I think that it could be useful to process related messages together in stateful functions.

cgrand20:06:46

Given how :echo evolved I'm tempted to rename it :read

dominicm20:06:12

When evaluating, is it possible to mark the position in a file that it is? (This might be a general clojure question)

cgrand20:06:30

I'm not sure I understand. Do you mean saying "eval this form which was read at line 42 of foo.clj"?

cgrand20:06:53

You have to set a couple of dynvars for the file name. Line numbers are metadata to most read forms – but you can't explicitly set them in the source because reading overwrites them.

dominicm21:06:09

You understand correctly what I'm asking.

dominicm21:06:36

I don't fully understand what you mean about metadata. But I think it's important that they can be set.

dominicm21:06:04

So, as a client to unrepl, should I set the line no & column?

cgrand23:06:33

See :set-source