Fork me on GitHub
#unrepl
<
2017-05-16
>
plexus08:05:45

^^ quick and dirty version of an Emacs unrepl client

plexus08:05:04

note that this is literally the result of an evening+a morning hacking around, with little to no attempt to do things "right". Consider it alpha/experimental.

plexus08:05:13

that said, do try it out of course 😉

pesterhazy08:05:15

can you eval a buffer?

pesterhazy08:05:22

trying it right now

plexus08:05:32

no, no buffer or in-buffer eval stuff is there

pesterhazy08:05:31

that would be cool at some point

pesterhazy08:05:13

I'm curious if the features to display correct line/column etc work well for features

plexus08:05:21

sending stuff to the process from a buffer would be easy to add, but to display results in the buffer instead of in the repl will require a lot more book keeping

pesterhazy08:05:51

hm results in the minibuffer below like inf-clojure?

plexus08:05:08

also no support for multiple repls/connections at this point, it just hard codes the repl buffer name

plexus08:05:51

like I said this is a proof of concept, it will need a bit of reworking to better handle its state management

pesterhazy08:05:54

sorry I'm a newb after 2 years of emacs, how do I load this?

plexus08:05:05

yeah look at the comments at the bottom

plexus08:05:25

do an M-x eval-buffer to load the code

plexus08:05:12

then do (unrepl-start '("lein" "run" "-m" "unrepl.repl/start"))

plexus08:05:35

or (unrepl-start '("telnet" "localhost" "3848")) if you're running on a socket repl

plexus08:05:44

then switch to the *unrepl-repl* buffer

plexus08:05:29

you can inspect the output coming from unrepl in the *unrepl-output* buffer

pesterhazy08:05:30

had to start using this command

JAVA_OPTS="-Dclojure.server.unrepl={:address,\"127.0.0.1\",:port,3848,:accept,unrepl.repl/start}" lein run -m clojure.main/main

pesterhazy08:05:25

also telnet didn't worked, I replaced it with nc instead

pesterhazy08:05:34

which is better anyway I think

plexus08:05:36

yeah that's what I did as well. the other should also work assuming it's in the right directory

pesterhazy08:05:54

note the commas instead of spaces

pesterhazy08:05:03

some weird quoting issue with lein I think

plexus08:05:25

yeah nc is actually cleaner, I had to add some code to skip past telnet's initial output

plexus08:05:43

ah interesting, must be a dfference in shell logic

plexus08:05:56

I'm uzing zsh on linux

pesterhazy08:05:07

here it's zsh on osx

plexus08:05:45

anyway good to know, I'll try to get some kind of unrepl-jack-in working at some point

pesterhazy08:05:44

what unravel does is to send the unrepl payload on connect

cgrand10:05:48

@pesterhazy I should definitely make your life easier on that point

pesterhazy10:05:05

You mean the multiple namespaces for multiple clients?

dominicm10:05:08

@pesterhazy to confirm my understanding from code: you mirror unrepl into your local project?

pesterhazy10:05:41

Yeah for now

dominicm10:05:35

@pesterhazy I think it's a good idea really. Although maybe with some vendoring for the ns.

cgrand10:05:09

If you don’t mind I’ll explain what I have in mind out of the thread

pesterhazy08:05:18

so you don't need to run a specially prepared repl, any socket server should do

plexus08:05:52

ah I see, you upgrade a plain old repl to unrepl

plexus08:05:01

makes sense

pesterhazy08:05:32

it's not perfect yet, because it uses a single ns so multiple connections could interfere if they use different unrepl versions

pesterhazy08:05:57

but at some point we can gensym the ns name to rule out this possibility

plexus08:05:40

now the next cool thing would be to install unrepl if it isn't there yet

pesterhazy08:05:38

honestly I'd just ship the unrepl payload with the emacs module, so the two are always in sync

pesterhazy08:05:52

at least for now

plexus08:05:41

ah yeah so with "payload" you actually mean the full source, I was assuming you just sent a (unrepl.repl/start) but that makes sense

pesterhazy08:05:15

right I mean the two src files (print.clj and repl.clj)

plexus08:05:42

cool, I'll look into that

plexus08:05:53

now onto some non-hobby hacking 😉

pesterhazy08:05:53

really cool that your project shows than edn.el is up to the task

pesterhazy08:05:02

that's what you use, right?

plexus08:05:33

yeah works quite well. just have to get used to using vectors and hash-maps in elisp, you don't see those every day 🙂

pesterhazy08:05:35

there was some worry about that project (bbatsov mentioned it wasn't maintained or something like that, I don't recall)

plexus08:05:35

no problems so far, obviously haven't really pushed its limits yet. but I assume edn is stable and well defined enough

pesterhazy08:05:54

yeah well, not around the edges I think

pesterhazy08:05:02

like what is a valid symbol char etc

pesterhazy08:05:12

all those things are up for interpretation 🙂

cgrand10:05:48

@pesterhazy I should definitely make your life easier on that point

cgrand10:05:27

I believe that the general shape of unrepl is not going to change so it’s time to take care of other aspects, especially the upgrade from a standard repl (or from a nrepl)

cgrand10:05:20

I’m going to add a helper ns to unrepl (well to the reference impl of unrepl 😉) for clients to perform the upgrade

cgrand10:05:04

So a client won’t have to duplicate unrepl code unless it modifies it (and then yeah the ns should be changed)

pesterhazy11:05:31

not sure I understand

pesterhazy11:05:43

to my mind, shipping the unrepl code with the client is a virtue

pesterhazy11:05:05

because it means that the target system doesn't need to be instrumented

cgrand11:05:09

you ship it (as a dep not as a copy)

pesterhazy11:05:27

ah yes, that makes sense

dominicm12:05:21

shipping it as a dep means you become part of the jvm though, unravel is independent. So is cider/vim/etc.

cgrand12:05:40

there are resources files, meant to be sent over the wire

pesterhazy12:05:40

@dominicm, I can pull in the dependency as part of the unravel build process, no problem. Shouldn't affect users

dominicm12:05:25

Hmm, I think I'm missing something. How does that work @pesterhazy?

pesterhazy12:05:03

if you install unravel via npm install -g unravel-repl, that will include the repl.clj + print.clj from the unrepl repo

cgrand13:05:00

# How to upgrade a repl to unrepl 

(Audience: repl client authors)

Once you have a connection to a REPL, you should send the content of the resource file `unrepl/upgrade/blob.clj` to it.

Once sent, start reading the output and wait for vectors. Here are the vectors you may get:
 * `[:unrepl.upgrade/require some.ns.name :some/thing]` then send the content of the ns identified by `some.ns.name` to the repl, followed by `:some/thing` to delimit the end of the ns.
 * `[:unrepl.upgrade/failed "pr-str of an exception"]` something went wrong you are back in the plain repl
 * `[:unrepl/hello ...]` success!
 * anything else should be ignored.

cgrand13:05:29

So the blob performs an inversion of control to request missing resources to the client.

pesterhazy13:05:11

I already do this

pesterhazy13:05:25

I mean except for the error handling 🙂

cgrand13:05:47

ok cool, anything you do that I don’t?

pesterhazy13:05:19

I wait for the message [:unrepl/bye] and stop interpreting the stream as edn after that

pesterhazy13:05:15

because in an upgraded repl, I get the regular user=> prompt after the end

cgrand13:05:42

I only documented how to upgrade to unrepl (how to get to unrepl/hello), what happens after is not for UPGRADE.md

pesterhazy13:05:00

DOWNGRADE.md

pesterhazy13:05:27

but seriously this is useful information

pesterhazy13:05:56

can you elaborate on unrepl.upgrade/require?

pesterhazy13:05:02

I can't quite parse that sentence

cgrand13:05:05

ok so basically the blob upgrades to a short-lived “upgrade-repl” before upgrading to unrepl/start

cgrand13:05:48

this “upgrade-repl” is the one emitting the :unrepl.upgrade/* messages

pesterhazy13:05:56

why do we need this?

pesterhazy13:05:08

I mean, we want to lower the barrier for tooling authors

pesterhazy13:05:38

I thought it was great that @plexus could write a client in an evening

pesterhazy13:05:10

so I think one thing that would be cool would be to aim to keep that and make additional features optional

cgrand13:05:40

The other option was to systematically send the two files.

pesterhazy13:05:23

Which files do you mean? repl.clj and print.clj?

cgrand13:05:21

JVM client can do:

(defn upgrade!
  “Upgrades a repl to unrepl. Upon success the first form read on the input stream should be
   a :unrepl/hello message.
   in is the input stream from the repl (so its *out*)
   out is the output stream to the repl (so its *in*)”
  [in out]
  (binding [*in* in
            *out* out]
    (-> “unrepl.upgrade.blob” resource slurp println)
    (loop []
      (let [x (read)]
        (case (when (vector? x) (first x))
          :unrepl.upgrade/require
          (let [[_ ns eom] x]
            (-> ns resource slurp println)
            (println eom))
          :unrepl.upgrade/success true
          :unrepl.upgrade/failed (throw (ex-info “Can’t upgrade.” {:reason (second x)}))
          (recur))))))

cgrand13:05:46

@pesterhazy is your code for upgrading the repl simpler?

pesterhazy13:05:42

hm I just use the string [:unrepl/hello as a sentinel

cgrand13:05:15

you blindly send the two files? and wait for the sentinel?

pesterhazy14:05:00

yeah exactly 🙂

cgrand14:05:27

So what I propose is a replacement for

pesterhazy14:05:23

yeah that was definitely just an ad hoc concatenation of the files

cgrand14:05:53

maybe maybe maybe you are right

cgrand14:05:02

I think I’m going to write a custom lein task to create the blob by concatenation

pesterhazy14:05:27

sounds good. Does that handle renaming the namespaces too?

cgrand14:05:38

why rename?

cgrand14:05:51

(if you don’t change them :-))

pesterhazy14:05:21

to avoid conflicts between, say, unravel and emacs's unrepl-mode?

cgrand14:05:14

there’s no conflict if none patched unrepl

cgrand14:05:20

did you patch it?

pesterhazy14:05:49

I did (to add a :unrepl/bye action), but that can be easily upstreamed

cgrand08:05:08

pesterhazy: I don’t find your change in the unravel repo

pesterhazy14:05:06

but inevitably at some point two clients will use different versions of unrepl, no?

pesterhazy14:05:19

so they might overwrite each other

pesterhazy14:05:56

creating a race conditions - who connects last wins

cgrand14:05:56

there are two cases to consider: • two different versions of unrepl (one older than the other) • two forks

cgrand14:05:26

First case I can deal with it in the blob (and developing unrepl with no breaking change)

pesterhazy14:05:50

hm yeah but that seems like you're creating more work for yourself

pesterhazy14:05:00

instead of just having two independent versions of unrepl

cgrand14:05:01

Second case: bad bad forker & unrepl should provide ways to be extended without forking (eg for actions)

plexus20:05:11

cgrand: third case : you're working on unrepl itself over an unrepl session, and want to change things without interfering your running session

cgrand06:05:00

@plexus I didn't thought of it but yeah definitely

cgrand14:05:20

I don’t know

pesterhazy14:05:30

I agree with backward-compatibility as a goal

cgrand14:05:33

maybe yes I should just rename for now on

cgrand14:05:49

and consider a more satisfying solution later

pesterhazy14:05:58

that sounds like a good plan

cgrand14:05:20

or I can spend all my time devising a clever solution (which qualifies as an advanced form of procrastination)

pesterhazy14:05:24

maybe unrepl turns out to be super stable, so we realize soon enough those worries are all unfounded

pesterhazy14:05:37

some of those advanced procrastination experiments have already yielded fantastic results, like the ellipsis feature 🙂

richiardiandrea15:05:52

Cool Emacs mode 😀😀

richiardiandrea15:05:25

Starring the file

cgrand15:05:14

@pesterhazy congrats for scoring an unravel talk at euroclojure

pesterhazy16:05:36

are you coming for the conference @cgrand? how about everyone else?

pesterhazy16:05:04

it might be nice to get an unrepl unsession organized 🙂

richiardiandrea16:05:39

@pesterhazy I am going to be there

richiardiandrea17:05:14

still trying to find the time to work on a universal js/cljs repl...man, I wish I had some bureaucracy to take care of 😄

dominicm17:05:47

I'll be there

dominicm17:05:00

@pesterhazy @cgrand something we could adopt, is that there's some build process for mangling the ns of the unrepl clj files during build. Then it would be unique for each project: no clobbering

pesterhazy18:05:46

@richiardiandrea structured procrastination eh?

pesterhazy18:05:05

we should meet up in Berlin when you guys are here and talk repls 🙂

richiardiandrea18:05:24

oh that's for sure

pesterhazy18:05:56

@dominicm, yeah I think that would be a good option

dominicm18:05:37

Vital.vim does this, which I find infinitely curious

cgrand08:05:08

pesterhazy: I don’t find your change in the unravel repo