Fork me on GitHub
#unrepl
<
2017-07-27
>
richiardiandrea06:07:05

@dominicm that is a very interesting question and I am not a lawyer but it seems like we are not reditributing/copying things but taking and running the files only

pesterhazy07:07:39

@dominicm do you mean we should bundle the license with the blob?

dominicm07:07:37

@pesterhazy potentially yesyes. This is what JavaScript minifiers do (see most cljs advanced builds contain the react license)

pesterhazy07:07:14

I think that would make sense

richiardiandrea07:07:52

I am thinking of repackaging https://github.com/Lambda-X/boot-pack-source, maybe a rename to boot-sources makes sense but naming is hard

richiardiandrea07:07:58

suggestions? 😄

richiardiandrea07:07:28

the lib contains a couple of packages for moving (source) files within the fileset

richiardiandrea07:07:49

like pack-source takes deps and moves the sources files to a custom folder

richiardiandrea07:07:11

and this will include an unrepl-blob task for making the blob like so boot pack-source unrepl-blob

pesterhazy08:07:22

what would be the advantages over what @cgrand has already built?

dominicm08:07:43

I like the names: uglifyclj, replpack and replify, named after their js counterparts 😛

richiardiandrea08:07:48

@pesterhazy how do you mean? did you see the snipped above? I am basically blobifying/concatenating arbitrary deps

pesterhazy08:07:31

@richiardiandrea so this is for building a blob that includes, e.g. compliment?

pesterhazy08:07:41

sorry I'm being a bit slow today

richiardiandrea08:07:58

no no problem, I thought I was missing some point 😉

pesterhazy08:07:07

have you tried sending it to the repl?

pesterhazy08:07:13

e.g. using netcat

richiardiandrea08:07:21

not yet, today's goal

pesterhazy08:07:26

(cat unrepl-blob.clj compliment-blob.clj - ) | java -jar clojure.jar -m clojure.main)

richiardiandrea09:07:25

uhm first attempt is weird:

cat target/unrepl/blob.clj | java -cp "$CLOJURE_EXT" clojure.main 
Clojure 1.8.0
user=> #'compliment.core/documentation
user=> 
$ ...exited to shell...

richiardiandrea09:07:23

it is evaluated correctly though, because the printed var is the last var of compliment.core

richiardiandrea09:07:57

Oh I need the - wow...bash magic

richiardiandrea09:07:12

$ cat target/unrepl/blob.clj - | java -Dclojure.server.repl="{:port 5555 :accept clojure.core.server/repl}" -cp "$CLOJURE_EXT" clojure.main 
Clojure 1.8.0
user=> #'compliment.core/documentation
user=> (compliment.core/completions "def")
({:candidate "def", :type :special-form} {:candidate "defn", :type :macro, :ns "clojure.core"} {:candidate "defn-", :type :macro, :ns "clojure.core"} {:candidate "defonce", :type :macro, :ns "clojure.core"} {:candidate "deftype", :type :macro, :ns "clojure.core"} {:candidate "defmacro", :type :macro, :ns "clojure.core"} {:candidate "defmulti", :type :macro, :ns "clojure.core"} {:candidate "definline", :type :macro, :ns "clojure.core"} {:candidate "defmethod", :type :macro, :ns "clojure.core"} {:candidate "defrecord", :type :macro, :ns "clojure.core"} {:candidate "defstruct", :type :macro, :ns "clojure.core"} {:candidate "defprotocol", :type :macro, :ns "clojure.core"} {:candidate "definterface", :type :macro, :ns "clojure.core"} {:candidate "default-data-readers", :type :var, :ns "clojure.core"})
user=>

richiardiandrea09:07:57

this is awesome 😉

dominicm09:07:09

Was the decision made to not to ns munging of some kind?

richiardiandrea09:07:49

no well this is my own trial and error, no decision has been taken

dominicm09:07:32

ah, okay. just checking.

pesterhazy10:07:43

just so you know, beer and champagne doesn't mix well

richiardiandrea10:07:02

@pesterhazy do you vendor the blob? or you build it programmatically on release

richiardiandrea10:07:30

I yes I see it is just vendored

pesterhazy10:07:33

correct, there's also scripts/update-unrepl

richiardiandrea10:07:59

oh well, that will come later...I guess at this point what will go in the blob will only depend on the client

richiardiandrea10:07:41

the script can change in order to use boot for building it

richiardiandrea10:07:51

uhm I tried and [:connect true] is the only output I get unfortunately

pesterhazy10:07:29

could you put up a branch?

richiardiandrea10:07:03

sure, on my fork? or you can give me access wink wink nudge nudge 😄

richiardiandrea10:07:00

so just to be sure I did:

cat ~/git/unravel/resources/unrepl/blob.clj - | java -Dclojure.server.repl="{:port 5555 :accept clojure.core.server/repl}" -cp "$CLOJURE_EXT" clojure.main
and it works.

pesterhazy10:07:33

looking at it now

pesterhazy10:07:41

how did you create the blob?

pesterhazy11:07:01

could you add the compliment blob as a separate file to the repo, rather than pre-concatenating?

richiardiandrea11:07:57

just a sec a bit busy now 😉 basically this is the core of it:

(let [code (->> blob-files (mapv slurp) str/join)]
            (butil/trace* "Blob:\n%s\n" (butil/pp-str code))
            (io/make-parents output-file)
            (butil/dbug "Writing blob to %s...\n" (.getName output-file))
            (spit output-file
                  (prn-str
                   `(let [prefix# (name (gensym))
                          code# (.replaceAll ~code "unrepl\\.(?:repl|print)" (str "$0" prefix#))
                          rdr# (-> code# java.io.StringReader. clojure.lang.LineNumberingPushbackReader.)]
                      (try
                        (binding [*ns* *ns*]
                          (loop [ret# nil]
                            (let [form# (read rdr# false 'eof#)]
                              (if (= 'eof# form#)
                                ret#
                                (recur (eval form#))))))
                        (catch Throwable t#
                          (println "[:unrepl.upgrade/failed]")
                          (throw t#)))))))

richiardiandrea11:07:37

haven't really check the code that was in the blob, so maybe there is something else to do there (for example I see a replaceAll now

richiardiandrea11:07:13

oh well, of course it does not work facepalm

richiardiandrea11:07:40

there are no unrepl specific blobs

richiardiandrea16:07:32

./scripts/run localhost 5555
Unravel 0.2.2 connected to localhost:5555

Type ^O for full docs of symbol under cursor, ^D to quit
Enter #__help for help

user=> (compliment.core/completions "def")
({:candidate "def", :type :special-form} {:candidate "defn", :type :macro, :ns "clojure.core"} {:candidate "defn-", :type :macro, :ns "clojure.core"} {:candidate "defonce", :type :macro, :ns "clojure.core"} {:candidate "deftype", :type :macro, :ns "clojure.core"} {:candidate "defmacro", :type :macro, :ns "clojure.core"} {:candidate "defmulti", :type :macro, :ns "clojure.core"} {:candidate "definline", :type :macro, :ns "clojure.core"} {:candidate "defmethod", :type :macro, :ns "clojure.core"} {:candidate "defrecord", :type :macro, :ns "clojure.core"} #__1)
user=> (print
print         printf        println       print-dup     print-str     print-ctor    println-str
print-method  print-simple  

user=> (print

richiardiandrea16:07:02

compliment is super cool, I added the option as comment in the code, neat stuff can be achieved (like complete locals, but the surrounding form needs to be sent)

richiardiandrea16:07:28

cmd-complete could actually accept the same options that we just forward to compliment

richiardiandrea16:07:44

@bbrinck has also a very cool library for error parsing that we could make use of in unravel: https://github.com/bhb/expound

richiardiandrea16:07:49

this opens the door to some interesting question, how to send blobs conditionally? Expound makes sense only when clojure.spec is loaded server side

richiardiandrea17:07:17

ah ! maybe I missed this one, you folks have thought about everything 😄 What is side-loading? Server side, client side? 😄

cgrand17:07:13

Side-loading is an idea that has been floating around for quite a while (at some point I even had an implementation). Basically it's a way for the server to request a resource from the client. (In java it hooks up the classloading mechanism)

richiardiandrea17:07:45

ok it is that inversion of control we were talking about yes, I just did not know the name then 😉

dominicm19:07:55

did those messages send? they're all grey?

dominicm20:07:18

Looks like no

dominicm20:07:31

What were the objections to the side loading?

cgrand09:07:52

Side loading requires a dedicated connection (and client code for this connection). This is the price.

dominicm09:07:15

I don't understand the dedicated connection requirement.

richiardiandrea10:07:42

I think that for solving the particular problem above, the blob that contains expound for instance, could refuse to eval itself if it does not find its requirements (`(find-ns 'clojure.spec.alpha)` for instance)

richiardiandrea10:07:20

Also the number of blob combinations is potentially infinite and we might want to share them (example, the compliment one)...this actually makes the new tools.deps.alpha a good candidate for fetching things off topic 🙂

dominicm10:07:16

or generally pomegranate, no?

cgrand10:07:10

@dominicm loading can occur anywhere even in the middle of a read so it can't share the same input stream

dominicm10:07:56

@cgrand gotcha, so unravel would be forced to maintain a separate (or multiplexed) connection in order to support this feature (if it wanted to)

cgrand10:07:51

@richiardiandrea I'm under the impression we are slightly talking past each other.

dominicm10:07:50

This does somewhat open up the desire for a multiplexer again I suppose. I know it was canned for complexity, but just raising the point that it would remove the scary of maintaining a dedicated connection.

cgrand12:07:09

@dominicm multiplexing was not canned for complexity but for not being core. It's one way to create multiple connections. So it's rather orthogonal and clients willing to use it should add it to their blobs.

dominicm13:07:21

So not necessarily a dedicated connection then

cgrand16:07:00

@dominicm it's still a connection (it needs to be handled separately by the client after demux)

richiardiandrea16:07:19

in particular I say I so so like unravel as client but I want to be able to send some additional blob apart from the ones unravel provides. Maybe unravel can expose an option for this use case? This way the user controls what to send (in case she is using clojure alpha-1.9.0-alpha17) to some extent

richiardiandrea16:07:50

Uhm or maybe a simple series of load-file might suffice once the repl is up

cgrand21:07:57

About the whole blob thing: maybe we make reasoning and problem solving harder by complecting blob generation and namespaces isolation (gensyms etc.)

richiardiandrea10:07:46

For now I am going to publish a task that does the following things: resolves the deps (already part of boot-sources), sorts in dep order the files, concatenates. One step at the time. We can refine things once problems come up.