Fork me on GitHub
#babashka
<
2019-12-07
>
borkdude10:12:20

Sci var feature is merged to master: https://borkdude.github.io/sci/#vars

sogaiu13:12:23

nice looking docs 🙂 apart from the ususal commenting out of part of the socket-repl tests and changing the python exe name, all tests pass here.

borkdude13:12:35

@sogaiu Given that the babashka tests still pass on CI linux: <https://circleci.com/gh/borkdude/babashka/2127> I'm not sure what's the problem here: <https://github.com/borkdude/babashka/issues/140>

borkdude13:12:02

Maybe you can try to run the socket repl and try to use it as you would normally?

holyjak13:12:32

@borkdude Will it be possible to get helpful stack trace that refer to my code and not just sci? When running https://github.com/holyjak/blog.jakubholy.net/blob/cryogen/content/tmp-migration/old/md/fix-files.clj I am getting

ERROR !!! for file #object[.File 0x104963f4 ./pages/2--about/nocv.md.asc] : #error {
 :cause renameTo failed
 :via
 [{:type clojure.lang.ExceptionInfo
   :message renameTo failed [at line 80, column 21]
   :data {:type :sci/error, :row 80, :col 21, :message renameTo failed [at line 80, column 21]}
   :at [sci.impl.utils$rethrow_with_location_of_node invokeStatic utils.cljc 66]}
  {:type java.lang.Exception
   :message renameTo failed
   :at [java.lang.reflect.Constructor newInstance Constructor.java 423]}]
 :trace
 [[java.lang.reflect.Constructor newInstance Constructor.java 423]
  [clojure.lang.Reflector invokeConstructor Reflector.java 305]
  [sci.impl.interop$invoke_constructor invokeStatic interop.cljc 29]
  [sci.impl.interpreter$eval_constructor_invocation invokeStatic interpreter.cljc 262]
  [sci.impl.interpreter$eval_call invokeStatic interpreter.cljc 318]
  ...
which is really not helpful. Otherwise bb is awesome!

sogaiu13:12:51

@borkdude nc -q doesn't work on some linux distributions

borkdude13:12:23

@holyjak Can you post an issue with what you expect other than it is now? [at line 80, column 21] points to the location in your script

holyjak14:12:29

Ah, sorry, my bad! I assumed the line/column refers to SCI code since only that is mentioned in the stack trace. I expected at fix-files.clj at the top of the stacktrace (and possible the function call chain within that file).

borkdude14:12:53

That could be a nice addition

sogaiu13:12:14

"netcat" implementations vary

sogaiu13:12:17

there are multiple

borkdude13:12:23

@sogaiu ah you scared me there, so it isn't really a regression in babashka itself 😉

sogaiu13:12:33

you must read carefully sometimes 🙂

borkdude13:12:24

Feel free to propose any improvements to the tests so they also pass on other linuxes than CircleCI linux and PR those. I just did what worked for the given environments I had

sogaiu13:12:57

fwiw, i did mention it before -- is it ok to drop -q? then you don't even need to check for mac?

sogaiu13:12:02

i don't know what '-q' does

borkdude13:12:44

we can try. I must have added it for some reason 😉

borkdude13:12:57

@sogaiu the -q option on circleci:

-q secs			quit after EOF on stdin and delay of secs

borkdude13:12:15

I think it wouldn't quit after EOF else

borkdude13:12:17

so maybe -w0 works

sogaiu13:12:45

thanks, i will try -w0 here first

sogaiu14:12:23

FAIL in (socket-repl-test) (socket_repl_test.clj:41)
ctrl-d exits normally, doesn't print nil
expected: (str/ends-with? (:out (sh "bash" "-c" (if mac? "echo \"(inc 1336)\" | nc 127.0.0.1 1666" "echo \"(inc 1336)\" | nc -w0 127.0.0.1 1666"))) "1337\nuser=> ")
  actual: (not (str/ends-with? "" "1337\nuser=> "))

sogaiu14:12:50

i don't know what circleci supports, but here there is:

$ nc --help | grep EOF
      --send-only            Only send data, ignoring received; quit on EOF
      --no-shutdown          Continue half-duplex when receiving EOF on stdin

sogaiu14:12:15

yes, i noticed 🙂

sogaiu14:12:29

well dropping -q works here (using --send-only also fails)

sogaiu14:12:15

iiuc, the netcat that's here is not entirely a niche implementation -- i think it is: https://nmap.org/ncat/

sogaiu14:12:42

is it possible to find out which implementation cirecleci uses?

borkdude14:12:56

I get:

$ nc -h
[v1.10-41+b1]
on circleci

borkdude14:12:52

maybe we can replace netcat with something JVM-y

borkdude14:12:07

this test is not about netcat, but about testing the socket REPL after all

sogaiu14:12:07

it isn't going to work on windows anyway

sogaiu14:12:39

i'll close the pr

borkdude14:12:03

does clojure itself ship with a socket repl client?

borkdude14:12:05

maybe we could just use that

sogaiu14:12:06

i have been confused by what you mean by "repl client" -- you mean something that will initiate a net conn and use that?

borkdude14:12:47

well, something that connects to the port and sends stuff

sogaiu14:12:56

ok, so diff from #137 right?

borkdude14:12:23

well, if we implement 137, we could use that 😉

borkdude14:12:41

I guess we can use tubular for the tests

sogaiu14:12:03

will investigate tubular

sogaiu14:12:15

(i still don't understand 137)

sogaiu14:12:28

clj -r gives a nice repl -- no network -- this is not functionality that exists in bb, right?

borkdude14:12:09

I changed the wording in the issue

borkdude14:12:49

I'm not totally sure about #137 though, since it will encourage people to type in a REPL prompt while you can better just connect from an editor. Or maybe an answer to 137 could be: use tubular.

borkdude14:12:32

Maybe building a native version of tubular could be nice

sogaiu14:12:48

on a similar note, possibly a portable minimal netcat

borkdude14:12:29

137 probably doesn't involve a socket server, but only exposing the REPL to stdout/stdin

sogaiu14:12:50

that sounds right

sogaiu14:12:51

an external tool that handles stdout/stdin <-> network translation sounds possible?

borkdude14:12:42

I'm afk for a bit

borkdude15:12:11

it seems like tubular doesn't quit on ctrl-d ;)]

borkdude15:12:29

PR in progress

borkdude15:12:02

@holyjak Merged your PR. Maybe you could give it a try locally again before I bake a new version of bb

holyjak16:12:43

seems to work fine!

borkdude15:12:10

@sogaiu maybe we can just borrow this stuff to test the server: https://github.com/borkdude/tubular/blob/master/src/clj_sockets/core.clj

borkdude15:12:35

I'll give it a shot

borkdude19:12:24

@sogaiu I've tried emulating netcat with tubular for an hour or two, but I can't get the Ctrl-D test working

borkdude19:12:53

tubular has a clj-socket namespace in it which might be used for lower level testing (which I also tried, but no luck so far)

borkdude19:12:30

This kinda works:

(defn socket-command [expr]
  (with-open [socket (<http://java.net|java.net>.Socket. "127.0.0.1" 1666)
              writer (io/writer socket)
              reader (io/reader socket)
              sw (<http://java.io|java.io>.StringWriter.)]
    (.write writer (str expr "\n"))
    (.flush writer)
    (.write writer (str ":repl/exit\n"))
    (.flush writer)
    (loop []
      (when-let [l (.readLine ^<http://java.io|java.io>.BufferedReader reader)]
        (binding [*out* sw]
          (println l))
        (recur)))
    (str sw)))

borkdude19:12:59

but I don't know how to send ctrl-d instead of :repl/exit to verify that ctrl-d works

borkdude19:12:08

instead of :repl/exit

borkdude19:12:41

maybe by closing the writer?

(defn socket-command [expr]
  (with-open [socket (.Socket. "127.0.0.1" 1666)
              reader (io/reader socket)
              sw (.StringWriter.)]
    (let [writer (io/writer socket)]
      (.write writer (str expr "\n"))
      (.flush writer)
      ;; (.close writer)
      (.write writer (str ":repl/exit\n"))
      (loop []
        (when-let [l (.readLine ^.BufferedReader reader)]
          (binding [*out* sw]
            (println l))
          (recur)))
      (str sw))))

borkdude19:12:55

but closing the writer makes the reads fail

borkdude20:12:55

@sogaiu I put this code up in a socket-test branch

borkdude20:12:06

off to other things now

sogaiu20:12:03

tnx for trying - got head ache and was resting - will take a look hopefully soon

borkdude20:12:32

ooh take care, don't code too much 🙂

borkdude20:12:03

@sogaiu I went ahead and simplified the socket REPL tests without using netcat now, at the cost of deleting the ctrl-d test

sogaiu22:12:51

socket repl tests pass nicely 🙂

borkdude22:12:56

good to hear!

borkdude22:12:32

$ bb '(def x 1)'
#'user/x

sogaiu22:12:13

$ ./bb '(ns hi) (def x 1)'
#'hi/x

sogaiu22:12:14

$ ./bb "(ns foo) (ns bar) (binding [*ns* 'foo] (def x 1))"
#'bar/x

sogaiu22:12:48

just like clojure, right? 🙂

borkdude22:12:53

ah nice, that's consistent with clojure... I didn't even test that 😉

borkdude22:12:07

we should add a test for it in sci

borkdude22:12:15

(test/var_test.cljc)

sogaiu22:12:37

may take a look at the test in a bit