Fork me on GitHub
#sci
<
2020-07-04
>
borkdude15:07:09

@lee Can I view your rewrite-cljc test somewhere? I might use it as inspiration for incorporating clojure.spec + test.check maybe at one point in bb

lread15:07:42

yeah, will make public at some point… currently am using local/root hacks here and there… after I get rid of those.

lread15:07:03

I am currently trying to track down why with-out-str is not working within a deftest when interpreted by sci. Will let you know more after I distill.

borkdude15:07:21

does it work when you run the code outside a deftest?

lread15:07:56

yeah, that’s the confusing thing, am gonna look at what macro expansion is doing.

lread15:07:52

BTW, my current native-image ram usage blows CircleCI 3.5gb but should fit GitHub actions.

borkdude15:07:49

@lee Can you repro similar behavior with bb?

$ bb "(use 'clojure.test) (deftest foo (is (= \"hello\n\" (with-out-str (println \"hello\"))))) (run-tests *ns*)"

Testing user

Ran 1 tests containing 1 assertions.
0 failures, 0 errors.
{:test 1, :pass 1, :fail 0, :error 0, :type :summary}

borkdude16:07:35

I see a similar blow-up with clojure.spec. It would be really nice we could avoid that

borkdude16:07:51

the image size also blows up. Maybe it's one or a few specific tweaks

lread16:07:10

It gets weirder. It does pass for the foo case. I’ll share the particulars once I figure out what is distinct.

lread16:07:42

Perhaps a booboo on my part somewhere. Dunno yet. One thing that is different about the namespace I am hitting in my test is that is does have a (:refer-clojure :exclude [print]) - will fill you in more as I learn more.

lread16:07:25

Last executable I built is 56mb. I’ve made zero attempt at reducing anything.

lread18:07:29

I expect my with-out-str confusion might be me not understanding how sci works here. I’ll work through an example with babashka. We know that the following works:

(def a (with-out-str (print "fiddle sticks")))
(println "captured:" a)
And will output:
captured: fiddle sticks
But let’s say I’ve added a namespace to babashka that prints to stdout:
(ns lilapi.hiya)

(defn i-do-print []
  (println "Oh hello"))
And defined how it should be exposed in babashka like so:
(ns babashka.impl.lilapi
  {:no-doc true}
  (:require [lilapi.hiya :as hiya]
            [sci.core :as sci]))

(def hiya-ns (sci/create-ns 'lilapi.hiya))

(def lilapi-hiya-namespace
  {'i-do-print (sci/copy-var hiya/i-do-print hiya-ns)})
And in babashka.main, added require for [babashka.impl.lilapi :refer [lilapi-hiya-namespace]] and referenced it namespaces map via 'lilapi.hiya lilapi-hiya-namespace. When I now interpret the following:
(require '[lilapi.hiya :as hiya])

(def c (with-out-str (hiya/i-do-print)))
(println "captured:" c)
I get the following output:
Oh hello
captured: 
And you’ll probably tell me this is normal.

borkdude19:07:52

@lee sci's with-out-str uses sci/out to capture output

borkdude19:07:36

the reason for this difference is that sci programs don't get access to Clojure vars, in order not to have detrimental and lasting effects on those.

borkdude19:07:29

let me create an example, I can understand this is confusing

borkdude19:07:18

@lee :

user=> (require '[sci.core :as sci])
nil
user=> (defn foo [] (println "yello!"))
#'user/foo
user=> (sci/eval-string "(with-out-str (foo))" {:bindings {'foo foo}})
yello!
""
user=> (sci/eval-string "(with-out-str (foo))" {:bindings {'foo (fn [] (binding [*out* @sci/out] (foo)))}})
"yello!\n"

borkdude19:07:14

so you need to write a little wrapper for functions that write to *out*, rebinding *out* to @sci/out

lread20:07:40

aha, thanks, will try when I get back to kb!

lread21:07:05

Thanks, that worked! I could add some guidance to the README under https://github.com/borkdude/sci/tree/master#stdout-and-stdin but I think I only currently understand the how and am not entirely clear on the why.

borkdude21:07:08

The why is that sci implements its own var type, hence it has its own in and out using that var type. The reason sci has its own var type is that sci is designed to be safe, scripts/program fragments can only mutate what you give them. If sci programs were to directly have access to vars, users could do all kinds of stuff to mess things up in the Clojure runtime. That's why the separation exists.

lread21:07:58

Right, that makes sense.

lread21:07:08

Let me see if I kind of understand the workings, in your README example

(sci/binding [sci/out *out*
              sci/in *in*]
  (sci/eval-string "(print \"Type your name!\n> \")")
  (sci/eval-string "(flush)")
  (let [name (sci/eval-string "(read-line)")]
    (sci/eval-string "(printf \"Hello %s!\" name)
                      (flush)"
                     {:bindings {'name name}})))
Type your name!
> Michiel
Hello Michiel!
You make *out* (and *in*) available to the sci interpreter, and then in your example from above:
(sci/eval-string "(with-out-str (foo))" {:bindings {'foo (fn [] (binding [*out* @sci/out] (foo)))}})
"yello!\n"
You map *out* to sci’s out for the interpreted code.

lread21:07:14

I think I can do a little PR for the README on this - if it is of interest to you.

borkdude22:07:03

- When you sci-bind sci/out to *out* then clojure.core/println from the SCI runtime has the effect of writing to whatever *out* is bound to - When you clojure-bind *out* to sci/out then clojure.core/println from the Clojure runtime has the effect of writing whatever sci/out is bound to

borkdude22:07:18

so there is a nice symmetry

lread22:07:36

It only makes sense to do the latter in interpreted code, correct?

borkdude22:07:55

The latter happens for example in the pprint link I sent earlier

lread22:07:33

Oh right that’s not interpreted. Kind of a bit mind bendy.

borkdude22:07:40

In interpreted code there is no sci/binding or sci/out, only binding and *out*

borkdude22:07:51

ostensibly, but these are backed by sci's implementations of these

borkdude22:07:02

one other reason the distinct var type exists is that users can create new (dynamic) vars in their programs and this should not be visible to the outside clojure runtime, so "no side effects", it only affects the sci ctx

lread22:07:51

sciandboxed

lread22:07:09

You do talk about this under Vars https://github.com/borkdude/sci#vars , maybe it just takes a while to digest.

borkdude22:07:11

Maybe the "why" can be clarified more with some words I posted here. PR welcome 🙂

lread22:07:31

I think I’d better sleep on it simple_smile then I’ll make an attempt. I feel like a diagram might even be useful?

borkdude22:07:21

sleep sounds good. ttyl

lread22:07:39

thanks for the interesting chat! g’night!