Fork me on GitHub
#sci
<
2021-12-30
>
zane08:12:55

The link to the codox documentation in the README.md is broken.

zane11:12:45

I’m using sci in the context of osascript and I’m having trouble getting sci to evaluate some forms that work fine when compiled. For instance, this expression works when compiled:

(-> (js/Application "Safari")
    (.-windows)
    (.at 0)
    (.-tabs)
    (.at 0)
    (.url))
;; => ""
But the same expression triggers an error when evaluated by sci:
execution error: Error: #error {:message "Can't convert types.", :data {:type :sci/error, :line 3, :column 1, :message "Can't convert types.", :sci.impl/callstack #object[cljs.core.Volatile {:val ({:line 3, :column 1, :ns #object[cV user], :file nil, :sci.impl/f-meta {:ns #object[cV clojure.core], :name println, :sci/built-in true, :arglists ([& objs]), :doc nil}})}], :file nil, :locals {}}, :cause #object[Error Error: Can't convert types.]} (-2700)
I imagine this is the result of some subtle differences in how sci handles interop, but I’m not sure how to troubleshoot from here.

borkdude11:12:35

Can you make a repro? I'll be back in a few hours

zane11:12:04

Absolutely. Heading to bed now, but will tackle that tomorrow.

borkdude15:12:24

I have a repro:

$ osascript -l JavaScript out/main.js '(+ 1 2 3)'
:hello
:args #js [(+ 1 2 3)]
6
(ns 
  (:require [sci.core :as sci]))

(defn ^:export run [args]
  (println :hello)
  (println :args args)
  #_(prn (-> (js/Application "Safari")
           (.-windows)
           (.at 0)
           (.-tabs)
           (.at 0)
           (.url)))
  (sci/eval-string (first args)
                   {:classes {'js goog/global
                              :allow :all}}))

(set! (.-run goog.global) run)
;; shadow-cljs configuration
{:deps true
 :builds
 {:main {:target :browser
         :output-dir "out"
         :modules {:main {:entries []}}
         :compiler-options {:optimizations :simple}}}}
npx shadow-cljs release main

borkdude15:12:41

$ osascript -l JavaScript out/main.js '(def x (js/Application "Safari")) (def docs (.-documents x)) (some? docs) '
:hello
:args #js [(def x (js/Application "Safari")) (def docs (.-documents x)) (some? docs) ]
out/main.js: execution error: Error: #error {:message "Can't convert types.", :data {:type :sci/error, :line 1, :column 62, :message "Can't convert types.", :sci.impl/callstack #object[cljs.core.Volatile {:val ({:line 1, :column 62, :ns #object[sci.impl.vars.SciNamespace], :file nil, :sci.impl/f-meta {:ns #object[sci.impl.vars.SciNamespace], :name some?, :sci/built-in true, :arglists ([x]), :doc "Returns true if x is not nil, false otherwise."}})}], :file nil, :locals {}}, :cause #object[Error Error: Can't convert types.]} (-2700)

borkdude15:12:50

But I don't understand why it works like this

zane15:12:58

Weird, right? I’m puzzled too.

zane15:12:53

Ah, this is more clever than what I was doing:

(set! (.-run goog.global) run)

borkdude15:12:15

I'm under the impression that some of the behavior of osascript is "lazy" or so:

var safari = Application("Safari");
var docs = safari["documents"];
console.log(docs);

borkdude15:12:23

/tmp/osa.js: execution error: Error: Error: Can't get object. (-1728)

borkdude16:12:05

@zane This is kind of what happens in SCI:

var safari = Application("Safari");
var docs = safari["documents"];
var doc = docs[0];
var docurl = doc["url"];
console.log(docurl.apply(doc));
That reproduces the error for me: /tmp/osa.js: execution error: Error: Error: Can't convert types. (-1700)

zane16:12:14

Ah, interesting.

zane16:12:15

That explains why my attempts to do something similar with aget and so on were unsuccessful.

borkdude16:12:27

You might be able to ask the internet why the above doesn't work

borkdude16:12:34

I would be curious to learn

zane16:12:24

I’ll do some digging.

zane17:12:10

My understanding is still fuzzy, but:

zane17:12:58

It appears that many of the values in JXA are “Object Specifiers”. They appear to be like “paths” or a “pointers”. They are not like regular JavaScript objects in that when you access a property on one you always get back another object specifier. When you finally call the specifier you get back an actual JavaScript object if the path refers to a value. If the path does not refer to a value you get the error we’ve been seeing.

zane17:12:22

>> Application("Safari").documents[0] instanceof ObjectSpecifier
=> true

zane17:12:55

>> Application("Safari").documents[0].foo instanceof ObjectSpecifier
=> true

zane17:12:09

>> Application("Safari").documents[0].foo.bar.baz instanceof ObjectSpecifier
=> true

zane17:12:22

>> Application("Safari").documents[0].foo.bar.baz()
!! Error: Error: Can't convert types.

zane17:12:07

This is true regardless of what you provide as arguments.

>> Application("Safari").documents[0].foo.bar.baz("anything")
!! Error: Error: Can't convert types.

zane17:12:18

So it would seem that when sci attempts to call a function the .apply returns an object specifier that refers to an invalid location. That object specifier, when called, triggers the error.

zane17:12:52

I’m wondering whether we can work around this by assigning .apply to the object specifier prototype.

borkdude17:12:37

hmm, worth a shot :)

zane17:12:48

Otherwise it may be that sci just isn’t a good fit for JXA, which is a bummer.

zane17:12:28

If this worked one could create whole MacOS apps powered by sci. simple_smile

zane17:12:44

>> Application("Safari").documents[0].url.__proto__["apply"] = Application("Safari").documents[0].url.__proto__["callAsFunction"]
=> [function anonymous] {
  "length":0,
  "name":"",
  "prototype":{"constructor":[function anonymous]}
}
>> Application("Safari").documents[0].url.apply()
=> ""
>> Application("Safari").documents[0].text.apply()
=> "About\nStore\nGmailImages\n\n\n\n  \nAdvertising\nBusiness\nHow Search works\nCarbon neutral since 2007\nPrivacy\nTerms\nSettings\n"
😆

borkdude17:12:34

nice hack :)

zane17:12:42

It’s definitely .apply that sci is using?

borkdude17:12:25

try using ["documents"] in your JS code as a closer approximation

zane17:12:03

Thank you!

borkdude17:12:48

or [k] with k = "documents", perhaps they optimize that direct string access to some special accessor or so

zane17:12:20

Hmm. I think a call other than .apply is triggering this.

borkdude18:12:03

@zane Perhaps you can run with a local/root sci and then insert some debugging statements in there

1
borkdude19:12:51

I've tried and added debugging here:

(js/console.log "getting field name" field-name)
             (aget obj field-name)]
      :

borkdude19:12:59

$ osascript -l JavaScript out/main.js '(def x (.-documents (js/Application "Safari"))) x'
:hello
:args #js [(def x (.-documents (js/Application "Safari"))) x]
out/main.js: execution error: Error: Error: Can't convert types. (-1700)

borkdude19:12:15

no output so perhaps it's happening already with the static field access of the global object

zane19:12:28

That should work, actually.

zane19:12:39

I think.

zane19:12:14

I think the error you’re getting there is triggered when the result is printed.

borkdude19:12:45

sorry I was wrong, I needed to restart my shadow server

borkdude19:12:47

$ osascript -l JavaScript out/main.js '(def x (js/Application "Safari")) x'
:hello
:args #js [(def x (js/Application "Safari")) x]
invoking method Application
Application("Safari")

borkdude19:12:07

$ osascript -l JavaScript out/main.js '(def x (.-documents (js/Application "Safari"))) x'
:hello
:args #js [(def x (.-documents (js/Application "Safari"))) x]
invoking method Application
getting field name documents
out/main.js: execution error: Error: Error: Can't convert types. (-1700)

borkdude19:12:17

printing the type of the resulting field value:

$ osascript -l JavaScript out/main.js '(def x (.-documents (js/Application "Safari"))) x'
:hello
:args #js [(def x (.-documents (js/Application "Safari"))) x]
invoking method Application
res function Object() {
    [native code]
}

borkdude19:12:23

$ osascript -l JavaScript out/main.js '(def x (.-documents (js/Application "Safari"))) x'
:hello
:args #js [(def x (.-documents (js/Application "Safari"))) x]
invoking method Application
res function Object() {
    [native code]
}
getting field name documents
res function Object() {
    [native code]
}
out/main.js: execution error: Error: Error: Can't convert types. (-1700)

borkdude19:12:38

so it seems the interop works ok

borkdude19:12:50

but when using the value then something weird happens

zane19:12:43

We have slightly different test harnesses, I believe.

zane19:12:51

❯ ./out/obb -e '(.-documents (js/Application "Safari"))'
Application("Safari").documents

borkdude19:12:54

That does work. But note:

$ osascript -l JavaScript out/main.js '(def x (.-documents (js/Application "Safari"))) (prn :foo) (def doc (aget x 0)) (prn :dude) (prn (.url doc))'
:hello
:args #js [(def x (.-documents (js/Application "Safari"))) (prn :foo) (def doc (aget x 0)) (prn :dude) (prn (.url doc))]
invoking method Application
res function Object() {
    [native code]
}
getting field name documents
res function Object() {
    [native code]
}
:foo
out/main.js: execution error: Error: #error {:message "Can't convert types.", :data {:type :sci/error, :line 1, :column 69, :message "Can't convert types.", :sci.impl/callstack #object[cljs.core.Volatile {:val ({:line 1, :column 69, :ns #object[sci.impl.vars.SciNamespace], :file nil, :sci.impl/f-meta {:ns #object[sci.impl.vars.SciNamespace], :name aget, :sci/built-in true, :arglists ([array idx] [array idx & idxs]), :doc "Returns the value at the index/indices. Works on JavaScript arrays.", :sci.impl/inlined #object[Function]}})}], :file nil, :locals {}}, :cause #object[Error Error: Can't convert types.]} (-2700)

borkdude19:12:27

It still prints :foo. But when you are going to do something with x then it barfs.

zane19:12:43

Instead of

(set! (.-run goog.global) run)
try just appending
function run(args) { return namespace.of.run(args); }

zane19:12:00

That should make (.-documents …) work at least.

borkdude19:12:16

That does work. See output above.

zane19:12:22

Ah, sorry. Missed that.

borkdude19:12:49

It's when you are doing something to that documents value, when it barfs.

zane19:12:25

:thinking_face:

zane19:12:41

Wait, this works for me:

(-> (js/Application "Safari")
    (.-documents)
    (aget 0))

zane19:12:47

❯ ./out/obb -e '(-> (js/Application "Safari") (.-documents) (aget 0))'
Application("Safari").(0)

zane19:12:14

It’s only this that fails for me now:

❯ ./out/obb -e '(-> (js/Application "Safari") (.-documents) (aget 0) (.url))'
./out/obb: execution error: Error: Error: Can't convert types. (-1700)

borkdude19:12:36

$ osascript -l JavaScript out/main.js '(-> (js/Application "Safari") (.-documents) (aget 0))'
:hello
:args #js [(-> (js/Application "Safari") (.-documents) (aget 0))]
invoking method Application
res function Object() {
    [native code]
}
getting field name documents
res function Object() {
    [native code]
}
Application("Safari").(0)

zane19:12:47

Excellent.

borkdude19:12:09

it's as if it builds some lens into the object which is evaluated by need or so

zane19:12:30

Something like that.

zane19:12:13

You can deref the “lens” by applying the value. e.g. In the osascript REPL.

>> Application("Safari").documents[0]()
=> Application("Safari").documents.byName("Google")

zane19:12:55

I think it should be possible to patch Sci to special case instances of js/ObjectSpecifier. I’m just not where to apply the patch yet.

zane19:12:11

Might be needed in several places.

borkdude19:12:34

That could work in a forked sci, special for obb

1
zane19:12:09

Or I think obb could just set! the appropriate methods before running.

borkdude19:12:35

yeah, if you can get away with it like that, then it would be an easy workaround

zane22:12:51

Wow, that was harrowing.

zane22:12:35

Getting this working.

zane22:12:45

But I think it’s good now!

borkdude22:12:27

oh really? :-D what did you do?

zane22:12:05

Needs cleaning up, but it should give you the idea.

zane22:12:52

Key insights are that object specifiers don’t have .apply, and will crash if map? is called on them.

zane22:12:16

❯ out/obb -e '(-> (js/Application "Safari") (.-documents) (aget 0) (.url))'
favorites://

🎉 2
borkdude22:12:52

btw I think you're doing one thing too many here:

+             (aget obj field-name)
+             (gobject/get obj field-name)]

borkdude22:12:06

either the first or the last can go

zane22:12:06

Whoops, yeah.

zane22:12:32

Let me see if it still works if I back out that change.

zane22:12:15

Still works.

zane23:12:06

I think this could be pretty useful to folks.

zane23:12:42

Do you have thoughts about how it should relate to Babashka? e.g I’d be happy to use a different name if you’d prefer.

borkdude23:12:02

btw, I think you might get away with a (set! ...) to replace the existing SCI impl if you don't want to fork.

borkdude23:12:43

I'm fine with obb, but perhaps you can come up with a better name, either way I'm fine with it

borkdude23:12:03

osciscript?

borkdude23:12:38

I'll let it up to you :)

borkdude23:12:22

That is for preventing the browser to hang. Just be aware you're relying on impl details, but as long as you have tests and compile with a specific version of SCI, it should be fine

zane23:12:32

Sounds good!

borkdude23:12:22

If you find there are (performance) improvements to be made in the interop, feel free to provide feedback.

1
zane23:12:42

“Is my video enabled on my Zoom call?”

#!/path/to/obb

(-> (js/Application "System Events")
    .-processes
    (.byName "")
    .-menuBars
    (aget 0)
    .-menuBarItems
    (.byName "Meeting")
    .-menus
    (aget 0)
    .-menuItems
    (.byName "Stop Video")
    .exists)

🎉 1
borkdude23:12:27

off to bed now. kudos!

zane23:12:36

Thanks for all the help!

zane23:12:41

(and for sci)

borkdude10:12:16

Sure thing! Perhaps it would make sense to host this under the babashka organization with you as the main admin/maintainer? I would be ok with this. But also fine if you want to host under your personal account.

1
borkdude17:01:56

Just let me know how you want to proceed and I'll set some repo up if necessary

zane21:01:13

@U04V15CAJ I think that hosting it under the babashka organization sounds great! Let’s keep the “obb” name. I like the symmetry with nbb.

👍 1
zane21:01:41

Also, candidly, I don’t have a ton of experience maintaining open-source projects. I’d be grateful any advice or mentorship you’d care to provide. simple_smile

borkdude21:01:46

Added you as an admin there and sure, I will mentor :)

1
gratitude 1
zane21:01:04

I’m ‘zane’ on GitHub, if that wasn’t already clear.

borkdude21:01:32

yes, that's the one I added as admin

1
teodorlu16:12:53

Possibly stupid question - but does a default sci environment have any IO access, or is it completely pure?

teodorlu16:12:26

Seeing that access to stdin/stiout must be provided explicitly (https://github.com/babashka/sci#stdout-and-stdin) I'm inclinded to believe that the default env is pure. But I'd like to be sure :)

teodorlu16:12:52

Also, are there docs other than the README? (not implying that there should be)

borkdude16:12:38

Yes, should be pure/free of mutations to the host unless allowed

👍 1
borkdude16:12:56

README is the main doc

👍 1
teodorlu17:12:54

great - thanks! That was what I hoped for.