sci

orestis 2023-11-20T14:41:39.230869Z

Just joined to say what we've added an experimental automation workflow in our product, built on SCI. The ability to run the code against dummy data in the browser, and also against real data in the server, makes a really nice dev experience. Thank you @borkdude

👍 1
orestis 2023-11-20T14:43:12.269159Z

Well, also joined to validate an assumption that I have - if I provide an allow list to the evaluation options, that means essentially a very very restricted environment. The code we'd be running would be untrusted so we want to really limit the available constructs to some basic conditionals and then env-provided functions.

👍 1
Ingy döt Net 2023-11-20T15:29:55.175579Z

I implemented a load-file for the SCI context. foo.ys loads bar.ys But when I move both files into test/ directory, then test/foo.ys can't find bar.ys unless I specify test/bar.ys What's the best way to make sure the loaded file path is relative to the caller?

borkdude 2023-11-20T15:31:54.911639Z

Perhaps you can use sci/file (available as *file* inside of SCI programs)

borkdude 2023-11-20T15:32:18.369179Z

Every var has also :file as metadata

Ingy döt Net 2023-11-20T15:32:45.348259Z

(defn ys-load [file]
  (let [file (io/file file)
        ys-code (slurp file)
        clj-code (ys/ys-compile ys-code)]
    (sci/with-bindings
      {sci/ns @sci/ns
       sci/file (.getAbsolutePath file)}
      (sci/eval-string* (sci-ctx) clj-code))))

Ingy döt Net 2023-11-20T15:33:04.824349Z

I got that from the SCI readme

borkdude 2023-11-20T15:34:06.074039Z

you could maybe make load-file a macro, this way it gets expanded while sci/file is still bound to the file you're processing

borkdude 2023-11-20T15:34:21.626039Z

and then expand to something like load-file* as a function which then gets the absolute path

Ingy döt Net 2023-11-20T15:34:56.224809Z

oh, you think the path is wrong?

Ingy döt Net 2023-11-20T15:35:00.218209Z

I'll test that

borkdude 2023-11-20T15:35:34.745129Z

what do you mean, the path is wrong?

borkdude 2023-11-20T15:36:05.636349Z

The sci/file var is only bound during analysis

borkdude 2023-11-20T15:36:14.916509Z

and macros are processed during analysis

borkdude 2023-11-20T15:36:19.892079Z

function calls are too late

borkdude 2023-11-20T15:36:24.790989Z

hope you get the picture

borkdude 2023-11-20T15:57:51.403489Z

@ingy Here's a more full example (which I typed out in a REPL)

user=> (defmacro my-load-file [path] (let [current-file @sci/file] `(clojure.core/load-file* ~current-file ~path)))
#'user/my-load-file
user=> (defn load-file* [current-file path] [:do-your-thing current-file path])
#'user/load-file*
user=> (sci/binding [sci/file "user.clj"] (sci/eval-string "(defn foo [] (load-file \"dude.clj\")) (foo)" {:namespaces {'clojure.core {'load-file (sci/copy-var my-load-file nil) 'load-file* (sci/copy-var load-file* nil)}}}))
[:do-your-thing "user.clj" "dude.clj"]

Ingy döt Net 2023-11-20T16:09:04.793639Z

I did get the picture wrt binding time. And the load file path was indeed wrong for me. What I don't get is what sci/file or *file* does. To venture a guess, it affects how slurp works? or maybe how io/file works?

borkdude 2023-11-20T16:09:39.112899Z

sci/file works exactly the same as *file* on the JVM

Ingy döt Net 2023-11-20T16:09:58.622329Z

ok, but I don't know what *file* does

borkdude 2023-11-20T16:10:32.296719Z

https://clojuredocs.org/clojure.core/*file*

borkdude 2023-11-20T16:11:06.154869Z

this one has more explanation: https://stackoverflow.com/questions/12692698/file-variable-not-working/12693068

Ingy döt Net 2023-11-20T16:14:32.079929Z

I get *file* holds the current file path. What I'm trying to suss out is if the file functions resolve relative paths relative to *file* or relative to the current directory.

borkdude 2023-11-20T16:15:00.091659Z

never relative path

Ingy döt Net 2023-11-20T16:15:01.920559Z

but I can figure it out...

Ingy döt Net 2023-11-20T16:15:16.835389Z

?

borkdude 2023-11-20T16:15:25.121079Z

but if you want to implement it like that, you can use the idea I posted above

borkdude 2023-11-20T16:15:31.956299Z

which is not getting through I'm afraid

borkdude 2023-11-20T16:16:53.746759Z

functions can't use the value of *file* since that var is only bound during compilation time

Ingy döt Net 2023-11-20T16:17:03.236149Z

I'm saying if *file* is set to /tmp/foo/aaa.clj where does slurp think the "bbb.clj" would be?

borkdude 2023-11-20T16:17:06.682449Z

so you have to use a macro if you want to capture the file during compilation time

borkdude 2023-11-20T16:17:22.387429Z

slurp is a function and it never uses *file*

borkdude 2023-11-20T16:17:46.480059Z

in general JVM Clojure never uses the concept of "load something relative to this file"

borkdude 2023-11-20T16:18:02.868629Z

everything is based on classpath or current working directory, mostly classpath idea is central though

Ingy döt Net 2023-11-20T16:18:16.543049Z

ok so I need to do that math myself, after getting the starting point right

borkdude 2023-11-20T16:18:20.823019Z

but this doesn't prevent you from making your own load-file which does use the relative file idea, but then it has to be a macro

borkdude 2023-11-20T16:18:29.146219Z

like I explained above, so maybe read it again :)

Ingy döt Net 2023-11-20T16:18:44.475989Z

I fully understand the macro binding thing

borkdude 2023-11-20T16:18:49.278969Z

ok

borkdude 2023-11-20T16:19:22.985259Z

*file* is just some compile time thing. it is used to set the :file value on var metadata, which is probably the main thing

Ingy döt Net 2023-11-20T16:20:49.892939Z

I gotta run to the airport, but I think I have what I need now. Thanks!

borkdude 2023-11-20T16:21:01.134379Z

👍

borkdude 2023-11-20T16:21:06.762819Z

have a good flight!

✅ 1