Fork me on GitHub
#sci
<
2020-06-30
>
patrick.farwick05:06:31

Hello Use case: I am trying to load a file that has a few functions that will add/remove/change data to a map. I am having some issues getting sci to load the file because I would like to use specter to navigate that map. The file loads in the load-fn function but when calling sci/eval-string with the {:load-fn load-fn} argument I get an error of

Caused by java.lang.Exception
   Could not require com.rpl.specter.
I tried to not include the library and use a binding of something like select to sr/select but from my understanding that works in the string itself but not the functions being slurped as source. I could post the couple of errors I ran into while trying a few different things here but didn't feel it was totally necessary yet. Am I missing something? Are there clever ways around this issue?

borkdude07:06:36

@patrick.farwick The issue is that your code probably somewhere does (require '[com.rpl.specter]) and sci tries to find that namespace but it can't find it in the options you gave it.

borkdude07:06:54

@patrick.farwick If you want this to work you'll have to provide those functions with {:namespaces {'com.rpl.specter {'select select}}}

borkdude07:06:13

:bindings is just a shorthand for {:namespaces {'user ...}}

lee15:06:44

Iโ€™m still having fun playing with sci as a way to run rewrite-cljc tests over a natively compiled rewrite-cljc lib. Iโ€™ve had some small successes, but am now wondering if/how this will work for parts of the rewrite-cljc API that are macros.

borkdude15:06:11

sci can do macros ๐Ÿ™‚

borkdude15:06:33

@lee you can try (sci.core/copy-var 'some-macro rewrite-clj/some-macro (sci.core/create-ns 'some-ns nil)) https://borkdude.github.io/sci/doc/codox/sci.core.html#var-copy-var

lee15:06:53

Oh. Ok! Thanks for the pointer, I shall give it a try!

patrick.farwick15:06:05

Hey @borkdude, I appreciate the help (I am pretty positive you answered my github actions / circleci question as well ๐Ÿ™‚ ) Would you expect

(sci/eval-string "(require '[com.rpl.specter]) (select [:a] {:a 1 :b 2})" {:namespaces {'com.rpl.specter {'select select}}})
to work? I get the error
CompilerException java.lang.RuntimeException: Unable to resolve symbol: select in this context, compiling:
`

patrick.farwick15:06:34

I wasn't completely done typing that but slack decided I was ๐Ÿ˜

borkdude15:06:25

if you get a compiler exception, then it's happening before sci even sees it

borkdude15:06:35

so a regular Clojure error in your program probably

patrick.farwick15:06:59

so my thought process after that was, okay, maybe even in the namespaces section it still needed {'select sr/select} because specter is also included in the file I was calling this from, but get the error:

CompilerException java.lang.RuntimeException: Can't take value of a macro: #'com.rpl.specter/select, compiling:

borkdude15:06:27

@patrick.farwick See the comment I just made to lread: try sci.core/copy-var

patrick.farwick15:06:26

Okay let me try that

lee16:06:57

Success with copy-var on macro, thanks @borkdude! Although I had to copy from the original macro and not the potemkin copy. Iโ€™ll dig into why thatโ€™s the case later.

patrick.farwick16:06:29

good to hear! I am still struggling a bit to get this to work but I think some of the nuances around this are alluding me

patrick.farwick17:06:24

Like this kind of works:

(sci/eval-string "(require '[edit :as e]) (e/select [:a] {:a 1 :b 2}))" {:namespaces {'edit {'select (sci/copy-var sr/select (sci/create-ns 'edit nil))}}})
in the sense that it seems to be passing specters select function to the edit namespace in the eval string, but then looking at specters select function it is immediately calling compiled-select* so then I get this error:
ExceptionInfo Could not resolve symbol: com.rpl.specter.impl/compiled-select* [at line , column ]  clojure.core/ex-info (core.clj:4739)
which does make sense if I am not able to include all of specter into sci? Is there a more general way to pull in a whole library? It seems as if calling {:namespaces {'com.rpl.specter {...}}} is more designed for passing specific functions from a library Maybe I am thinking about this problem the wrong way. Effectively I just want the functions in this edit.clj file. I think it would be fine running them in the core namespace if there was a way to load the functions into the ns, rather than trying to load the file and pass libraries in order to run the functions in sci.

borkdude17:06:25

@patrick.farwick A macro is simply a function transforming an s-expression into another s-expression. If the resulting s-expression calls compiled-select* then you'll also need to include that

borkdude17:06:22

@patrick.farwick Maybe it helps if you include select as a function? (fn [x] (select x))

borkdude17:06:48

I'm not familiar with that API, but if you do it like that, the expansion happens outside of sci. However, select won't be a macro anymore

patrick.farwick18:06:26

Hmm. I see what you are saying about the macro / s-expression piece. I am not totally sure what you mean by including select as a function. There isnt a particular need for select to be a macro. I just thought it was the only way to get specter's select into the sci eval string

borkdude18:06:27

What I mean is, you can try to lift the macro in a function, like in:

user=> (map #(when % %) [nil false 2 3 4])
(nil nil 2 3 4)

borkdude18:06:38

whereas (map when ...) doesn't work

borkdude20:06:15

@patrick.farwick You might be able to do something with ns-publics and copy-var

patrick.farwick20:06:39

Yeah, I was kind of thinking about that. So maybe more of a #beginners question (but to be fair I am kind of a beginner ๐Ÿ™‚ ), the functionality of sci is to contain the code that is being run outside of your core program right? I was toying around with just (load-file "edits.clj") into (vals (ns-publics 'edit)) and that seemed to get me the functions that I was looking for. I mean this whole process kind of revolves around RCE but maybe sci contains some of it? I do really appreciate the time and effort you put into helping.

borkdude20:06:25

What's RCE?

patrick.farwick20:06:37

remote code execution

patrick.farwick20:06:15

which, it is an internal tool pulling from a file in a location i expect, but i was thinking more of "good software engineering practices"

borkdude20:06:34

I'm not sure what exactly you are looking for, but the principle around which sci is built: it doesn't execute anything other than you give it, unless it's a pure function from clojure core which doesn't have noticable effects on the outside world.

borkdude20:06:58

So if you want to make it dangerous, you can, but by default it should be relatively harmless to execute random code

patrick.farwick20:06:09

Yes. exactly. that makes sense.

borkdude20:06:10

For inspiration you can also look at the other projects that are using sci. The "biggest" one is probably babashka.

borkdude20:06:09

These projects are listed in the README of sci

patrick.farwick20:06:41

Yeah, I was looking through a few of them last night along with just searching for projects that included sci across github but was a bit stuck still

patrick.farwick20:06:41

I think my trouble has to do with the fact the "dangerous" piece i need to import is effectively a whole library.

borkdude20:06:26

@patrick.farwick babashka has imported several whole libraries

borkdude20:06:49

so has bootleg, spire.

borkdude20:06:05

it takes work, but doable.

patrick.farwick21:06:19

Okay, let me check again and try to understand what is going on better

patrick.farwick21:06:53

Okay, Well I think i found the couple of examples. I am trying to wrap my head around them in order to give it another try lol