Clojurians Log v2

It looks like it is possible with Dart now. I would like to use it on the JVM if possible

that's probably a gap in the JVM impl. I'd be fine receiving an issue or PR for it

I will have a look

probably similar to :static-methods

since we know the classname at analysis time

what are you making if I can ask? (this always motivates me to continue working on things ;))

Trying to build a sandbox, originally for a web framework, but not sure anymore

:)

good answer. trying to build something I'm not sure what it's going to become anymore.

haha yeah not a great motivator maybe

experimental mode = the fun mode ;)

But for a proper sandbox the constructors need to be tackled anyway. I was now thinking about a filesystem abstraction where java.io.File needs to work, but not with all the priviliges from the JVM

yeah, a read-only java.io.File for example?

yeah and/or only subdirectory access. So no access to the parent directory

ah for an AI thing?

it could be used for that

right

People are using vms which are cheap these days, but maybe a Sci sandbox is cheaper

sure

@jeroenvandijk the native tests serialize everything by shelling out

so passing functions doesn't work

just skip it on native

ah i was just typing i have to fix graal. Thanks

I guess we also want to deny constructors like how you can deny :instance-methods? So when there is :closed true • and no :constructor then no constructor access • :constructor true => use normal constructor Otherwise • :constructor fn => call fn as contructor • :constructor true => use normal constructor

yes, I was also thinking about that. correct

I'm making the changes in the right place right, in analyze-new

or is this a question?

yes sorry, for confirmation

I think (Foo. ...) goes through analyze-new but it'd be good to check

if so yes

Ok did an update. I didn't work on any deny or override logic on the cljs or cljd part

fine. let's fix clj first

I added a question on gh

> why is there only one top level :closed for constructors? Ah, uh because there is only one constructor? And :constructor is never a map. But yeah what if you just want to deny the constructor 🤔

wait

can you give an example of what you have now and what would be the alternative? not sure what you mean with top level

No you are right, there is something missing. But i don't know yet what would be best.

{:class String
 :close true ;; <= top level
 :constructor true ;; or (fn [_])
 :instance-methods { :close true ;; <= section level
                     'length true }}

{:class String
 :constructor false}

you just add a function that throws I'd say

it's :closed btw

ah yeah true. In the tests i have :closed 😅

yeah probably the easiest indeed to do it via a function. I can add a test just to illustrate this. The behaviour is already there

ok cool

oops, sorry I forgot about your PR in the latest SCI release... too caught up with the JS JIT stuff ;)

Yeah no worries. I like your JS JIT stuff

I think this approach could work on the JVM as well

Maybe

I haven't looked into how it actually works, but is it something like estimating if the analyzed code can safely be evaluated with the system eval ?

yes

well kinda

haha it depends 🙂

it precompiles certain shapes e.g. for loops

but it's all stll integrated with the interpreter. some parts can be interpreted while numerical operations and loops can still be compiled

Can it work together with :interrupt-fn?

yes it does

cool, i have to learn how this works

Or maybe not 😅 It's pretty advanced low-level js code

Great you could make it work

I'm guessing with a clj version the thing that would be send to clojure's eval would be more clojury

It's all isolated to one little namespace (600 lines). The structure of the interpreter we already set up for this

In Clojure we could do a similar thing: produce clojure code that goes through eval, but calls back into the interpreter when it needs "escapes"

but the problem on the JVM is that many eval s can get you into some kind of problem since every eval produces a new classloader I think

Ah yeah I was just thinking that. So then you are thinking of combining it with Crema? I saw an adr about it in the commits

Crema could also work, but you would still need clojure's eval - or you could indeed do a direct byte code thing

crema is only interesting for native-images

here I was thinking about JVM usage

> since every eval produces a new classloader So this would produce some overhead? That would reduce the performance gains?

I don't know. and do many people use SCI on the JVM for sandboxing? is it worth the effort? also don't know :)

for JS it was certainly worth it

(and it only added 5kb gzip to the bundle)

Yeah good questions. Maybe once the idea is out people will start to report the desire

Longer term I would be interested in a performance tweak like this, but for now I'm fine i think

It mostly matters for hot loops

In a previous project I had my own very basic clojure interpreter where I was using eval in production. It was to create custom pricing functions. I didn't discover any problems at the time (it was high volume traffic)

But it was always something like (eval '(fn [~args] ... )) and then the function would be called during runtime

ah it was this discussion I think: https://clojurians.slack.com/archives/C03S1KBA2/p1681999289711929

Ah I see. Too bad he doesn't share some example code that is being eval-ed

each fn is compiled to a class

Ah: The issue was solved by adding JVM option -XX:MaxMetaspaceSize=512m. Turns out, code generated by (eval) doesn't have a GC root. So when we limit the metaspace, its GC is triggered as expected

In our production setup, I think we would have maximum 200 evalled functions at a time. With some updates, maybe 2000 a day. But then our instances were rolled over via autoscaling. So instances would not live much longer than a few days. Maybe we were lucky 😁

so maybe not an issue at all

Cool. Also the https://stackoverflow.com/questions/71447267/clojure-memory-leaks-using-eval is probably not how you would want to do it. It is calling eval a million times. Probably you would do it max only a few hundred times in code base at analyse time (depending on how big the code base is)? And then call the compiled function at runtime?

true, it's only a limited set of evals per analyzed node. but I guess people could call sci/eval-string millions of times

anyway, they could set the perm gen space and maybe jit should be opt-in

I now made it opt-out so people using SCI on JS will actually see the benefits sooner

Yeah true. Maybe sci/eval-string can have an internal counter for mal practise 🙈. Better practise would be to do (sci/eval-string "(fn [...] ..)})") , no?

opt-out is also a good strategy. I think it is hard to make everything fool proof

to re-use that fun? yeah, I don't know - it depends what you want I guess

That's how I do it mostly. Most things are static, most of the time. Doesn't need re-evaluation in a hot loop. In my cases so far I think

Maybe the clj JIT is interesting for https://github.com/datalevin/datalevin

it uses SCI in JVM (non graal)?

Yeah I don't know the details, but I think some query and transaction code goes through Sci https://github.com/datalevin/datalevin/blob/1f4fce0bfb03042f14880b042c9dba477c70e9b0/src/datalevin/interpret.clj#L52

So pretty hot loops I'm guessing

I mean, hot loops in SCI code. database functions usually don't contain hot loops

but re-use most of what is mapped in by the config

Yeah that's true