Clojurians Log v2

@mkvlr has joined the channel

Hello 👋 We’re currently evaluating asami as our client-side triple store to complement datomic on the server and we’re excited about the durable storage on IndexedDB. It seems `zuko.util` basically brings in all of ClojureScript and has a significant size penalty https://github.com/threatgrid/zuko/blob/961ccaa484b30f28b625e5598b5288cd436f9b3c/src/zuko/util.cljc#L9-L14. Here’s the top offenders from our shadow-cljs build report in our (already pretty bloated) bundle. Is that intentional and required?

@kommen has joined the channel

When you say that it’s brought in all of ClojureScript, I take it that this is in Clojure, right?

If so, then no. I hadn’t expected that at all!

no, that’s in our ClojureScript bundle with :advanced

hmm

That’s probably because that namespace keeps a map to all the functions in cljs.core

These are made available at runtime for predicates in queries

but it’s not required for ClojureScript (where c-eval isn’t implemented), or is it?

If you do a query with (for instance): [:find ?e :where [:e :type :my-type] [?e :value ?x] [(> ?x 5)]] Then you’ve called for cljs.core/> The same goes for any predicate you use. It also applies to bindings like: [:find ?label :where [:e :type :my-type] [?e :value ?x] [(str "value=" ?x)]]

So it’s a runtime requirement

We actually limit which functions can be called, but that’s done in Asami. It could be done in Zuko to limit the number of things brought into the map

does asami only need those? (def raw-lookup {'= = 'not= not= '< < '> > '<= <= '>= >=}))

I’ll do some experiments

we also suspect known-namespaces might include the same functions several times under different keys (`'cljs.core` , 'clojure.core , "cljs.core", "clojure.core")

Yes, but that won’t bring anything else in

hmm, I don’t think I understand…

I’m AFK for a bit, sorry. But I’ll come back to this soon

no worries, I’ll play around with this in the meantime

https://github.com/threatgrid/zuko/pull/10

this get is down from 1.85 MB to 550 kb.

(I haven’t tested this yet, hope travis takes care of that 😼)

Wow. I had no idea that the previous code would create duplicate structures!

I’m thinking it might help to take the Asami whitelist and shift it into Zuko. I can’t see any reason why the fn-for would ever be used in a situation that would require anything that isn’t in that list

ah, asami does have a whitelist? I’m away for a bit now but can take a look later or tomorrow

we tried to be generous with it, so there are some things in there that may seem excessive

https://github.com/threatgrid/asami/blob/main/src/asami/sandbox.cljc

We’ve listed the disallowed functions, but they’re not referenced, so I can remove them

@delaguardo has left the channel

@andrea712 has joined the channel

great, thanks! I’ll continue on this tomorrow. Another thing I’ll try that should reduce the bundle size is just storing the derefed vars (i.e. functions) instead the whole functions (with docstring etc) in the map. Or do you use that metadata anywhere in asami?

never

I’m actually building the map manually right now 🙂

So it never has to reference the entire contents of the namespace

curious to find out where this leads us with the bundle size

what’s the reason behind disallowing -> etc in the sandbox? Because they’re macros?

yes

I’d like to have and or or in there, but can’t

ok, maybe another interesting use case for https://github.com/borkdude/sci, could also be an opt-in thing

for example we’re already using sci in our app, so it would be free in terms of bundle size

the whole interpreter only adds ~500 kb in our case

sci also has a :allow and :deny setting or you can just set {:namespaces {'clojurecore {'-> nil}}}

@borkdude does that even affect the bundle size then? I mean can you build a slimmer version of sci this way?

@mkvlr Those settings do not affect the bundle size, but you can get a slimmer bundle size if you copy sci/impl/namespaces.cljc to your project and just delete what you don't need

@briancabbott has joined the channel