Fork me on GitHub
#asami
<
2021-03-09
>
mkvlr15:03:13

Hello 👋 We’re currently evaluating asami as our client-side triple store to complement datomic on the server and we’re excited about the durable storage on IndexedDB. It seems `zuko.util` basically brings in all of ClojureScript and has a significant size penalty https://github.com/threatgrid/zuko/blob/961ccaa484b30f28b625e5598b5288cd436f9b3c/src/zuko/util.cljc#L9-L14. Here’s the top offenders from our shadow-cljs build report in our (already pretty bloated) bundle. Is that intentional and required?

quoll15:03:36

When you say that it’s brought in all of ClojureScript, I take it that this is in Clojure, right?

quoll15:03:55

If so, then no. I hadn’t expected that at all!

simongray15:03:09

won’t :advanced compilation get rid of it anyway?

mkvlr15:03:21

no, that’s in our ClojureScript bundle with :advanced

quoll15:03:45

That’s probably because that namespace keeps a map to all the functions in cljs.core

quoll15:03:22

These are made available at runtime for predicates in queries

mkvlr15:03:30

but it’s not required for ClojureScript (where c-eval isn’t implemented), or is it?

quoll15:03:11

If you do a query with (for instance): [:find ?e :where [:e :type :my-type] [?e :value ?x] [(> ?x 5)]] Then you’ve called for cljs.core/> The same goes for any predicate you use. It also applies to bindings like: [:find ?label :where [:e :type :my-type] [?e :value ?x] [(str "value=" ?x)]]

quoll15:03:18

So it’s a runtime requirement

quoll15:03:50

We actually limit which functions can be called, but that’s done in Asami. It could be done in Zuko to limit the number of things brought into the map

mkvlr15:03:34

does asami only need those? (def raw-lookup {'= = 'not= not= '< < '> > '<= <= '>= >=}))

quoll15:03:49

No. Those are just the quick and easy ones.

mkvlr15:03:13

I’ll do some experiments

mkvlr15:03:44

we also suspect known-namespaces might include the same functions several times under different keys (`'cljs.core` , 'clojure.core , "cljs.core", "clojure.core")

quoll15:03:19

Yes, but that won’t bring anything else in

mkvlr15:03:58

hmm, I don’t think I understand…

quoll15:03:47

I’m AFK for a bit, sorry. But I’ll come back to this soon

mkvlr15:03:02

no worries, I’ll play around with this in the meantime

mkvlr16:03:30

this get is down from 1.85 MB to 550 kb.

mkvlr16:03:58

(I haven’t tested this yet, hope travis takes care of that 😼)

quoll17:03:20

Wow. I had no idea that the previous code would create duplicate structures!

quoll17:03:00

I’m thinking it might help to take the Asami whitelist and shift it into Zuko. I can’t see any reason why the fn-for would ever be used in a situation that would require anything that isn’t in that list

mkvlr17:03:19

ah, asami does have a whitelist? I’m away for a bit now but can take a look later or tomorrow

quoll17:03:51

we tried to be generous with it, so there are some things in there that may seem excessive

quoll17:03:05

We’ve listed the disallowed functions, but they’re not referenced, so I can remove them

mkvlr17:03:32

great, thanks! I’ll continue on this tomorrow. Another thing I’ll try that should reduce the bundle size is just storing the derefed vars (i.e. functions) instead the whole functions (with docstring etc) in the map. Or do you use that metadata anywhere in asami?

quoll17:03:45

never

👍 1
quoll17:03:58

I’m actually building the map manually right now 🙂

quoll17:03:45

So it never has to reference the entire contents of the namespace

mkvlr18:03:42

curious to find out where this leads us with the bundle size

mkvlr18:03:59

what’s the reason behind disallowing -> etc in the sandbox? Because they’re macros?

quoll18:03:26

I’d like to have and or or in there, but can’t

mkvlr18:03:35

ok, maybe another interesting use case for https://github.com/borkdude/sci, could also be an opt-in thing

mkvlr18:03:00

for example we’re already using sci in our app, so it would be free in terms of bundle size

mkvlr18:03:31

the whole interpreter only adds ~500 kb in our case

borkdude18:03:27

sci also has a :allow and :deny setting or you can just set {:namespaces {'clojurecore {'-> nil}}}

mkvlr18:03:10

@borkdude does that even affect the bundle size then? I mean can you build a slimmer version of sci this way?

borkdude18:03:29

@mkvlr Those settings do not affect the bundle size, but you can get a slimmer bundle size if you copy sci/impl/namespaces.cljc to your project and just delete what you don't need

👍 1