Clojurians Log v2
datahike
Hey there 👋
I was attempting to use the Datahike JavaScript library in the browser, and I ran into an issue while using Vite (for bundling my JS).
There was a stack trace for "Something is not a function", i tracked that error down to a specific call being made for a file-system check (`fileExistsSync`).
After digging a little deeper, it seems that there is some code present inside Datahike (Environ I think) that checks for whether the require function is available, and if present it determines the runtime is nodejs (which is normally fine I think)
However, the Datahike library exports the browser version of Datahike as a CommonJS module, and I think Vite detects this and attempts to resolve the modules dependencies by providing the require function, which eventually causes the browser bundle to think it's running in a server environment.
That all being said, I think a potential fix would be to provide a ESM wrapper around the browser version, this seemed to resolve my issues when debugging locally.
Happy to help with this more, please let me know your thoughts
Hey @hagstromsg. That would be great! I have been working on the npm packaging lately to make this easier because I wanted to ship the browser playground for http://datahike.io. I thought I managed to make the latest npm packages not depend on the node file store anymore. Are you able to build the npm packages on your end. Maybe you can more directly see what went wrong.
Hi @whilo, yeah i can try building the npm packages locally to debug further.
From what i gathered so far, there are runtime checks that look for the presence of the require function. One specific area is the library Environ that's used to look for environment variables.
Here's the specific snippet: https://github.com/weavejester/environ/blob/aa90997b38bb8070d94dc4a00a14e656eb5fc9ae/environ/src/environ/core.cljc#L68-L78
What's happening on my side is that the require function is provided during bundling (with Vite), so the runtime checks get confused.
If I used a different module bundler, it's possible that I wouldn't run into this issue. Did you use a particular bundler for the playground?
I just compiled with shadow and bundled manually. I have created a PR and would fix it along these lines https://github.com/replikativ/datahike/pull/792, lmk whether this works and whether something should be done better. You can also open a PR against this branch if needed.
@whilo taking a look at this now
I think the ESM code is correctly be detected and used instead, though I ran into two different issues.
The first issues was related to Vite, it was able to detect and use the ESM definition, but it still attempted to process the require statements present in the code. Eventually, I found that i can disable this processing by configuring Vite to exclude the datahike dependency from it's optimizedDeps. This seems to allow for datahike to load on the page successfully.
(Btw, I double-checked if the previous CJS version of datahike could be excluded from optimizedDeps, and that did not seem work since it failed to process the CJS code)
The second issue I ran into because I was creating a database through the Datahike api, and it failed to create a database because of the UUID. At first i was providing a string, then I attempted to use uuid helper, but that did not work either. So I'm wondering if something was recently changed around UUIDs and whether the uuid helper needs externs or something
Yes, could be. Will have to take a look later.
If there is something I can do to get rid of the require problem, lmk.
yeah no worries, just thought i mention it in case I may have configured something incorrectly.
for the require issue, there's a possibility that we could use shadow-cljs to produce an esm target directly. This might help with the CLJS code that checks for require
The UUID is maybe a bit annoying. I use it as a global store id, which will allow you to sync and replicate the store in the replikativ ecosystem without problems of identification.
yea the UUID seems useful, im eager to try out the replikativ p2p stuff with datahike
Better vite/browser bundling support merged and released.
just wanted to leave a little update: i got the UUID stuff resolved, I seemed to be accidentally using two versions of datahike at the same time, which caused a weird issue with creating uuids, now everything is working smoothly
I'll leave a comment on the PR for adding an extra exports field for the "types" to be inferred with LSP
Saw it, thanks. I will add that.
https://github.com/replikativ/beichte/ 0.2.6 — Effect analysis for Clojure compiler pipelines
beichte infers effect levels for Clojure expressions and vars, useful for compiler builders who need to know what code is safe to transform (AD, GPU, SIMD, CSE, parallelization).
Four-level effect lattice: :pure < :local < :mutation < :io
Feature flags: :throws, :random, :reflects, :allocates
Pre-annotated registry: ~300 clojure.core vars + common Java methods out of the box
Compilation target budgets: ask "is this code compilable for GPU?" and get a yes/no answer
Source-level analysis: recursively reads .clj source from classpath, no AOT required
Explicit caching: no global mutable state — pass a context for compiler pipelines, get fresh analysis in the REPL
(require '[beichte.core :as b])
(b/analyze '(+ 1 2)) ;=> :pure
(b/analyze '(println "hi")) ;=> :io
(b/analyze '(swap! a inc)) ;=> :mutation
(b/compilable? :gpu '(+ 1 2)) ;=> true
(b/compilable? :gpu '(println "hi")) ;=> false
(b/compilable? :ad '(Math/sin x)) ;=> true
;; Explicit context for cross-call caching in compiler passes
(let [ctx (b/make-context)]
(b/analyze-var #'my-fn ctx)
(b/analyze-var #'other-fn ctx))tools.analyzer.jvm. Apache 2.0.
clojars org.replikativ/beichte
https://github.com/replikativ/beichte