This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-07-16
Channels
- # announcements (3)
- # babashka (25)
- # beginners (71)
- # calva (18)
- # clj-kondo (52)
- # cljs-dev (94)
- # cljsrn (12)
- # clojure (33)
- # clojure-europe (52)
- # clojure-nl (2)
- # clojure-uk (27)
- # clojurescript (18)
- # clojureverse-ops (4)
- # datomic (64)
- # deps-new (27)
- # depstar (5)
- # events (5)
- # fulcro (5)
- # graalvm (12)
- # graalvm-mobile (82)
- # helix (2)
- # introduce-yourself (1)
- # juxt (5)
- # lsp (10)
- # malli (7)
- # missionary (1)
- # off-topic (41)
- # pathom (69)
- # pedestal (6)
- # re-frame (4)
- # reagent (8)
- # releases (9)
- # remote-jobs (8)
- # shadow-cljs (3)
- # sql (46)
- # tools-deps (44)
- # uncomplicate (1)
- # vim (83)
Just discovered this channel via borkdude’s tweet of the repo that is in the subject of the channel. Very cool! So since Asami https://github.com/threatgrid/asami has local storage capability, and can be graalvm compiled, any idea what would be needed for the following: - Asami graalvm compiled, but accessible from react-native ClojureScript app? I’m guessing that there would need to be some sort of binding so that Asami compiled could be interacted with from ClojureScript. Maybe somebody with a bit more graalvm experience can chime in on this? > Why would it be cool to have this? Just because we could have datalog database on mobile (with localstorage!). Now that I wrote the above, I guess the more succinct question is: what is needed to make graalvm compiled Clojure code accessible to react-native ClojureScript code? Thanks
I am currently using DataScript + React Native; All running in a JS thread, no “native”.
@dotemacs Another graalvm compatible datalog db is datalevin btw. So moar options :) But isn't Asami Clojure_Script_ compatible as well, could that be used from React Native + CLJS directly?
It could and it does work, but the local storage is not yet implemented on ClojureScript side. Only on Clojure side. This is a reply to @borkdude’s question above.
Right, I am using DataScript, hence my biased question… I rolled my naive DataScript durable storage.
Since we brought up this interesting question and because of some perf issues in DataScript, I am theoretically curious if running it under GraalVM can yield any perf. improvements.
(I realize my question probably has no clear answer, it all needs to be benchmarked 🙂 )
From what I understand, Datalevin doesn’t retain the history of transactions. So it’s slightly different to DataScript & Asami in that sense.
@dotemacs DataScript also does not retain history by design (you have to roll your own implementation)
When you say > I rolled my naive DataScript durable storage Do you mean that you read the data into DataScript from a file and the write the data to a file upon the app going into background?
So it would linearly slow down; but I’ve had 1000s of datoms and it hasn’t been an issue.
It’s for data that the user generates on one device, so it would take years for it to grow super big.
I don’t doubt it… I just wanted to stay as close to Datomic as possible, and DataScript more or less offers that.
So you have to be careful; I didn’t expect this initially and tripped me up. The DataScript home page oversells/oversimplifies the product, to put it mildly…
It was my own ignorance and lack of benchmarking. Other than that, I have had no issues so far.
I did look at mentat aka datomish which was kind of a re-write of DataScript with a Sqlite backend:
https://github.com/mozilla/mentat/tree/clojure
(note the branch there clojure
).
And then a subsequent re-write in Rust… which seems to compile but haven’t got it running on device yet. Not that I tried very hard.
@borkdude Apparently it’s complicated; There’s a number of issues on github brining up performance; To do it well, you need to get into the weeds of “database science”; I’ve dabbled in general database design and it gets deep 🙂
Query planning, etc. The datalog queries are very expressive and it’s not trivial to optimize them; the fact that you’re doing everything in one thread (JS) doesn’t help either, I think.
e.g. when you need to execute the same query over and over while the data hasn't changed
It’s a bit tedious but it mostly works; Basically only re-query things that you know have changed based on the most recent transaction;
One of the “proper” ways to achieve better perf. would be some version of differential dataflow (not trivial to add, afaik) https://timelydataflow.github.io/differential-dataflow/
I think the RN runtime of JavaScript core is a bit restricted as opposed to running in Safari… (just speculating as to the reasons)
@dotemacs mobiletest compiles the graalvm binary to a shared library. This shared library is then accessed from the mobile UI code. Not sure how to do this from React Native/CLJS. This probably relies on iOS specific APIs.
@borkdude So when you say “graalvm version”… Is that running in like a real native thread, or something like Sci? I apologize, still not very well versed in all the differences.
@raspasov what mobiletest does: it compiles everything to "real" native and then exposes hooks to SCI, so you can have dynamic behavior when you eval stuff in SCI.
SCI is an interpreter which you can use as glue code between your "real" compiled functions. Not everything is interpreted by SCI, only the glue code, let's say.
so if you have {:namespaces {'foobar {'foobar foobar}}}
and then eval (require 'foobar) (foobar/foobar)
in SCI, then the foobar call is exactly the same as outside SCI
I have to go through the mobiletest example again with the latest updates; thank you 🙂
When you compile DataScript/etc database to “real native”, I assume you’d have to communicate with it via async function calls with the React Native JS thread.
Right… when you say “shared library” that’s basically a regular method/fn call for other native code. Basically a “real” native library?
no, but JVM-based vs JS-based probably does: https://twitter.com/nikitonsky/status/1398080743025356805
The poor query performance of Datascript is mostly due to its lack of query optimization. I did some trivial optimization in Datalevin and the performance increases about 30% to 50% depending on dataset. I also have a plan to do a rewrite of the query engine in Datalevin. In any case, the slow performance of triple-store (compared with relational DB) is a well known problem, but that should not excuse Datomic and Datascript from doing any optimization at all. Currently, Asami does a decent job in query optimization, Crux claims to do that as well, but the effect may not be obvious from what I heard. Of course, this is just my opinion and I am obviously biased.
@dotemacs
> what is needed to make graalvm compiled Clojure code accessible to react-native ClojureScript code?
TLDR: You would need to make a native module, https://reactnative.dev/docs/native-modules-ios
Roughly, the steps required would be:
1. Compile your clojure code to a shared library with native-image
You need to explicitly mark functions that are available as part of the shared library API. Further, the API can only receive and return C data structures like ints, chars, pointers, structs, etc.
2. Create a native module that exposes the shared library API to your js code.
Not sure how it would affect performance or memory usage, but I assume there is some overhead in the bridge between JS and native code.
I think an interesting alternative would be to compile all your clojure code to native and talk to react native code directly. I've been looking at some of the UI options for mobile and I think one of the major complaints against react native is that the JS/native bridge is slow.
I figured that those would be the steps @smith.adriane but what I’m unsure of is this part: > You need to explicitly mark functions that are available as part of the shared library API How would something like that be done for Clojure functions that would then be compiled via GraalVM?
It looks like:
(defn compile-interface-class
([]
(compile-interface-class nil))
([opts]
(with-bindings {#'*compile-path* "library/classes"}
((requiring-resolve 'tech.v3.datatype.ffi.graalvm/expose-clojure-functions)
{
#'clj_print {:rettype :void
:argtypes [['bs :pointer]]}
#'clj_prn {:rettype :void
:argtypes [['bs :int64]]}
#'clj_eval {:rettype :int64
:argtypes [['bs :pointer]]}
#'clj_start_server {:rettype :void
:argtypes []}
#'clj_print_hi {:rettype :void
:argtypes []}
}
'com.phronemophobic.mobiletest.interface nil)))
)
where clj_print
, clj_prn
, etc are just normal clojure functions that receive and return the arg types listed