xtdb 2021-07-23 | Slack Archive

Samuel McHugh10:07:51

So I've popped up a bit the past few days. I'm trying out crux as my persistent storage for my project. I'm ignorant of the alternatives though and maybe there is something out there that is closer to what I need. I wrote a first draft of my project using a big, in-memory atom so a lot of my state update is done via swap. Now I want to move-on to something persistent so I reached for crux. I'm having to re-write quite a lot of code in the process and the translation isn't simple. I did find https://www.juxt.pro/blog/crux-dev-diary-june-2020#_transaction_functions but the main problem with this approach is that I'm not directly returned the new value for the entity. I have to go fetch it again out of the db. That doesn't feel very functional. Finally I also need to maintain synchronization between the server and client which I'm doing via web-sockets. the re-frame db on the client side is accessed in a fundamentally different way than queries in crux so it adds alot of mental overhead. Am I trying to fit a square peg in a round hole here? Is there some form of persistence that looks closer to in-memory atoms?

jarohen10:07:20

Hey @U0151JJA3FT 👋 Crux is fundamentally asynchronous, because of the architecture of different members of the cluster integrating via the shared transaction log, so there'll always be a little indirection between submitting the transaction and seeing its effects. It still essentially behaves like a reduce over a sequence of pure transformations - quite similar to how swap! behaves, I think, albeit async?

jarohen10:07:20

re re-frame: obviously depends on your application 🙂 but I'd guess the results of the entity/`pull` APIs (which are more document-oriented, rather than the Datalog queries which are more relational) would be closer in structure to what you're storing in the re-frame DB

jarohen10:07:49

> Am I trying to fit a square peg in a round hole here? I don't think so, at least, but I'm biased 🙂 > Is there some form of persistence that looks closer to in-memory atoms? Something more synchronous, maybe, if it's the asynchrony that's the main difference?

bbss13:07:58

@U0151JJA3FT there is https://github.com/jimpil/duratom

bbss13:07:46

wouldn't want to send you away from crux, as is it is an awesome database, but if all you need is a durable atom..

🙂 3

Samuel McHugh13:07:45

I need concurrency protections but yeah, I’m just awaiting every transaction. I guess something like manifold’s deferred let-flow blocks allow me to opt-in to a more functional-like setup because you can deref the deferred and treat it like a normal value. I did come across duratom in my search. I'll look at it as well. Since this is a personal project I think I'm going to take the opportunity to keep learning crux but it'll just require some adapting. I'm noticing that if I un-nest my data and make it more relational it fits much better. I just need to rethink my data's structure. Maybe that was kind of obvious. Thanks for the answers guys!

bbss13:07:33

once your data becomes more relational you'll find the query language + pull features very useful.

bbss13:07:15

I am really enjoying combining crux with fulcro, but that has a fairly steep learning curve.

🙂 3

Samuel McHugh17:07:37

Are there any docs on the ellipsis literal, or the square bracket query syntax? I'm trying to slightly adapt the example query from the test but I'm getting a Query didn't match expected structure error

;; example which works
(set (d/q '[:find [?name ...]
            :where [_ :name ?name]] test-db))

;; adapted version which doesn't
(crux/submit-tx my-node
                [[:crux.tx/put {:crux.db/id 1
                                :x #{1 2 3 4 5 6}
                                :foo :bar}]])

(set (q '{:find [x ...]
          :where [[e :x x]
                  [e :foo :bar]]}))

refset17:07:49

Crux doesn't support . or ... in the find vector currently (it's just nice-to-have sugar), but you can use a subquery to work around it

refset20:07:28

hmm, maybe subqueries isn't the answer, but the distinct aggregate does the job

{:find [(distinct v)]
 :where [[e :crux.db/id v]]} ; => #{[#{1 3 2}]}

Samuel McHugh08:07:56

thanks! distinct is definitely what I want here.

☺️ 3

2021-07-23

Channels