Fork me on GitHub
#crux
<
2020-04-17
>
Jorin07:04:54

Since Crux is also experimenting with different query languages like SQL, SparQL and GraphQL, thought I gonna share this interesting Twitter thread here: https://twitter.com/simonw/status/1250803209871847426

jarohen07:04:35

ha, yes - we've been discussing this one on our internal slack 🙂

Jorin07:04:21

As far as I can tell the thread is still missing discussions around datalog 😉

mpenet07:04:15

apples vs oranges imho, graphql is source agnostic, resolvers can hit anything, a mix of dbs, services etc

Jorin07:04:18

yes, this is exactly what these comments in the Twitter thread look like 🙂 But some of them have some really interesting points and new input

mpenet07:04:49

well arguably sql could too, but it's highly tied to dbs right now

Uri10:04:55

maybe a silly question but if i want to say this in json: {:crux.db/id :zoe :closest-friends [:amy :ben :chris]}

Uri10:04:03

how can I do that?

Uri10:04:47

(namely differentiating between keys and strings when they are values)

Uri11:04:33

How are referencens handled in crux? Say I have a json I want to transact, and I want some values to actually be references to other entities, either already in the db or entities from this transaction (which have not been given an id yet) - can something like that be done or is everything just values and it's up to me to handle the ids?

Uri11:04:47

If it's up to the user - how does crux "know" how to find {name: "john", "age": 10} "people named "john"'s ages"

jarohen11:04:12

in Crux you specify the entity ID (it's the only schema we need in a document) - we don't give it an internal id, so you don't need anything from us to refer between documents, even in a transaction you haven't submitted yet Clojure keywords, UUIDs, URLs (amongst others) are all valid IDs

jarohen11:04:14

we also don't require you to define what's a 'reference' ahead of time - if a value happens to be the key of another document, you can join them

jarohen11:04:49

so, if you have the id of the entity in hand, you can get that entity out of crux using (crux.api/entity (crux.api/db node) :john) (or node.db(...).entity(...) if you're using the Java API)

Uri11:04:49

so I declare beforehand that "name" is my id field?

jarohen11:04:18

if you don't have the entity ID, and you want to ask 'for all the documents with name: "John", give me their ages', you'd use a query along the lines of:

(crux.api/q (crux/db node)
            '{:find [age]
              :where [[e :name "john"]
                      [e :age age]]})
(if "john" is user-input you'd use :args to avoid injection attacks)

jarohen11:04:11

ah, sorry, no - Crux requires that you specify a :crux.db/id value, so your document might be {"crux.db/id": <id>, "name": "john", "age": 10}

jarohen11:04:32

but you're free to choose how you uniquely identify your entities (using one of the valid ID types, above)

Uri11:04:36

and if my document is nested, i don't get any indexing done from nested values right?

jarohen11:04:16

in nested maps/objects, yep, that's correct, we don't index those

Uri11:04:26

but this is almost the same as just replacing the nested objects with their ids i imagine

Uri11:04:43

(how does crux know when a string is an id vs just a string, when it is in value)

Uri11:04:11

{... children: ["<id1>", "<id2>"]}

Uri11:04:02

(one motivation for this question is whether I'm limited when using json instead of edn, because my code is javascript/python)

jarohen11:04:16

unfortunately we don't allow arbitrary strings as ids as yet, they currently have to be converted into UUIDs, URLs or Clojure keywords - although we've got an issue in the pipeline to allow this in the near future

jarohen11:04:19

we're also working on how to best support those types within a JSON API - because of exactly the reason you've suggested there

Uri12:04:45

hmm seems like I need to wait 😞 I think I need at least these things to experiment with this: 1. Some basic tutorial on setting up e2e on kubernetes (not step by step, but generally something like - 1. install kafka 2. install rocksdb 3. take this docker image and give it a kafka url... etc') 2. Some examples on how to load and query data using json or alternatively client libraries for edn for python and javascript

Darin Douglass12:04:28

The EDN format GitHub has a doc with different language implementations: https://github.com/edn-format/edn/wiki/Implementations It looks like python and JavaScript both have a couple libraries to choose from.

Uri09:04:45

Thanks! So just these basic setup instructions then...

refset12:04:48

one solution for JSON that works right now is to use map IDs everywhere like {":crux.json/id": "my-string-id"}

Uri12:04:09

so {"crux.db/id": "<entity id>", friends: [{":crux.json/id": "<friend-entity-id>"}, ...]}?

refset12:04:01

Yep that will work, so long as cheshire (or whatever you use for edn<->json) reliably translates keywords when you prefix with : -- you would need a : in front of that crux.db/id Also note that ":crux.json/id" is just a convention I made up. The main corner case that I'm aware of with blinding converting between edn<->json for Crux is that edn sets can't be trivially represented, but in general once you start writing json into Crux this way I wouldn't attempt to pull it out again as edn and use it from separate code, i.e. just keep all usage of Crux json in & out