Fork me on GitHub
#datomic
<
2018-05-02
>
jjfine17:05:59

the datomic docs show a way to find the lengths of the 5 longest/shortest tracks in the database. is there a way also return the entity id of each of those tracks in the same query?

d._.b18:05:43

I'm a little confused on peer vs client library. I have a peer server and the transactor running. I'm using the peer library. Prior to this, I was simply running the transactor and directly connecting to it. The transactor and peer URLs are identical. How do I know if I'm accessing storage through a peer or directly through the transactor?

marshall19:05:07

Peers never connect through peer-server

marshall19:05:16

peer-server is only there to “serve” clients

marshall19:05:43

the URL you use is actually the Storage URI - Peers go to storage first, look up the active transactor, and connect to it

marshall19:05:04

when you connect with a client you use an endpoint, either your Datomic Cloud system endpoint or the address of the peer server

marshall19:05:47

https://docs.datomic.com/on-prem/getting-started/connect-to-a-database.html#repl - the middle of that section shows an example of connecting to peer-server with client

d._.b19:05:00

also, i figured out my issue yesterday. i was doing something silly with :db/unique on an attribute

đź‘Ť 4
marshall19:05:40

sorry i missed that - i would have asked for the schema

d._.b19:05:11

question for ya: is there anything out there for bulk import that's more general? the mbrainz-importer seems to have all the pieces, but it's not really built for use as a utility for any old dataset

d._.b19:05:19

i have a few million records, and i can cobble together the core.async stuff, but it'd be nice if i could just hand my transit to a bulk import utility, even if the only guarantee is that it gives me "slightly better than sequential transactions" performance

d._.b19:05:26

basically im looking for the simplest path to take (def xs [{:a/b 1} {:a/b 2} ...]) where (count xs) => 500000 and get better than (d/transact conn xs) performance.

marshall20:05:24

there’s nothing I know of that’s pre-baked; that sort of thing tends to be a relatively custom ETL with quite a bit of stuff that is dataset-dependent

marshall20:05:34

I think the mbrainz import stuff is definitely a good starting point

Drew Verlee20:05:32

Is it possible to programmatically go from a relational schema (say with postgres) to datomics graph schema?

Alex Miller (Clojure team)20:05:20

a table is a set of rows (entities) and columns (attributes)

Alex Miller (Clojure team)20:05:43

every cell in the table is a tuple [<row> <column> <value>]

Alex Miller (Clojure team)20:05:06

and then you have to figure out an approach to foreign keys

Alex Miller (Clojure team)20:05:52

you basically need to map a fk value in one table to the entity representing the pk in the referred table

Alex Miller (Clojure team)20:05:44

I’ve only hand-waved past a few dozen critical details, but that’s the broad shape of it

Drew Verlee20:05:32

Thanks alex! right. I”m actually working through handling the foreign keys part right now. I just thought i would ask before i potentially went down a rabbit hole 🙂 In datomic, the refs aren’t constrained though right? so i can say this is a ref, but not really declare what to specifically?

Alex Miller (Clojure team)20:05:02

they are constrained to one entity :)

Alex Miller (Clojure team)20:05:11

but entities don’t have a “type”

marshall20:05:46

there are some user blogs out in the wild describing how they’ve approached this

Drew Verlee20:05:54

Right. I’m free to have: person/dog -> cat person/dog -> dog

Alex Miller (Clojure team)20:05:56

ah, cool, hadn’t seen any of those

Drew Verlee21:05:26

i’m going to give those a read. The use case is mostly academic. I wanted to create a tool that helped with testing by generated insert statements for my relational db. i’m looking to handle the foreign key constraints, so i need a way to walk the foreign keys. My original model puts the relational db into a simple graph. This works fine, but, as its a personal project, i wanted it to be perfect and i was remiss that you can’t really spec the graph as the keys are unique values and not really an entity map. So after some thought i was like , well datomic is a graph that has a defined schema, that would would make it clear to the user the shape. Then i thought about what it means that in order to solve a problem with a relational db it makes sense to reformat it into a datomic and how i wouldn’t have this testing problem and about 500 others if we were using datomic in the first place ….and now i need a drink 🍺