Fork me on GitHub
#datomic
<
2018-11-14
>
lwhorton16:11:13

there’s no way to forward-declare an entity ref, right? i can’t tx data into a db like {:foo/bar [:bar/id 123]} where bar/id (and the associated entity) don’t yet exist in the db?

marshall16:11:03

you can assert a nested entity, thereby creating it

marshall16:11:16

{:foo/bar {:bar/id 123}}

marshall16:11:41

@lwhorton ^ however, if you are using a lookup ref then yes, the entity needs to exist already

lwhorton17:11:54

ah, cool. i guess i need to order my etl properly

mgrbyte17:11:27

@lwhorton see https://docs.datomic.com/cloud/transactions/transaction-data-reference.html#sec-7-1 about referencing other entities created within the same transaction

lwhorton17:11:11

:thumbsup: i just ran some experiments with this and bumped into the “entity [ref val] doesn’t exist” problem

mgrbyte17:11:25

i've been using val (from your above example) as a tempid when I know it's unique within a transaction and linking that way.

lwhorton17:11:09

yea, i’m going to have a tiny ETL process that xforms external data into tx-data with using “tempids” that are really external foreign keys. the problem right now is just tracing my schema’s refs so i can ensure correct import order (and avoid trying to lookup-ref some entity that doesnt yet exist)

👍 4
grzm20:11:34

Is there a dictionary of Datomic errors? This one is currently inscrutable to me:

{
                "Type": "clojure.lang.ExceptionInfo",
                "Message": "Server Error",
                "Data": {
                    "CognitectAnomaliesCategory": "CognitectAnomaliesFault",
                    "CognitectAnomaliesMessage": "Server Error",
                    "DatomicAnomaliesError": {
                        "Error": "Unable to create catalog entry",
                        "Cause": {
                            "Conflict": true
                        }
                    }
                }
Is this a user-level issue? (e.g., attempting to transact tx-data that would cause a unique conflict) Or is it something lower down, say, at some interaction with DynamoDB (implied by the "catalog entry")?

marshall20:11:17

@grzm solo or production?

grzm20:11:48

solo, but seen in production as well

marshall20:11:23

can you file a ticket on the support portal so we can investigate?

grzm20:11:18

Will do.

lwhorton21:11:58

what’s the conventional “last step” after a query w/ entity pull to get the entities as a collection instead of a collection of single-entry collections? do we just use mapcat?

lwhorton21:11:08

(i ask because this seems like the pull syntax does something important wrt laziness, and i would hate to overflow-bomb myself trying to map over a huge entity result)

favila00:11:58

:find [?x ...] gets you a flat coll of ?x

favila00:11:22

(map peek) would work too as a post-processing step

favila00:11:53

everything in query is eager

lwhorton00:11:16

ah, okay. if it’s eager it won’t matter. but i do like peek better

steveb8n21:11:09

@lwhorton re forward declaration: I learned a trick for this. all my entities have a shared uuid attribute. just before the save, I copy the str version of the uuid into the :db/id as a “temp-id”. this means that all references to that entity can now also use the string version in their ref attrs and Datomic will resolve them

lwhorton21:11:45

hmmm. interesting. do you have the uuid (squuid?) generator as some out of band process elsewhere before you try to transact?

steveb8n22:11:23

I don’t bother with squuids anymore (not required) so I just create the uuid in calling code for insert (it’s always read first for update)

steveb8n22:11:57

btw only works in apis that allow string tempids. not sure if the older peer apis support this

marshall14:11:43

Yes, I like this approach as well. I’ve used it successfully in several projects

lwhorton17:11:25

i’m not sure i follow completely, could you provide a few lines of code as an example? do you mean each entity has a :db/ident :my/uuid? where do you store refs to your uuids when they’re created? or is this just a trick for single large transactions and not multiple txs?