Fork me on GitHub
#datomic
<
2018-12-19
>
igrishaev09:12:03

Does anyone know how to resolve refs in nested maps when transacting data? Say I have a vector of maps:

{:account/id 42 
 :account/likes [{:like/id [:account/id 5166]}]}
But when I load the whole dataset, Datomic says
Unable to resolve entity: [:account/id 5166] in datom [-9223301668109571139 :like/id [:account/id 5166]]
Yet the account 5166 really exists in the vector. What should I do?

kirill.salykin09:12:11

it suppose to be :db/id, isnt it?

tomjack09:12:20

on-prem docs have the following note which I think is relevant

tomjack09:12:22

> When used in a transaction, the lookup ref is evaluated against the specified attribute's index as it exists before the transaction is processed, so you cannot use a lookup ref to lookup an entity being defined in the same transaction.

igrishaev09:12:09

what would be the best way to load nested likes then?

kirill.salykin09:12:38

do you use pull api?

kirill.salykin09:12:02

are they reference not by :db/id?

igrishaev09:12:02

No, I’m talking about data creation

tomjack09:12:00

perhaps you can get away with using {:account/id 5166} instead of [:account/id 5166]

igrishaev09:12:38

@tomjack I’ll try, but what exactly does it mean?

igrishaev09:12:15

is is another form for ref lookup?

tomjack09:12:53

no, it is just another nested map, "an entity with account/id 5166"

igrishaev09:12:58

or is that a temp id?

tomjack09:12:17

if account/id has db.unique/identity then the upsert behavior will happen

tomjack09:12:45

no temp ids in sight, but you could use them

igrishaev09:12:15

wait, that’s something different. I need to lookup a ref inside a nested map

igrishaev09:12:44

an account has a list of likes, where each like references another account

igrishaev09:12:29

so how can I reference a user who has been liked? I have all the IDs I need

kirill.salykin09:12:19

are you sure id is correct?

kirill.salykin09:12:34

can you pull data with this lookup?

igrishaev09:12:09

yes. for example, I don’t load likes at all, but then I can pull a user with such id

tomjack09:12:20

it's not different. you don't need to lookup a ref

tomjack09:12:32

try it 🙂

tomjack09:12:20

I'm assuming :account/id is :db.unique/identity, though

kirill.salykin09:12:38

otherwise you cant lookup, right?

igrishaev09:12:40

@tomjack oh my god that works!

tomjack09:12:58

lookup refs made more sense (in tx data) when tempids could not be omitted

igrishaev09:12:01

now I’ve got the idea, thank you!

kirill.salykin09:12:18

interesting, thanks for that case 🙂

kirill.salykin09:12:26

didnt know datomic can do like this

igrishaev09:12:01

as I see it, the idea is to create nested accounts of the fly and then update them

tomjack09:12:37

lookup refs are also useful for db.unique/value, I guess

tomjack09:12:59

(and can be handy generally and outside of tx data..)

avfonarev12:12:22

I'm getting a ssh: connect to host [ip] port 22: Connection refused on a fresh Datomic Cloud stack. Other ssh connections work. Is there a way to trouble shoot?

avfonarev13:12:10

Ok, that was a problem of parallelism. I was being clever and set up the permissions as soon as they appeared in the AWS Console, but before the stack formation completed... Restarting the bastion server helped...

markbastian16:12:24

I'm trying to understand the estimated costs of Datomic Cloud using the calculator at https://aws.amazon.com/marketplace/pp/prodview-otb76awcrb7aa?ref=vdr_rf. The "Datomic Cloud" choice is kind of confusing. If I choose a particular EC2 type (e.g. i3.xlarge) it has a Software/hr and an EC2/hr price along with a Total/hr price (e.g. $0.312, $0.312, $0.624). This selection is then reflected in the "Estimated Infrastructure Cost," but only the "1/2 value" is used (e.g. $0.312). What is the actual number I should use to estimate my costs for a month of usage with a particular config? To keep it simple, let's say I choose i3.xlarge (costs shown above) and have one instance running 24/7 for a 30 day period. Would I expect to pay 24 x 7 x $0.312 = $224.64 or twice that? We can ignore data storage for the moment. It looks like that is just $0.10/GB-month.

johnj17:12:06

Twice that, since production requires at least two nodes, so $224 x 2 for the i3.large, for the i3.xlarge would be $449 x 2

johnj17:12:57

don't forget storage and data transfer costs and other aws services datomic cloud uses 😉

markbastian17:12:18

The two nodes isn't baked in to the config? Is the second for HA?

johnj17:12:38

but I don't think you can start with i3.xlarge though, these instances are only enabled as additions to the production i3.large ones

johnj17:12:05

yes, for HA but both do work at the same time, it leverage both nodes.

johnj17:12:33

the price you see in aws is per node, production requires two

shaun-mahood17:12:48

@markbastian The estimate per hour for the solo topology comes out the same as production if you set them both to the same instance type, so I would expect to estimate twice what the tool shows for the production topology. I think it's a limitation in the estimation tool - hopefully someone who knows for sure will give you a more certain answer.

markbastian17:12:42

Yeah, the choice of solo vs. prod vs. anything else in the "Fulfillment Option" dropdown doesn't seem to have any effect on the details. I think it's what you choose. Even then, though, there isn't any "bottom line" number. The "Datomic Cloud" choice you make only reflects the Software/hr price and the "Estimated Infrastructure Cost" seems to reflect the EC2/hr price. I'm guessing the actual cost is the Total/hr, but I don't know if you get billed again for the Estimated Infrastracture Cost item. If solo is $1/day the answer the agrees best with that would be $0.035/hr (Total/hr).

johnj17:12:16

a single DB uses only one node for transactions though

johnj17:12:06

lets hope on-premise doesn't get abandoned 😉

donaldball20:12:28

Is is Incorrect to assert :db/doc values on entities that have :db/ident values but are not actually attribute entities?

favila20:12:37

it's not incorrect to assert :db/doc on any entity

tomjack20:12:46

it is so correct 🙂

favila20:12:48

I often add db/doc to transactions (like a commit message)

benoit20:12:53

As long as it respects the semantic of the attribute which is "Documentation string for an entity".

Joe Lane21:12:14

Hey friends, I’m x-posting from the datomic forum. Looking for guidance on Cognito claims information in the ion apigw request object. Is it stable? Was this documented and I missed it? https://forum.datomic.com/t/datomic-ions-using-cognito-user-pool-apigateway-authorizer/748?u=jplane

steveb8n23:12:52

I looked into this as well. I’m currently using AppSync which means I have an extra VTL transform layer so I can’t help with API-GW specific knowledge

steveb8n23:12:18

but I did learn about the difference between the 2 tokens that Cognito generates

steveb8n23:12:15

it’s annoying because you need the access token to make your request but that token doesn’t contain claims

steveb8n23:12:29

not sure if this helps but I thought I’d mention it since it’s related

Joe Lane23:12:28

I was going down the appsync route and backed out for lacinia

Joe Lane23:12:53

There were so many problems I ran into that didn’t even throw errors I just dropped it.

Joe Lane23:12:58

That was a few months ago.

steveb8n23:12:51

I agree with that. I’m planning to move to Lacinia as well

steveb8n23:12:11

It’s been working well but not enough upside

Joe Lane23:12:11

lacinia pedestal + pedestal.ions works really well.

Joe Lane23:12:17

(so far anyways)

steveb8n23:12:28

cool. I’m looking forward to it

Joe Lane23:12:30

but I’m much farther and moving faster than I was with appsync

steveb8n23:12:17

I think AppSync is good if you want to use Amplify.js but otherwise it’s extra complexity for not much extra value

johnj22:12:40

Is it recommended to give every entity an external unique identifier?

Joe Lane22:12:15

I’ve been giving a UUID to all my graphql entities

Joe Lane22:12:46

I think maybe it could be thought of as dont expose datomic db/id’s externally

johnj22:12:57

Ok, I'll think this a bit more

johnj22:12:25

I guess your index strategy comes into play

favila23:12:00

it's ok to share :db/id for short-lived uses (interactive, in-memory); it's ok to not give an external identifier to entities you don't want to be directly addressable from outside your system.

👍 4
favila23:12:40

entities at the other end of an is-component attribute usually should not have an external id

johnj23:12:20

exactly the case I was figuring it out for (entities that belong to others by means of :db.type/ref and isComponent, and not directly addressable), couldn't think of a use case for given them external keys, thx.

steveb8n23:12:16

I have a UUID on every entity in my db. It is useful for a lot of reasons so I consider this a best practice now

johnj23:12:54

do you need AVET for those entities?

steveb8n23:12:59

In some cases I also derive (type 3) UUIDs from entities so that I can count on stable ids, even when not persisted

Dustin Getz03:12:49

what? how do you do this

johnj23:12:20

only reason I can think of

steveb8n23:12:21

I have not added any extra indexing but I’m also not prod/live yet