Fork me on GitHub
#datomic
<
2016-02-11
>
casperc06:02:22

@val_waeselynck: Ah yeah sorry, that wasn’t a given. It is in a transaction i am using the lookup ref. I am basically doing a big data import from another database with a lot of tables that are linked by ids. For each id I am creating a ref, which I create by finding the entity that it references. The reference I am trying to resolve should be there, but in case it isn’t (due to data error) I still want the transaction to succeed. However when using lookup refs, the transaction fails when the lookup ref doesn’t resove to en entity. Currently I am using a query at the peer to find the id if present, but it is not as nice a solution as just putting a lookup ref in the transaction (and it adds the problem of making sure that the peer is synchronised with all the relevant transactions).

akiel06:02:48

@ljosa: thanks @bkamphaus Can you say something more here? Does the order depend on the index used?

val_waeselynck07:02:09

@casperc Id use the query in a transaction fn

casperc08:02:08

@val_waeselynck: Good point, that might be the way to go there.

Ben Kamphaus14:02:51

@akiel: no order is promised as when the entities are retrieved the underlying semantic is a set. Regarding implementation as vector as opposed to entity API — due to flexibility of nested pull specifications, it’s possible to get conflicts in the set which would reduce the count of elements and mislead about the number of matched entities. The tradeoff, as you recognize, is not being guaranteed correctness in equality, etc. of retrieved vectors since order is not promised.

akiel15:02:42

@bkamphaus: I understand your point with nested pull specifications. I also understand that promising an order would constrain future implementation changes. But is it possible to declare the ordering as undefined but consistent within a version of datomic and either within one point in time or over all points in time?

hjrnunes15:02:20

Can I add an entity like this, nesting components is shown?

Ben Kamphaus15:02:34

@akiel: I understand the case you’re making and can make a note of the request. To set expectations, though, we’re pretty conservative on making guarantees, and consistent ordering within a few caveats like within but not between versions is not likely to be one we’ll be making. At least for the short term it’s just a tradeoff when using pull.

akiel15:02:57

@bkamphaus I’m fine with this. Thanks.

akiel16:02:57

@hjrnunes: I’m not sure whether you can specify nested entities. But if so, they miss the :db/id (d/tempid …) part.

akiel16:02:17

But it’s alsways possible to add the entities first and reference them later by there tempid (in the same transaction)

hjrnunes16:02:29

so I’m trying to do two different things here, I guess

hjrnunes16:02:08

some of the components reference existing entities, others reference new entities that are to be created

hjrnunes16:02:30

I understand your point re. the second case i.e. the new ones

hjrnunes16:02:41

but what about the first case?

hjrnunes16:02:12

btw, I’m trying to use lookup refs but I tried with the actual id in long format, and still get the same error

Ben Kamphaus16:02:36

@hjrnunes: See: http://docs.datomic.com/transactions.html#nested-maps-in-transactions — I believe issue is that you need to use map form, not list form for parent entity (can’t nest the map as a value in a :db/add list form).

hjrnunes16:02:16

so the entire transaction needs to be a map then?

Ben Kamphaus16:02:27

@hjrnunes: not the entire transaction, but the nested map has to be inside a map. You could specify other [:db/add …] or [:db/retract …] forms in the transaction as a whole.

hjrnunes16:02:28

I see; does the map format implicitly means :db/add?

Ben Kamphaus16:02:22

@hjrnunes: Yes, internally transformed into the add form, see: http://docs.datomic.com/transactions.html#map-forms

hjrnunes16:02:43

ok, so I guess the right transaction would look something like this then:

Ben Kamphaus16:02:13

@hjrnunes: I would just put the :recipe/name assertion in the map as well, to be honest. As opposed to passing the same tempid twice.

Ben Kamphaus16:02:36

I.e. if you’re just asserting the entity, one big map is typically the most readable form.

hjrnunes16:02:47

yeah I suppose that’s a good idea

hjrnunes16:02:53

Btw, can I use lookup refs in nested components?

Ben Kamphaus16:02:14

I believe you should be able to, not sure if I’m done that specifically before but I would expect that you can.

hjrnunes16:02:12

ok, i’ll give it a try

hjrnunes16:02:54

well, I’m getting something more puzzling now

hjrnunes16:02:56

IllegalArgumentExceptionInfo :db.error/not-a-data-function Unable to resolve data function: :db/id datomic.error/arg (error.clj:57)

Ben Kamphaus16:02:45

@hjrnunes: that error would indicate you’re using :db/id in a list form somewhere rather than map form.

hjrnunes16:02:37

Yeah, I thought that initially

hjrnunes16:02:41

I’ll double check

hjrnunes16:02:12

just to confirm

hjrnunes16:02:58

is the tx data supposed to be wrapped in a vector when it’s passed on to transact if it is a map?

hjrnunes16:02:27

i.e. (transact db tx-data) or (transact db [tx-data]) assuming tx-data is a map?

Ben Kamphaus16:02:57

Using the example from: http://docs.datomic.com/transactions.html#nested-maps-in-transactions

(d/transact conn [{:db/id order-id
                   :order/lineItems [{:lineItem/product chocolate
                                       :lineItem/quantity 1}
                                      {:lineItem/product whisky
                                       :lineItem/quantity 2}]}]

hjrnunes16:02:18

right, so I can’t see what the issue is then

hjrnunes16:02:00

that’s my tx map, it gets wrapped in a vec before it goes to transact

Ben Kamphaus16:02:26

@hjrnunes: and you’re sure you’re wrapping it in a vec and not converting it to one?

hjrnunes16:02:22

actually, I’m doing that exactly

hjrnunes17:02:04

@bkamphaus: perfect, got it working. Thank you sir!

currentoor18:02:37

Is it bad to include indexes on most schema attributes?

Ben Kamphaus18:02:10

@currentoor: nope, in fact the overhead is fairly cheap, I’d probably turn :avet indexes on anything that wasn’t blobby, though we’ve improved perf. for that specific problem (large string or binary values in :avet).

currentoor18:02:47

@bkamphaus: Oh ok awesome. I need to add them retroactively. Anything I need to be careful about?

Ben Kamphaus18:02:33

nope, I would just review the docs on schema alteration - http://docs.datomic.com/schema.html#Schema-Alteration - to get example forms of the alteration transactions you need to submit.

currentoor18:02:45

Cool, thanks!

timgilbert20:02:52

Hey, maybe kind of a dumb question, but I've got my transactors deployed to AWS/DDB with IAM roles, and my deployed peer machines can connect to them just fine from the aws-peer-role roles, but now I can't figure out how to tell Amazon to treat my laptop as also being in that peer role during development.

timgilbert20:02:55

What do people generally do to get this working, is there a way I can avoid setting up environment variables on every development laptop?

settinghead21:02:26

would datomic be suitable for real-time survey program? say a lot of people cast votes on items and also see real-time updates on vote counts

settinghead21:02:52

my main concern is the write throughput. any insight in this would be appreciated

Ben Kamphaus21:02:46

@timgilbert: for running a transactor or peer on a laptop against ddb, as it’s a dev/testing scenario only, I just have my AWS user access keys in my environment. I think to use roles outside of AWS resources you’re still stuck with a user who must assume the role.

Ben Kamphaus21:02:43

@settinghead: nothing about your use case intrinsically disqualifies Datomic. I guess the main question is what’s your best estimate of the throughput you’d need?

currentoor22:02:57

Is a sorted collection of squuids the same as if they were ordered by the time they were created?

currentoor22:02:16

Assuming they were created simply with (squuid)