Fork me on GitHub
#datomic
<
2020-08-27
>
adamtait02:08:12

I have a query on a Datomic Ion that consistently returns partially different results when run remotely vs on the Ion. The query returns 9 results (both remotely & on-ion) but remotely returns correct entity-ids and on-ion returns 3 correct entity-ids and 6 empty/new entity-ids. I think this will be hard to reproduce b/c I can’t even reproduce it on a 2nd Ion with the same schema (it works correctly there). Here’s the query in case anyone can see something obviously wrong:

(d/q
    '[:find ?mp
      :in $ ?low ?high
      :where
      [?mp :membership-period/end ?end]
      [(< ?low ?end)]
      [(< ?end ?high)]]
    db inst-low inst-high)
Has anyone experienced this strange inconsistent behavior before? I’d love to hear from you.

joshkh08:08:03

i've seen inconsistencies when using > < operators on dates. try using [(.after ?date1 ?date2)]

Joe Lane13:08:13

Are you using (Date.) in the ion? I would confirm time zones aren’t messing this up.

adamtait22:08:38

Thanks both of you! I’m using the standard Java (Date.) objects in the comparison. I also thought it might be a timezone issue, so I ran the same query with wider date ranges and got similarly strange results. I was able to get the query working consistently using [(.before ?low ^java.util.Date ?end)] . I’m still not sure why Datomic was returning :db/ids for non-referenced entities in the results but at least I can move on… Thanks again!

cmdrdats07:08:54

Good morning! I remember seeing/hearing somewhere that :db/index is now superfluous, or, at least, we should actually always make it :db/index true , since it's basically doing that under the hood in Datomic now anyhow... I can't find the source now, but the docs still state :db/index defaults to false - so is this something we still need to consider for our schema?

favila10:08:23

You are probably thinking of datomic cloud: it makes a value index for everything so it doesn’t have a :db/index attribute at all

favila10:08:47

For on prem nothing has changed, and you should still consider whether to index or not

cmdrdats13:09:00

ah, cool, thanks!

cmdrdats07:08:14

(I'm using on-prem datomic)

Ben Sless09:08:35

Hi all, working through the https://docs.datomic.com/on-prem/tutorial.html I find myself missing a slightly more complex use case. For an example of order items, what if I wanted to edit an order and retract a specific item or update an item's amount? I find it a bit complicated when dealing with componenets

Vishal Gautam12:08:51

What’s so complicated about that ?

Vishal Gautam12:08:12

Do you have a code that you would like to share?

Ben Sless15:08:47

@UGMEQUCTV sure

(def schema
  [{:db/ident :product/name
    :db/cardinality :db.cardinality/one
    :db/unique :db.unique/identity
    :db/valueType :db.type/string}

   {:db/ident :order/items
    :db/valueType :db.type/ref
    :db/cardinality :db.cardinality/many
    :db/isComponent true}

   {:db/ident :order/id
    :db/valueType :db.type/long
    :db/cardinality :db.cardinality/one
    :db/unique :db.unique/identity}

   {:db/ident :item/product
    :db/valueType :db.type/ref
    :db/cardinality :db.cardinality/one}

   {:db/ident :item/amount
    :db/valueType :db.type/long
    :db/cardinality :db.cardinality/one}])

Ben Sless15:08:57

@(d/transact  conn schema)
@(d/transact conn [{:product/name "coke"} {:product/name "pizza"}])

Ben Sless15:08:07

Then a simple order

@(d/transact
  conn
  [{:order/id 1
    :order/items
    [{:item/product [:product/name "pizza"] :item/amount 3}
     {:item/product [:product/name "coke"] :item/amount 1}]}])

Ben Sless15:08:31

Now assuming there was a mistake and the client ordered just one pizza, can I write it with reference only to the order/id? Because if I do the following it just adds another item

@(d/transact
  conn
  [{:order/id 1
    :order/items [{:item/product [:product/name "pizza"] :item/amount 1}]}])

Ben Sless15:08:49

Or do I have to refer to the item's eid?

Vishal Gautam17:08:33

@UK0810AQ2 you will need the item id to update the amount. So here is the code that does that

(defn update-product [product-name amount]
  (let [item-id (d/q '[:find ?eid .
                       :in $ ?product
                       :where
                       [?eid :item/product ?product]]
                     (d/db conn) [:product/name product-name])]
    (d/transact conn [{:db/id item-id
                       :item/amount amount}])))

(update-product "pizza" 1)

(d/q '[:find ?count .
       :where
       [?id :item/product [:product/name "pizza"]]
       [?id :item/amount ?count]]
    (d/db conn))

Ben Sless17:08:49

I see. No way around it, then. Thanks

Jake Shelby22:08:03

just to push on this use case some more (because I've had similar wishes to @UK0810AQ2 's) ... @UGMEQUCTV since you are getting the item id first, and then transacting it, it's not concurrent safe, especially since Ben is talking about possible retracting the sub-entities (like replacing) ... I wonder if there is a way to do this in one transaction, maybe using a TX fn??

Jake Shelby22:08:39

(another similar use case to this, is a many cardinality attr, and you want to replace the current set of values with new values, but it's important to do it in one transaction for concurrency concerns)

Ben Sless08:08:45

yeah, it looks like to be safe and atomic you must use a tx fn

cjsauer13:08:46

Somewhat out of date now, but Valentin’s Datofu library had some helpers for resetting to-many relations as a tx fn. It might be worth studying the code for ideas: https://github.com/vvvvalvalval/datofu#resetting-to-many-relationships

👍 6
Kevin Mungai12:08:29

Hi, while choosing an AWS region for Datomic Cloud I found out that https://aws.amazon.com/local/africa/ is https://aws.amazon.com/marketplace/pp/prodview-otb76awcrb7aa?qid=1598530512130&amp;sr=0-1&amp;ref_=srh_res_product_title. How do I go about choosing the Africa Region (Capetown)? Also, is it possible to migrate from one AWS region using Datomic Cloud to another AWS Region? e.g. AWS EU (London) to AWS Africa Region (Capetown). Thanks in advance for your answers.

souenzzo13:08:35

About move across instances, Once you do a datomic backup, um can restore it in any other instance, including a local instance in your machine.

marshall13:08:33

@U2J4FRT2T That is true for Datomic On-Prem, not for Datomic Cloud

😱 3
marshall13:08:09

@U015L4S1C72 Datomic Cloud is not currently available in the South Africa region. I can investigate whether we could add support for that region in a future release. We don't currently have a built-in mechanism for transferring a Datomic Cloud database from one AWS region to another. However, because Datomic keeps the history of your entire database (the log) you could write a tool that reads the log of an existing database and transacts the same data into a new database in your new region.

Kevin Mungai13:08:52

Looking forward to it. I am from Nairobi, Kenya and the nearest AWS Region is South Africa, followed by UAE, India and then the EU. The EU regions are not bad because African fibre optic cables connect to the EU countries and many African companies use cloud providers that are thousands of miles away from their customers. Adding more regions would certainly help with latency. Thanks.

marshall14:08:29

We do have some India regions available for now I investigate the Cape Town region

Kevin Mungai14:08:40

Sorry I didn't find the India Region. Maybe am restricted from choosing the India Region due to my location.

marshall14:08:21

My apologies - I believe we're working on the Mumbai region support now

marshall14:08:25

it should be available in a future release

Kevin Mungai14:08:09

Thanks. Looking forward to it.

kardan15:08:06

I guess we’re all interested in different regions, I been looking out for Stockholm 🙂

marshall16:08:46

@U051F5T93 we attempted to launch in Stockholm region earlier this year, but AWS Marketplace didnt have the correct support there yet

6
kardan18:08:09

That became an odd emoji.. 🙂