Fork me on GitHub
#datomic
<
2019-11-06
>
Jon Walch03:11:58

Anyone run into this? This is what happens when my application (running in EKS) tries to connect to my datomic cloud. Datomic cloud is working fine, tested it with the proxy. I already double checked that the EndpointAddress is correct in my cloud formation stack

{:type clojure.lang.ExceptionInfo
   :message Unable to connect to .<stack_name>.
   :data {:cognitect.anomalies/category :cognitect.anomalies/not-found, :cognitect.anomalies/message entry.<stack-name>.: Name does not resolve, :config {:server-type :cloud, :region us-west-2, :system <system-name>, :endpoint .<stack-name>., :endpoint-map {:headers {host entry.<stack-name>.}, :scheme http, :server-name entry.<stack-name>., :server-port 8182}}}
   :at [datomic.client.impl.cloud$get_s3_auth_path invokeStatic cloud.clj 178]}]

Jon Walch03:11:23

going to try peering my VPCs

cjmurphy08:11:42

Using on-prem I'm looking to generate a tempid then use it to find the real-eid from the :tempids map that is returned from transact!. A generated tempid looks like {:part :db.part/user, :idx -1000305}. It would make sense to me if instead it was a negative number of 19 digits length, because the :tempids map keys are all 19 digit negatives. Can someone help with the error in my understanding? Thx.

onetom09:11:16

you can use strings as tempids even with the on-prem version of datomic. is there any specific reason for using the d/tempid function?

cjmurphy09:11:12

Err no - I somehow must have just come across it and thought that was the way to generate 'the next tempid'. I can just use gensym I guess. How do people normally generate tempid strings?

onetom09:11:58

i have the feeling that u might not even need to generate if you don't care about what the tempids are.

onetom09:11:29

it's not necessary to explicitly specify a tempid anymore, UNLESS you want to reference a newly transacted (or modified) entity in another fact/entity within the same txn

onetom09:11:36

also, tempid strings only have to be uniq within a transaction, so you can simply number them with range

onetom10:11:07

maybe if u can share more specifics about your use-case, then we can help easier. im working on some tsv import code now and trying to write tests for it. it looks something like this:

(defn mk-rule [n]
  (let [rule-expr (str "rule-" (if (keyword? n) (name n) n))]
    {:db/ident  (keyword rule-expr)
     :rule/algo :regex
     :rule/expr rule-expr}))

(deftest re-import-rules
  (testing "remove rule"
    (tx [(mk-rule :to-be-deleted)
         (mk-rule :unchanged)])
    (is (= #{:rule-to-be-deleted
             :rule-unchanged}
           (q '[:find (set ?r-ident) .
                :where
                [?r :rule/algo+expr]
                [(datomic.api/ident $ ?r) ?r-ident]])))))

onetom10:11:49

note how i just made up a convention of identifying my temporary rule entities with a rule- prefix in another test, i just made up a bunch of rules and simply numbered, like this:

(tx (map mk-rule (range 10)))
then i could create an entity referencing them, like this:
(tx [{:txn/id            1
        :txn/matching-rule [:rule-2 :rule-3]}])
im using db/idents here, so i don't have to fuss around with tempid resolution, since im working on an in-memory db, but the naming principle is the same...

onetom10:11:11

also, if u use nested entity maps in tx-data, then the assignment of the nested entity ids to the containing entity's ref attribute is done automatically by the d/transact logic

cjmurphy10:11:18

This is a Fulcro application. The idea is that there are fulcro-tempids on client. They get sent to the server. The idea is to generate datomic tempids to go with them as pairs in a map. (key will be Fulcro, val will be datomic tempid). After transact! get two maps. Can use them to get a map of fulcro-tempid -> real-eid. The client can then use that map to do the remapping of the client state.

onetom10:11:30

i've also noticed that u were talking about transact!. that's an old function, if i understood correctly. the current https://docs.datomic.com/on-prem/clojure/index.html documentation doesn't even mention it anymore. it just simply uses transact

cjmurphy10:11:16

I'm using an old version of Datomic.

onetom10:11:42

and upgrading is not an option?

onetom10:11:40

because writing more code which is not necessary when using newer datomic feels like unnecessary pain

cjmurphy10:11:07

Well the upgrading ability ran out.

cjmurphy10:11:28

Only lasts for a year.

onetom10:11:23

that seems like a recent enough version though to support string tempids and transact without a bang

cjmurphy10:11:50

Yes I'll start using transact now I know.

cjmurphy10:11:01

i.e. today.

onetom10:11:47

(i also just noticed this change a few days ago, when coming back to datomic after 2-3 years ;)

cjmurphy10:11:06

So I should just generate negative number and str them?

cjmurphy10:11:29

Yeah I noticed it but ignored it!

onetom10:11:31

so it sounds like you dont need a fulcro-tempid -> datomic-tempid because the fulcro-tempid can be just a string and u can use that directly in your tx-data

cjmurphy10:11:25

Yes that's what I thought too, as long as it is a string, which I can convert it to if its not.

cjmurphy10:11:13

So I might just use the random-uuid from in there.

onetom10:11:04

isn't something like (->> tx-data (d/transact conn) :tempids vals) enough?

cjmurphy10:11:36

That gives me the real-eids.

onetom10:11:41

what else is associated to the "fulcro-tempids" on the client side?

cjmurphy10:11:12

Well back on the client, in client state, there are client tempids (yes "fulcro-tempids").

onetom10:11:17

aren't those already some uniq strings?

onetom10:11:51

because it sounds like you can just use those directly as the datomic tempids

cjmurphy10:11:52

Fulcro can change them to real ids, but needs the map that can do that.

cjmurphy10:11:08

They are from that function above.

cjmurphy10:11:59

So for each one of them (a TempId) there needs to be a val which is a real-eid.

onetom10:11:55

and what is that TempId? which namespace is it from for example? but i guess i can't add more to this topic now. i have to get back to work too.

cjmurphy10:11:36

The problem is already solved in my mind, doing as you say, using 'fulcro-tempid' as the tempids to datomic. String conversion not really an issue.

cjmurphy10:11:53

Thank you very much.

👍 4
Brian15:11:56

Using Datomic Cloud I have an entity with a :tags attribute with a :db.type/keyword with :cardinality/many. The allowed keywords are :a :b :c :d :e and any combination of those keywords is allowed as the value of the :tags attribute. Now I want to update the value of :tags with a new combination of those keywords. How can I say "remove current values of :tags and add these new values"?

Brian15:11:24

I am flexible on the schema so if a structural change makes sense, I can do that

Brian15:11:29

I could d/pull on the entity to pull back it's tags and then retract them one by one but that seems like the wrong way

ghadi15:11:14

you want it to be atomic -- if you read then transact you'll have a race @brian.rogers

ghadi15:11:39

there are a few patterns for handling this: install a transaction function is one

ghadi15:11:51

another possibility is to avoid the race is to do a CAS then retry https://docs.datomic.com/cloud/best.html#optimistic-concurrency

ghadi15:11:08

you'll need to add an attribute that you can CAS upon

ghadi15:11:56

:tags/version 4 then send in [:db/cas entity :tags/version 4 5] alongside your asserts+retracts

Brian15:11:30

Thank you @ghadi! That gives me exactly what I needed to think about 😃

Jon Walch18:11:02

@marshall Is this documentation still up to date? https://docs.datomic.com/cloud/operation/client-applications.html#create-endpoint I don't see "LoadBalancerName" in my Datomic CloudFormation Output section. I'm using a Solo topology. Look like one can't connect using this method for a solo topology. Do I have to do VPC peering for solo?

Jon Walch19:11:39

Accessing Datomic from a separate VPC in versions older than 388 only can be achieved with VPC Peering Well I'm on the latest version so this seems out of the question too. What am I supposed to do?

Jon Walch19:11:07

Just tried adding my EKS VPC to the private datomic route 53 hosted zone, no dice on that either

marshall20:11:53

You could run the SOCKS proxy in your EKS vpc

Jon Walch20:11:58

Thanks! I think I'm just going to go with Production