Fork me on GitHub
#datomic
<
2023-11-28
>
jdhollis17:11:27

Hi, all. I’ve got a pull pattern that needs to be refactored. I need to use the same expression for 3 different attributes.

jdhollis17:11:32

What’s the best way to dynamically construct a pull expression?

jdhollis17:11:40

I’ve tried injecting it via an argument to d/q but got a recursion error. I’ve also tried to constructing it using quotes + splicing, but I haven’t been able to untangle how to make that work properly (if it’s even possible).

danbunea18:11:08

hi, I do a lot of dynamic queries such as let [ where (into [] (concat (conditions table-namespace-alias auth-path) ;; [?id ?field] [(list rule-name rule-variable '?user-id)])) query {:find [(list 'pull id-field-variable pattern)] :in (if has-ids? ['$ '% '?field pattern [id-field-variable '...] '?user-id] '[$ % ?field pattern ?user-id]) :where where}]

danbunea18:11:36

Sorry, I am on my phone but I guess you'll see how I pass the pattern to the pull part of the query

jdhollis18:11:50

So maybe I'm just not quoting liberally enough…

danbunea18:11:57

This is how I pass some params

danbunea18:11:14

rules) ] (->> (apply d/q query (if has-ids? [db rules id-field pattern ids user-id] [db rules id-field pattern user-id])) (mapv first)))))

jdhollis19:11:21

Is there any way to get Datomic to output what it ultimately thinks is the query (instead of executing it)?

jdhollis20:11:13

I think I'm finally getting it. Thanks for the example!

👍 1
steveb8n20:11:52

(let [pull-expr [:id :name]
        query (apply-template
                '[common-pull]
                '[:find (pull ?parent common-pull) (pull ?child common-pull)
                  :in $
                  :where
                  [?parent :id _]
                  [?child :parent ?parent]]
                [pull-expr])]
    (d/q query db))

👍 1
steveb8n20:11:49

we use this for composing the read part of queries and rules for composing the where part of queries. works well

jdhollis21:11:46

(def ^:const country-pattern
    '{:country/display-name
      [:alias/name]
      :country/region
      [:region/name
       {:region/_sub-regions
        [:region/name
         {:region/_sub-regions
          [:region/name]}]}]})

  (def ^:const post-location-pattern
    (let [city-and-country-pattern (merge `{:city/country
                                            [:country/name
                                             ~country-pattern]}
                                          country-pattern)]
      `[:city/name
        :country/name
        ~city-and-country-pattern]))

jdhollis21:11:31

Here's where I'm headed. Just having to be careful on quoting and how I compose everything.

jdhollis21:11:54

(The queries I'm working on don't need support for runtime variability at this point.)

indy20:11:33

Is there a way to know if datomic peers/transactor are connecting to memcached? I've set the correct configuration for the transactor (properties file) and the peer (system property "datomic.memcachedServers"). The stat command on memcached tells me that it's not being used. I did a (count (d/datoms db :eavt)) but still couldn't get any hits.

favila20:11:48

There’s a metric named “Memcache” which tells you number of requests and the hitrate on both transactors and peer

favila20:11:04

And also MemcachedPutMsec and MemcachePutFailedMsec

favila20:11:38

On a fresh peer you can also try d/q with an io context set https://docs.datomic.com/cloud/api/io-stats.html#query and see if any memcache keys are in the :reads map

👍 1
favila20:11:42

If your object cache is big enough and you index infrequently, it may very well be that no reads happen from memcached, only writes.

indy21:11:22

Was about to post that I already checked cloudwatch but couldn't find those metrics (can see other transactor metrics). I too was suspecting it might be that memcached wasn't even required, I just have ~600 datoms (just trying out on-prem) which would trivially fit in the object cache. But wouldn't datomic try to connect to memcached at all? Asking because the active connections on my elasticache memcached node is 0. (I'm able to telnet to memcached from the peer/transactor node)