is there a way to directly pull the :db/ident of a :db.type/ref? e.g. (pull ?e [:the/ref]) results in {:the/ref #:db{:id 123 :ident :ident-value}}, and i’m looking for a way to just get {:the/ref :ident-value}

easy to do with scissors afterwards, just wondering if theres syntax in pull to remove the extra step. separately: datomic is nuts

What is the idiomatic way to query entities for dates? I want to get all the posts with created-at inst that fall in range of dates. Would that be database asof and since filters or something else?

@piotr.kurnik One option is to index the created-at attribute and use a query.

If you're interested in transaction time, you have the log API Datomic on-prem.

We are seeing a intermittent timeouts from our client trying to connect to datomic cloud.

2019-03-04 08:56:49 ERROR middleware:286 - Exception encountered processing request
clojure.lang.ExceptionInfo: Datomic Client Timeout {:cognitect.anomalies/category :cognitect.anomalies/interrupted, :cognitect.anomalies/message "Datomic Client Timeout"}

Anyone else seen this before and can advise on a course of action?

hi, is it possible to connect to the localhost transactor running on port 4334 for the ddb database type without going to dynamo to lookup the IP? (i want to use ssh port forwarding from remote 4334 to local 4334 and then connect using the ddb protocol to the local 4334 port). is that possible?

i tried using datomic:dev://... but got Connection refused (likely due to either ddb v dev protocol mismatch or missing authentication or ??? )

oh, oops. i see the mention of ddb-local in the transactor.properties file

i will try that

oh, nope. i bet that refers to an actual dynamo running locally

@johanatan no; datomic writes the address of the transactor into storage

and the peer looks it up before connecting to the transactor

https://docs.datomic.com/on-prem/deployment.html#getting-connected

so even if i know the transactor is running on localhost, there's no way to connect to it directly?

the database URI is a storage address

the peer will connect to storage (whatever it is, in this case ddb), to look up the transactor endpoint

yes i know all of that. i know how it "normally" works. but what i want to do here is abnormal.

my dev is not in any VPC

so cannot connect to dynamo to get the ip

and even if it could, that ip would not be reachable

would rather not have to set up a VPN for this use case

the address of your transactor is configured via the transactor properties file

the peer must be able to read storage

it can’t work otherwise

yes, that is all true

my transactor and peer are running in ec2

my local laptop is on my desk

there is no vpn between them

🙂

if you’re trying to connect to your cloud database from your laptop, you’re running a peer (or client)

ah, sure. it's in the repl i suppose

if you’re using peer it must be able to read storage

technically i'm using datomic.api/connect

with a datomic:... db-uri

which is the peer library

right

so what's the easiest way to accomplish the end goal i have here?

is setting up a vpc/vpn my only option?

is your transactor running in a private vpc?

it's running on a lightsail instance

or something otherwise inaccessable remotely?

i doubt there is any vpc

but i can ssh into it

and port forward ports

if you can ssh to it, it must have a publicly accessible IP address (presumably the same address your cloud peer uses to connect)

which is already written to storage

so all you need to do is export AWS credentials locally that allow you to access dynamo

hmm, it would seem that it wrote a "private"/internal IP to storage. i would also rather not open my database port to the world (seems like asking for trouble)

you may need to use HOST & ALT-HOST to accomplish this if you cant resolve the address it is using from your laptop; alternatively you might be able to do something like a bastion server that handles port forwarding and remote dns resolution, but you’d have to have an instance in your VPC (or whatever) to do that with

mm, ok. alright. thanks so much for your input. some things for me to ponder.

is it possible to run a query that returns multiple different counts in a single query, where each count is for the specific attribute value, and not for the set of attribute values together, sort of like frequencies does in clojure.core?

so I ended up doing this:

(d/q '[:find ?attr ?value (count ?i)
       :where
       [?i :inventory/store ?s]
       [?s :store/short-code "demo-store"]
       (or
        (and
         [(ground :model) ?attr] 
         [?i :item/model ?value])

        (and
         [(ground :size) ?attr]
         [?i :item/size ?value]))]
     (d/db (db/get-conn)))

Are there any recommendations for batch size when doing a one-off data import to Datomic Cloud? I'm looking at about 100K entities that are logically part of the same group.

@marshall i think the following will do the trick (with my ssh tunnel in place) without having to open my db port to the world: http://www.vincecutting.co.uk/web-development/redirect-ip-address-back-to-localhost-on-mac-osx/

i tried it with the AWS internal IP first and it worked like a charm.

interesting that reads don't require the transactor

only when attempting a write was this necessary