Fork me on GitHub

If I want to cash some datomic query results, what's the best cache key I could derive from a database? My guess would be (defn cache-key [db] ((juxt :id :basisT) db), but then there are :filt :history :raw attributes too which feel relevant, I'm just not sure what is their exact meaning My expectation would be that d/q and d/pull* and d/datoms calls would return the same results for databases if their cache-key are the same.


I think you want at least `

[(:id db) (or (d/as-of-t db) (d/basis-t db)) (d/since-t db) (d/is-history db)]


and if (d/is-filtered db) you can’t cache it at all


@marshall @jaret just confirming that :db/tupleAttrs requires the target schema to be transacted already, and that if you try to transact the targets with the tupleAttrs at the same time, you get :db.error/invalid-tuple-attrs anomay?


@robert-stuttaford that is correct. You need to transact the schema first.


Hi, I'm attempting to follow the ion-starter tutorial. When I get to the push section I run into this problem:


$ clojure -A:ion-dev '{:op :push}'
Downloading: com/datomic/ion-dev/0.9.251/ion-dev-0.9.251.pom from datomic-cloud
Downloading: com/datomic/ion-dev/0.9.251/ion-dev-0.9.251.jar from datomic-cloud
Error building classpath. Could not find artifact com.datomic:ion-dev:jar:0.9.251 in central ()


It seems to be finding the jar in datomic-cloud, but then fails for some reason

Alex Miller (Clojure team)17:03:05

Are your aws credentials set?


I'm pretty sure they are. i created a datomic database

Alex Miller (Clojure team)18:03:35

aws sts get-caller-identity


yes, they are set

Alex Miller (Clojure team)18:03:54

You’re not behind a proxy or anything?

Alex Miller (Clojure team)18:03:04

Well let me ask this, what version of clojure tool are you running? clj -Sdescribe


not behind a proxy


{:version "" :config-files ["/usr/local/Cellar/clojure/" "/Users/bamarco/.clojure/deps.edn" ] :config-user "/Users/bamarco/.clojure/deps.edn" :config-project "deps.edn" :install-dir "/usr/local/Cellar/clojure/" :config-dir "/Users/bamarco/.clojure" :cache-dir "/Users/bamarco/.clojure/.cpcache" :force false :repro false :resolve-aliases "" :classpath-aliases "" :jvm-aliases "" :main-aliases "" :all-aliases ""}


no prob, I appreciate the help

Alex Miller (Clojure team)18:03:54

do you see anything in ~/.m2/repository/com/datomic/ion-dev/0.9.251 ?

Alex Miller (Clojure team)18:03:32

the output you have looks like it downloaded the pom but not the jar, but that's extra weird


it is an empty folder

Alex Miller (Clojure team)18:03:06

anything in ~/.m2/repository/com/datomic/ion-dev ?

Alex Miller (Clojure team)18:03:21

like ea metadata xml file?


nope just the empty folder

Alex Miller (Clojure team)18:03:08

I guess those are on the repo, and you see _remote.repositories files in your local repo


I'm not sure what you mean. where do I check for _remote.repositories?

Alex Miller (Clojure team)18:03:38

does this work aws s3 cp . ?


fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden

Alex Miller (Clojure team)18:03:01

can you try export AWS_REGION=us-east-1 and try again?


same error

Alex Miller (Clojure team)18:03:34

that call is effectively what the clj s3 maven transporter is doing to download the file so it is definitely identity related somehow

Alex Miller (Clojure team)18:03:43

that bucket should be public read though to head/get object

Alex Miller (Clojure team)18:03:31

more specific but I'd guess this fails too:

Alex Miller (Clojure team)18:03:37

aws s3api head-object --bucket datomic-releases-1fc2183a --key maven/releases/com/datomic/ion-dev/0.9.251/ion-dev-0.9.251.jar


yup same error

Alex Miller (Clojure team)18:03:47

I can repro if my aws credentials are set to bad values

Alex Miller (Clojure team)18:03:49

not sure how you could get bad values but the sts returns you an identity though

Alex Miller (Clojure team)18:03:17

can you try explicitly setting AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to your iam creds?


tried explicitly setting and it failed


curl -x :[port] .[system].[region].
does work

Alex Miller (Clojure team)18:03:22

can you try aws --debug s3api head-object --bucket datomic-releases-1fc2183a --key maven/releases/com/datomic/ion-dev/0.9.251/ion-dev-0.9.251.jar

Alex Miller (Clojure team)18:03:50

in particular anything useful in the response body part

Alex Miller (Clojure team)18:03:34

I'm guessing not but you never know

Alex Miller (Clojure team)19:03:05

I don't understand why a valid iam user would not be able to access a public object in a bucket. I've pretty much only seen this with region issues but we tried that above.


DEBUG - Making request for OperationModel(name=HeadObject) with params: {'body': '', 'url': u'', 'headers': {'User-Agent': 'aws-cli/1.18.12 Python/2.7.16 Darwin/19.3.0 botocore/1.15.12'}, 'context': {'auth_type': None, 'client_region': 'us-east-1', 'signing': {'bucket': u'datomic-releases-1fc2183a'}, 'has_streaming_input': False, 'client_config': <botocore.config.Config object at 0x10d0816d0>}, 'query_string': {}, 'url_path': u'/datomic-releases-1fc2183a/maven/releases/com/datomic/ion-dev/0.9.251/ion-dev-0.9.251.jar', 'method': u'HEAD'}


2020-03-11 14:56:05,098 - MainThread - awscli.clidriver - DEBUG - Exception caught in main() Traceback (most recent call last): File "/usr/local/aws/lib/python2.7/site-packages/awscli/", line 217, in main return command_table[parsed_args.command](remaining, parsed_args) File "/usr/local/aws/lib/python2.7/site-packages/awscli/", line 358, in call__ return command_table[parsed_args.operation](remaining, parsed_globals) File "/usr/local/aws/lib/python2.7/site-packages/awscli/", line 530, in call__ call_parameters, parsed_globals) File "/usr/local/aws/lib/python2.7/site-packages/awscli/", line 650, in invoke client, operation_name, parameters, parsed_globals) File "/usr/local/aws/lib/python2.7/site-packages/awscli/", line 662, in makeclient_call **parameters) File "/usr/local/aws/lib/python2.7/site-packages/botocore/", line 316, in apicall return self.makeapi_call(operation_name, kwargs) File "/usr/local/aws/lib/python2.7/site-packages/botocore/", line 626, in makeapi_call raise error_class(parsed_response, operation_name) ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden 2020-03-11 14:56:05,105 - MainThread - awscli.clidriver - DEBUG - Exiting with rc 255 An error occurred (403) when calling the HeadObject operation: Forbidden

Alex Miller (Clojure team)19:03:43

seems like I'm using a much newer version of aws+python than you, but request is basically the same. doesn't shed much light though, as it's really an IAM issue.

Alex Miller (Clojure team)19:03:46

I'm not sure what else to try. maybe make different iam creds and see if those work.

Alex Miller (Clojure team)19:03:13

unless the datomic team folks have an idea of something


when debugging IAM or 403 issues, it is really important to know whether you are using AWS_PROFILE AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY and where your region info comes from: explicit AWS_REGION or something in the profile file


if you are calling this from EC2, there are other possibilities, but it looks like you're on a mac


yup on a mac


and if using AWS_PROFILE, whether your profile is an ordinary profile with credentials, or you're using something like aws-vault or an assume role profile


aws sts get-caller-identity will tell you who you are

Alex Miller (Clojure team)19:03:55

the region is right in the debug trace above

Alex Miller (Clojure team)19:03:04

and this bucket is public read


but where your credentials came from is a separate question

Alex Miller (Clojure team)19:03:23

and we tried both profile and keys


what does aws sts get-caller-identity return?


{ "Account": "<my-account-#>", "UserId": "<my-user-id>", "Arn": "arn:aws:iam::<a-number>:user/datomic-admin" }


is datomic-admin the user you expect to be?


what policies does that user have?


arn:aws:iam::${ACCOUNT#}:policy/datomic-admin-#{DATOMIC_SYSTEM}-{AWS_REGION} is it just ^ ?


because if so, that is not sufficient, even for a public-read bucket @alexmiller


aws s3api head-object --bucket datomic-releases-1fc2183a --key maven/releases/com/datomic/ion-dev/0.9.251/ion-dev-0.9.251.jar

An error occurred (403) when calling the HeadObject operation: Forbidden


^^ me creating a user + access_key, adding the datomic-admin policy to that user, then trying to download the jar ^^

Alex Miller (Clojure team)19:03:43

well that seems bad if so


I think this is the problem. This is my first time using aws.


no worries, this stuff is complex.


were you following a particular guide to set up datomic access?


if so, I'd love the link so that we can improve it


let me find it


Most developer accounts in aws have a pretty permissive s3:GetObject policy that would allow getting from the datomic-releases-1fc2183a bucket


thank you


yes, this allows access to datomic (as you see), but it does not have permissions to download the datomic jars from Maven.


obviously production user does not need to be an administrator, but for the purposes of this tutorial, yes


so I should make a dev user with admin permissions?


no you can keep this user and augment it with the AdministratorAccess policy


Thanks @ghadi and @alexmiller this solved it.


no problem -- we will update the docs, thanks for persisting


How do I crack open the #object[datomic.promise ...] return by a (d/transact ...) call? My goal is to check for failure/success. I see the contents of the object in my console but I'm not able to programmatically access it which I'd like to


did you de-reference it?


I tried (deref <obj>) but that errors out with a syntax error. Although it then seems to also print out the correct error


It results in:

Syntax error (Exceptions$IllegalArgumentExceptionInfo) compiling at ... ; syntax error
:db.error/not-an-entity Unable to resolve entity ... ; error I want to grab from obj

marshall20:03:16 synchronous transact can throw


you’d need to put the deref’d call in a try if you want to grab the exception