This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-03-11
Channels
- # announcements (21)
- # aws (2)
- # babashka (20)
- # beginners (47)
- # bristol-clojurians (2)
- # calva (63)
- # cider (24)
- # clj-kondo (22)
- # cljs-dev (3)
- # cljsrn (6)
- # clojars (3)
- # clojure (147)
- # clojure-europe (21)
- # clojure-france (2)
- # clojure-italy (3)
- # clojure-losangeles (1)
- # clojure-nl (3)
- # clojure-spec (2)
- # clojure-uk (70)
- # clojurescript (37)
- # core-logic (6)
- # cursive (4)
- # data-science (2)
- # datomic (99)
- # events (1)
- # figwheel-main (20)
- # fulcro (26)
- # graalvm (6)
- # graphql (5)
- # kaocha (8)
- # leiningen (20)
- # meander (22)
- # nrepl (4)
- # off-topic (27)
- # pathom (5)
- # pedestal (3)
- # re-frame (20)
- # reagent (4)
- # shadow-cljs (43)
- # spacemacs (11)
- # tools-deps (55)
- # tree-sitter (6)
- # vim (8)
- # xtdb (18)
- # yada (14)
If I want to cash some datomic query results, what's the best cache key I could derive from a database?
My guess would be (defn cache-key [db] ((juxt :id :basisT) db)
, but then there are :filt :history :raw
attributes too which feel relevant, I'm just not sure what is their exact meaning
My expectation would be that d/q
and d/pull*
and d/datoms
calls would return the same results for databases if their cache-key
are the same.
I think you want at least `
[(:id db) (or (d/as-of-t db) (d/basis-t db)) (d/since-t db) (d/is-history db)]
@marshall @jaret just confirming that :db/tupleAttrs
requires the target schema to be transacted already, and that if you try to transact the targets with the tupleAttrs at the same time, you get :db.error/invalid-tuple-attrs
anomay?
@robert-stuttaford that is correct. You need to transact the schema first.
thanks!
Hi, I'm attempting to follow the ion-starter tutorial. When I get to the push section I run into this problem:
$ clojure -A:ion-dev '{:op :push}'
Downloading: com/datomic/ion-dev/0.9.251/ion-dev-0.9.251.pom from datomic-cloud
Downloading: com/datomic/ion-dev/0.9.251/ion-dev-0.9.251.jar from datomic-cloud
Error building classpath. Could not find artifact com.datomic:ion-dev:jar:0.9.251 in central ( )
Are your aws credentials set?
aws sts get-caller-identity
Can tell you
You’re not behind a proxy or anything?
Well let me ask this, what version of clojure
tool are you running? clj -Sdescribe
{:version "1.10.1.536" :config-files ["/usr/local/Cellar/clojure/1.10.1.536/deps.edn" "/Users/bamarco/.clojure/deps.edn" ] :config-user "/Users/bamarco/.clojure/deps.edn" :config-project "deps.edn" :install-dir "/usr/local/Cellar/clojure/1.10.1.536" :config-dir "/Users/bamarco/.clojure" :cache-dir "/Users/bamarco/.clojure/.cpcache" :force false :repro false :resolve-aliases "" :classpath-aliases "" :jvm-aliases "" :main-aliases "" :all-aliases ""}
just ruling stuff out
do you see anything in ~/.m2/repository/com/datomic/ion-dev/0.9.251 ?
the output you have looks like it downloaded the pom but not the jar, but that's extra weird
anything in ~/.m2/repository/com/datomic/ion-dev ?
like ea metadata xml file?
I guess those are on the repo, and you see _remote.repositories files in your local repo
does this work aws s3 cp
?
fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden
can you try export AWS_REGION=us-east-1
and try again?
that call is effectively what the clj s3 maven transporter is doing to download the file so it is definitely identity related somehow
that bucket should be public read though to head/get object
more specific but I'd guess this fails too:
aws s3api head-object --bucket datomic-releases-1fc2183a --key maven/releases/com/datomic/ion-dev/0.9.251/ion-dev-0.9.251.jar
I can repro if my aws credentials are set to bad values
not sure how you could get bad values but the sts returns you an identity though
can you try explicitly setting AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to your iam creds?
can you try
aws --debug s3api head-object --bucket datomic-releases-1fc2183a --key maven/releases/com/datomic/ion-dev/0.9.251/ion-dev-0.9.251.jar
in particular anything useful in the response body part
I'm guessing not but you never know
I don't understand why a valid iam user would not be able to access a public object in a bucket. I've pretty much only seen this with region issues but we tried that above.
DEBUG - Making request for OperationModel(name=HeadObject) with params: {'body': '', 'url': u'https://s3.amazonaws.com/datomic-releases-1fc2183a/maven/releases/com/datomic/ion-dev/0.9.251/ion-dev-0.9.251.jar', 'headers': {'User-Agent': 'aws-cli/1.18.12 Python/2.7.16 Darwin/19.3.0 botocore/1.15.12'}, 'context': {'auth_type': None, 'client_region': 'us-east-1', 'signing': {'bucket': u'datomic-releases-1fc2183a'}, 'has_streaming_input': False, 'client_config': <botocore.config.Config object at 0x10d0816d0>}, 'query_string': {}, 'url_path': u'/datomic-releases-1fc2183a/maven/releases/com/datomic/ion-dev/0.9.251/ion-dev-0.9.251.jar', 'method': u'HEAD'}
2020-03-11 14:56:05,098 - MainThread - awscli.clidriver - DEBUG - Exception caught in main() Traceback (most recent call last): File "/usr/local/aws/lib/python2.7/site-packages/awscli/clidriver.py", line 217, in main return command_table[parsed_args.command](remaining, parsed_args) File "/usr/local/aws/lib/python2.7/site-packages/awscli/clidriver.py", line 358, in call__ return command_table[parsed_args.operation](remaining, parsed_globals) File "/usr/local/aws/lib/python2.7/site-packages/awscli/clidriver.py", line 530, in call__ call_parameters, parsed_globals) File "/usr/local/aws/lib/python2.7/site-packages/awscli/clidriver.py", line 650, in invoke client, operation_name, parameters, parsed_globals) File "/usr/local/aws/lib/python2.7/site-packages/awscli/clidriver.py", line 662, in makeclient_call **parameters) File "/usr/local/aws/lib/python2.7/site-packages/botocore/client.py", line 316, in apicall return self.makeapi_call(operation_name, kwargs) File "/usr/local/aws/lib/python2.7/site-packages/botocore/client.py", line 626, in makeapi_call raise error_class(parsed_response, operation_name) ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden 2020-03-11 14:56:05,105 - MainThread - awscli.clidriver - DEBUG - Exiting with rc 255 An error occurred (403) when calling the HeadObject operation: Forbidden
seems like I'm using a much newer version of aws+python than you, but request is basically the same. doesn't shed much light though, as it's really an IAM issue.
I'm not sure what else to try. maybe make different iam creds and see if those work.
unless the datomic team folks have an idea of something
when debugging IAM or 403 issues, it is really important to know whether you are using AWS_PROFILE AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY and where your region info comes from: explicit AWS_REGION or something in the profile file
if you are calling this from EC2, there are other possibilities, but it looks like you're on a mac
and if using AWS_PROFILE, whether your profile is an ordinary profile with credentials, or you're using something like aws-vault
or an assume role profile
the region is right in the debug trace above
and this bucket is public read
and we tried both profile and keys
{ "Account": "<my-account-#>", "UserId": "<my-user-id>", "Arn": "arn:aws:iam::<a-number>:user/datomic-admin" }
arn:aws:iam::${ACCOUNT#}:policy/datomic-admin-#{DATOMIC_SYSTEM}-{AWS_REGION}
is it just ^ ?
because if so, that is not sufficient, even for a public-read bucket @alexmiller
aws s3api head-object --bucket datomic-releases-1fc2183a --key maven/releases/com/datomic/ion-dev/0.9.251/ion-dev-0.9.251.jar
An error occurred (403) when calling the HeadObject operation: Forbidden
^^ me creating a user + access_key, adding the datomic-admin policy to that user, then trying to download the jar ^^
well that seems bad if so
Most developer accounts in aws have a pretty permissive s3:GetObject policy that would allow getting from the datomic-releases-1fc2183a
bucket
yes, this allows access to datomic (as you see), but it does not have permissions to download the datomic jars from Maven.
you were following https://docs.datomic.com/cloud/ions/ions-tutorial.html right ?
it might be good to put something in https://docs.datomic.com/cloud/troubleshooting.html#troubleshooting-ion-push
https://docs.datomic.com/cloud/ions/ions-tutorial.html#org29202d3 make sure that user is an AWS Administrator @mail524
obviously production user does not need to be an administrator, but for the purposes of this tutorial, yes
no you can keep this user and augment it with the AdministratorAccess policy https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_job-functions.html#jf_administrator
Thanks @ghadi and @alexmiller this solved it.
How do I crack open the #object[datomic.promise ...]
return by a (d/transact ...)
call? My goal is to check for failure/success. I see the contents of the object in my console but I'm not able to programmatically access it which I'd like to
I tried (deref <obj>)
but that errors out with a syntax error. Although it then seems to also print out the correct error
It results in:
Syntax error (Exceptions$IllegalArgumentExceptionInfo) compiling at ... ; syntax error
:db.error/not-an-entity Unable to resolve entity ... ; error I want to grab from obj
@brian.rogers synchronous transact
can throw