Fork me on GitHub
#datomic
<
2019-10-14
>
dmarjenburgh10:10:14

The security/compliance dep requires that all S3 buckets are encrypted and all CMK keys have a rotation enabled. By default, the datomic template does not do this. Can anyone confirm my expectation that making these changes to the CFTs ourselves will not break anything?

ghadi13:10:05

AFAIK the buckets are encrypted @dmarjenburgh with KMS, but not using server-side encryption

tslocke16:10:20

Why is there both :db/retractEntity and :db.fn/retractEntity?

Joe Lane16:10:37

@tslocke On-prem: :db.fn/retractEntity, Cloud :db/retractEntity

tslocke16:10:19

@joe.lane yeah I noticed that, but still wondering why.

Joe Lane16:10:55

Ahh, sorry, I can't answer that one 🙂

tslocke17:10:11

With the client API, it seems, :in $ [?a-collection ...] is not allowed. What is the right way to use a collection as a param to a query?

Joe Lane17:10:09

that is allowed

tslocke17:10:46

Ahh my bad it's :find [?coll ...] that's not allowed. Maybe I can rearrange...

favila17:10:06

find destructuring equivalents: ?x . -> ffirst; [?x] -> first ; [?x ...] -> (mapv peek)

favila17:10:40

as in (->> query-result (mapv peek)) for e.g.

cjsauer17:10:54

@U09R86PA4 is (mapv peek) faster than (mapv first)?

favila17:10:37

there probably isn’t much difference

👍 4
favila17:10:37

(IOW I don’t know but I suspect there’s no difference or a marginal difference)

Brian18:10:17

Hey y'all! I'm walking a coworker through Datomic Cloud and he's getting an error on this dependency com.datomic/ion-dev {:mvn/version "0.9.234"} that says "Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact com.datomic:ion-dev:pom:0.9.234 from/to datomic-cloud (<s3://datomic-releases-1fc2183a/maven/releases>): Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 1A3956EEC6ECF1D5; S3 Extended Request ID: 2BxntHvQDQbz3eWoJql9cAFjdsdX2/g0Xu6b5hO7PzOZCnWSkyBomo83Jdh6DGHFDQYDGWhV/js=)" Any ideas as to what his problem might be? This comes up when he runs clj -A:dev

Brian18:10:12

We can't seem to get his ~/.m2/settings.xml solution to work

Brian19:10:07

Anyone have any idea? It seems that we can't pull com.datomic/ion-dev {:mvn/version "0.9.234"} out of Datomic's S3 bucket on his side when he runs Clojure -A:dev while trying to work with the ion tutorial repository

Joe Lane19:10:43

He needs to have aws credentials

Joe Lane19:10:02

has he configured the aws-cli yet?

Brian19:10:24

Yeah he has. He's been able to connect to our existing infrastructure and query our database that is running. That was when we'd removed the ion-dev from the deps.edn. But then of course he couldn't push any ions

Joe Lane19:10:33

in my deps edn I have this section

:mvn/repos {"datomic-cloud" {:url ""}
             "sonatype"      {:url ""}}

Brian19:10:28

The code we're working with (that works for me) looks like this https://github.com/Datomic/ion-starter/blob/master/deps.edn#L10

Brian19:10:51

Yes we also have :mvn/repos {"datomic-cloud" {:url ""}}

Brian19:10:12

do not have that sonatype part

Alex Miller (Clojure team)19:10:09

shouldn't need that to use ions

Alex Miller (Clojure team)19:10:20

the sonatype one that is

Alex Miller (Clojure team)19:10:32

that's just access to maven central snapshots

Brian19:10:42

I can't imagine it has to do with the deps as it's working fine for me on my side. And he has AWS permissions to access all the resources as he was able to query our database. It's just this connecting ion-dev dependency we can't seem to pull down

Alex Miller (Clojure team)19:10:48

are there AWS env vars set?

Alex Miller (Clojure team)19:10:50

and what is set in ~/.m2/settings.xml?

Alex Miller (Clojure team)19:10:01

don't post anything secret, just trying to get what's configured

Brian19:10:24

I thought so but how can we check for good measure? No such file existed in his /m2/ (I don't have one either) but he did try the structure here: https://clojure.org/reference/deps_and_cli#_procurers using the s3 and the maven-auth-repos using combinations of his creds found in /.aws/credentials and the creds he uses to sign into AWS but nothing came of it

Brian19:10:30

oops hold on let me edit

Alex Miller (Clojure team)19:10:16

echo $AWS_PROFILE
echo $AWS_ACCESS_KEY_ID
echo $AWS_SECRET_ACCESS_KEY

Brian19:10:01

Ah darn he's just clocked out 15 minutes ago. If those do or don't work, how should we proceed?

Alex Miller (Clojure team)19:10:16

the datomic-cloud s3 repo is not authenticated, so you should NOT have anything set in ~/.m2/settings.xml

Brian19:10:51

None of those echo statements produce any output for me actually

Brian19:10:13

Okay cool cool. I'll make sure he deletes that if he did end up keeping the file around.

Brian19:10:33

Should those produce output for him?

Alex Miller (Clojure team)19:10:05

one thing that is confusing is that you must have some aws env vars set, even though they will not be used to access the datomic-cloud repo

Alex Miller (Clojure team)19:10:04

because reasons, the s3 provider will attempt to look up the region of the bucket holding the repo and while the bucket is public, the IAM check on the region lookup will fail if you don't have some (any!) credentials set

4
Brian19:10:46

So then given that when we commented out the ion-dev, we were still able to use the bastion to connect to the database and run queries, and also able to pull in com.datomic/client-cloud {:mvn/version "0.8.78"} which I'm assuming is also hosted in S3, I couldn't quite see how the creds were not working

Brian19:10:11

If we were to totally redo the creds, should we just delete his ec2 keypair, delete the ~/.aws, and try it all over?

Alex Miller (Clojure team)20:10:03

I suspect things are working off the default creds

Alex Miller (Clojure team)20:10:31

but the s3 repo provider is super old and may not even look at that stuff

Alex Miller (Clojure team)20:10:52

I think it is worth setting those aws env vars to same as whatever you're using in credentials and see if it works

Brian20:10:53

Where does $AWS_PROFILE come from? I see the other two in ~/.aws/credentials/

Alex Miller (Clojure team)20:10:58

that specifies which profile in your credentials to use

Alex Miller (Clojure team)20:10:32

you can export AWS_PROFILE=default to use the default

Alex Miller (Clojure team)20:10:39

so that might be a good test

ghadi20:10:04

aws sts get-caller-identity may be useful

Brian20:10:53

I've written down these suggestions and I'll report back the results tomorrow morning when my coworker is back online. Thanks for all the help! =]