Fork me on GitHub
#datomic
<
2019-12-13
>
Brian Abbott01:12:24

Random… do we have any idea of how many datomic-deployment instances exist in the world? I am contemplating a book proposal to one of the major tech publishers… Justifying potential market size would be helpful. Also, is there anything anyone here would like covered beyond the beaten path for a DB book.

souenzzo03:12:15

At NuBank, a single bank from Brasil, there is more then 1000 transactors instances

👍 4
onetom15:12:28

we shall link their latest video, where they talked about this, to give some credibility to this statement. @U2J4FRT2T are you working for nubank?

souenzzo16:12:17

I will dig a video presentation with this info. Not working but we use the same stack and I talk a lot with nubankers

onetom02:12:13

Did anyone ran into not being able to start the socks proxy for a Datomic Cloud Solo installation?

$ datomic-access client -p dev -r ap-northeast-1 enterprise-sandbox
download:  to ../../.ssh/datomic-ap-northeast-1-enterprise-sandbox-bastion
fatal error: An error occurred (404) when calling the HeadObject operation: Key "enterprise-sandbox/datomic/access/private-keys/bastion.hostkey" does not exist
Unable to read gateway hostkey, make sure your AWS creds are correct.
Where is the best place to find answers to these kind of questions?

onetom02:12:18

My Solo version is the one before the last (Nov something marketplace version). datomic-cli is 0.9.33

onetom02:12:23

i also don't quite understand why would there be some private key stored in s3 to my system. when i was starting up the solo system i was asked for a keypair. isn't that keypair is the one which is used for both the primary compute group instances and for the bastion host too? where is this documented?

onetom03:12:45

since it was only complaining about the hostkey, i've removed that step from the datomic-access script and accepted the hostkey manually, BUT I had to enable SSH access in the bastion host's security group manually too. i understand it's a very cautious default, but is it documented anywhere? i've read a lot of docs and seen tons of videos, but none of those mentioned this requirement.

onetom15:12:42

this section has eluded me somehow... no idea why. the documentation looks great and makes sense now. thanks a lot for giving direction!

marshall14:12:36

@onetom the latest Datomic CLI requires the latest release of Cloud

onetom15:12:40

@U05120CBV since the nov 26th/27th was not working on nov 29th (when you said it's an "AWS marketplace issue") and i haven't seen any newer images on the marketplace, i just assumed it's still not working. i did take a peek at the release notes, but since it did not have any entries newer than nov 29th, it also suggested that i should just use the previous to last version. for the record, on the 29th of nov (HKT), the problematic cloud formation template url i found somehow from the aws console looked like this:

$ curl -s  | jq .Mappings.RegionMap
{
    "us-east-1": {"Datomic": "ami-066145045fc7a4ad0", "Bastion": "ami-09e416d6385c15902"},
    "us-east-2": {"Datomic": "", "Bastion": ""},
    "us-west-2": {"Datomic": "", "Bastion": ""},
    "eu-west-1": {"Datomic": "", "Bastion": ""},
    "eu-central-1": {"Datomic": "", "Bastion": ""},
    "ap-southeast-1": {"Datomic": "", "Bastion": ""},
    "ap-southeast-2": {"Datomic": "", "Bastion": ""},
    "ap-northeast-1": {"Datomic": "", "Bastion": ""}
} 
and it seems to be modified on nov 15th the last time:
$ curl -Is 
HTTP/1.1 200 OK
x-amz-id-2: gvBaTBQB/yS5MVpBf2kzkSHfS9mLZQbvts4BhOTrkrEjq6pD3g+g4ydf0m4knsJ+WBoPwN+FwDE=
x-amz-request-id: AF9C1169F7991A06
Date: Fri, 13 Dec 2019 15:16:05 GMT
x-amz-replication-status: COMPLETED
Last-Modified: Fri, 15 Nov 2019 15:27:33 GMT
ETag: "ae57666cf395e15fe66809522c78c92c"
x-amz-version-id: GIC.pH3ALeoNUyyaU_3.bmehFNcZL2E2
Accept-Ranges: bytes
Content-Type: application/octet-stream
Content-Length: 130983
Server: AmazonS3
now the release page (https://docs.datomic.com/cloud/releases.html#release-history) links to a slightly different url for the "same 569-8835 version", which is dated 11/26/2019, but it's last modified date is dec 03:
$ curl -Is 
HTTP/1.1 200 OK
x-amz-id-2: NO6zf0g58YfTPGxMSrZtg+99UNgEqx++RkChsDhEhcSU3GIze3uMZ373Dkg3by/EQWDQSTIYz5A=
x-amz-request-id: C70010966DC504E4
Date: Fri, 13 Dec 2019 15:15:52 GMT
Last-Modified: Tue, 03 Dec 2019 14:25:29 GMT
ETag: "6b67112cbd9bddb2ee4d59de71e5e6a3"
Accept-Ranges: bytes
Content-Type: binary/octet-stream
Content-Length: 107111
Server: AmazonS3
and it contains AMIs for many regions, as expected:
curl -s  | jq .Mappings.RegionMap
{
    "us-east-1": {"Datomic": "ami-0b853443711d20708", "Bastion": "ami-07dccb5098034c24d"},
    "us-east-2": {"Datomic": "ami-0ee324fea6a1a937e", "Bastion": "ami-0d2278c155c1d6754"},
    "us-west-2": {"Datomic": "ami-0ccaf9cb58eaa44db", "Bastion": "ami-0f410f80475d0894e"},
    "eu-west-1": {"Datomic": "ami-04d09a0a833d508eb", "Bastion": "ami-073e038edb8c675b6"},
    "eu-central-1": {"Datomic": "ami-07b3154c0242f0e87", "Bastion": "ami-04e29a583e47d8b80"},
    "ap-southeast-1": {"Datomic": "ami-0e4afb22a156fdbac", "Bastion": "ami-018ec097e75c16803"},
    "ap-southeast-2": {"Datomic": "ami-023f62ca869bf17a2", "Bastion": "ami-09360d51b0aa43a1b"},
    "ap-northeast-1": {"Datomic": "ami-0bc5ca724bf8882b5", "Bastion": "ami-0bcc43206700dc511"}
}

onetom15:12:49

That's not a very immutable move! ;D

marshall15:12:41

Unfortunately we don’t have any control over the Marketplace listing or when/how stuff gets dated there

marshall15:12:15

the Datomic docs releases page will always be the official record of what releases are available and will have links to the separate system CFTs

marshall15:12:44

in general, once you’ve subscribed to the product on the marketplace page, i would recommend you use the docs releases page from then on instead of going back to marketplace

👍 4
onetom15:12:40

thanks for the help! that dec 5 last-modified date still doesn't make sense though.

onetom15:12:49

i recorded all the details, in case it helps to make later releases less confusing. i feel like i had bad luck and just tried things at a time when they were in flux.

marshall15:12:51

probably depends on the date we received notification from AWS that the release shipped and the date when we finished testing and actually ‘issued’ the release on our docs

marshall15:12:25

the objective is for any release listed as ‘current’ or ‘latest’ on the datomic docs release page to always work as would be expected

marshall15:12:49

so if there are issues with the templates available directly from Marketplace we won’t post them to our releases page until those issues are resolved

marshall15:12:00

(which, incidentally, is what happened in this last case)

onetom15:12:45

thanks! our team is getting more and more excited about datomic, despite these hiccups. it took me a lot of explaining, but my efforts are starting to bear fruit! 🙂

marshall15:12:09

Glad to hear it!

onetom17:12:57

I'm just going thru a solo stack deletion process by following https://docs.datomic.com/cloud/operation/deleting.html i think examples like this:

aws --region (Region) application-autoscaling deregister-scalable-target --service-namespace dynamodb --scalable-dimension dynamodb:table:WriteCapacityUnits --resource-id table/datomic-(System)
aws --region (Region) application-autoscaling deregister-scalable-target --service-namespace dynamodb --scalable-dimension dynamodb:table:ReadCapacityUnits --resource-id table/datomic-(System)
could be better written as:
/usr/bin/env REGION="<region>" SYSTEM="<system>" \
bash -xc 'for dimension in Read Write; do aws --region $REGION application-autoscaling deregister-scalable-target --service-namespace dynamodb --scalable-dimension dynamodb:table:${dimension}CapacityUnits --resource-id table/datomic-$SYSTEM; done'
so the parts which need replacement are factored out to the beginning and will only need to be replaced once. while the for loops makes it slightly complicated, it also highlights the intent better. it was not obvious to spot that few letter difference between the 2 commands on the website, where the difference was even off screen... or a compromise:
REGION="<region>"
SYSTEM="<system>"
aws --region $REGION application-autoscaling deregister-scalable-target --service-namespace dynamodb --scalable-dimension dynamodb:table:WriteCapacityUnits --resource-id table/datomic-$SYSTEM
aws --region $REGION application-autoscaling deregister-scalable-target --service-namespace dynamodb --scalable-dimension dynamodb:table:ReadCapacityUnits --resource-id table/datomic-$SYSTEM
although this only works under bash , zsh and alikes, while the one using env works under fish or anything really, too.

joshkh18:12:03

is there a way to retract all values of a :db.cardinality/many attribute, or must i retract each value individually? something like:

[:db/retract eid :recipe/ingredients] ; throws exception
instead of
[[:db/retract eid :recipe/ingredients "milk"]
 [:db/retract eid :recipe/ingredients "sugar"]
 [:db/retract eid :recipe/ingredients "eggs"]]

Joe Lane19:12:21

@joshkh You must do the latter