Hi guys are you planning to add cloudwatch logs to datomic AMI?

@bkamphaus will likely know the answer to that, @lowl4tency

robert-stuttaford: it’s simple to add the support to CFN template. Curious mgiht be it presents here

my guess is that it was added after they last worked on the AMI

@bkamphaus: also interesting how do you add custom metrics for datomic.

@lowl4tency: no announcements to make re: cloudwatch logs. CloudWatch metrics and S3 log rotation ( http://docs.datomic.com/aws.html#s3-log-storage ) are the main AWS monitoring tools, and custom monitoring for metrics is configurable http://docs.datomic.com/monitoring.html#sec-2

bkamphaus: thank you, second link is very useful

Yes, I’ve configured the s3 bucket for logging, but I’m interested in realtime log watching. For example AWS CW is able to do it with awslogs daemon.

It looks like this

zookeeper logs example

We ship our transactor logs to Elasticsearch and view with Kibana.

@robert-stuttaford: domkm I would be super interested in permission stories because we just about to design and implement one. (im aware of the superpowers video and showed it to all my colleagues too)

statonjr: logstash and s3 source?

statonjr: just I prefer use one place and vendor for same tasks

bkamphaus: also, do you use packer for AMIs?

bkamphaus: https://packer.io/

@lowl4tency: We host our own stack of Logstash, Elasticsearch, Kibana, and Riemann at AWS.

statonjr: understood, do you send metrics from Riemann to ELK?

No. TBH, we haven’t integrated Riemann into the rest of our stack just yet, but the plan is to have Riemann read the stream of data and act accordingly.

Does riemann have metric alarms for your stack?

Could someone give an example of the best way to test if a query returns a value?

@lowl4tency: we’ve got a hack to have Datomic ship logs directly via SQS to our Logstash instances

well… a “hack” 😉

@onetom perhaps you could join dom and i on a skype call soon

@robert-stuttaford: Thanks for the invitation, but Im in HKT (GMT+8) unfortunately, so I was sleeping and just woke up