Fork me on GitHub
#observability
<
2022-01-12
>
timo14:01:45

Hi, do you guys use ulog or ken in conjunction with metrics from iapetos or do you don't need that anymore? How do you see a timeline of performance issues in something like Grafana with ulog/ken?

timo15:01:14

Looking at mulog samplers right now

greglook18:01:21

We use ken (primarily with http://honeycomb.io) but we also send events marked with certain keys to our metrics system, where they get recorded to InfluxDB and from there surfaced in Grafana dashboards. Main thing you'd do is write a function to subscribe to the ken tap and then record your events to your backend of choice.

timo20:01:40

so you would then probably compute something like counters/gauges/histograms within the backend of choice? (sorry, i only know prometheus where you get it properly served on /metrics)

timo20:01:08

actually watching your talk right now 🙂

👍 1
greglook23:01:08

Ah, I understand what you're asking better now - yeah, in our case you can do aggregates within the InfluxDB query itself, so basic stuff like sum/count/mean/p99 etc are directly computable on the raw metrics. We actually run http://riemann.io/ as the destination for all our metrics, which is then responsible for writing to influx - but it also gives us a place to do more sophisticated logic to power alerts and decide when to post to slack or page someone.

👍 1