Fork me on GitHub
#clojure-uk
<
2019-12-16
>
dharrigan06:12:56

Good Morning!

seancorfield07:12:20

Happy Monday 🙂

thomas08:12:32

Morning folks

rickmoynihan09:12:48

@mccraigmccraig: Do manifold/aleph provide any async wrappers to java.nio/nio2? Last I checked they depended on Netty’s underlying bytebuffer’s etc.

jiriknesl09:12:44

Good morning

mccraigmccraig09:12:39

but obvs those are open sets, via the Deferrable, Sinkable, Sourceable protocols

rickmoynihan10:12:57

yeah that’s what I thought — my main issue with nio/async is using it in practice without accidentally blocking… e.g. our 3rd party data parsers and database drivers are all written to work with blocking I/O ; so whilst we could nio the interactions with services etc, we can’t nio the parsing/handling of the data. Obviously we can pool the threaded/blocking sections and wait on them from deferred’s or go blocks, but it’d be a lot of work to validate it was correct, and there were no blocking actions in those stacks… e.g. calls through slf4j would need to be configured into async modes properly etc. Incidentally does anyone know of any parser generators that either work with nio, or any examples where people target nio with a parser generator? Or do folk just hand roll parsers for everything? Or just convey the data from a blocking io thread onto a go block or deferred?

mccraigmccraig10:12:28

yeah @rickmoynihan if you don't have non-blocking db drivers then async probably isn't going to work out very well - but for logging, why not just log to stdout and be done with it ?

rickmoynihan10:12:42

Because despite the balkanised logging ecosystem, it’s actually really powerful and gives you a lot of useful power to configure logging in different environments. Much better than println, once you get slf4j setup with your logging backend.

mccraigmccraig10:12:45

swings and roundabouts i guess - we use stdout via timbre/tools.logging/slf4j with fluentbit/k8s log collection to elasticsearch which keeps the jvm/clojure side very simple

mccraigmccraig10:12:16

we actually do have some incidental blocking in our clojure - we aren't using nio for local-fs access, but we only rarely use the local-fs for temp storage, so i'm ok with wrapping it in a future :man-shrugging: (always deferred/future , not clojure.core/future )

dharrigan10:12:22

How about the new reactive, non blocking jdbc drivers?

rickmoynihan10:12:43

@mccraigmccraig ok so you are using logging. Incidentally isn’t println blocking? I’m guessing you’ve set it to :async? in timbre.

rickmoynihan10:12:51

@U11EL3P9U: we’re not using SQL.

mccraigmccraig10:12:12

we're not using SQL either

mccraigmccraig11:12:00

yep, println is blocking and despite timbre :async? there's plenty of dependency code which calls tools.logging or slf4j method which will probably block (maybe - we're not using a standard appender, so i should look at that)

mccraigmccraig11:12:05

but it's not waiting for rust to spin or network, and our apis do saturate cpu under load so they aren't suffering from stdout write backpressure

jasonbell10:12:51

Morning

👋 4