This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-12-16
Channels
- # adventofcode (36)
- # announcements (30)
- # babashka (1)
- # beginners (161)
- # boot-dev (2)
- # bristol-clojurians (3)
- # clj-kondo (10)
- # clojure (125)
- # clojure-europe (10)
- # clojure-italy (4)
- # clojure-nl (7)
- # clojure-spec (1)
- # clojure-uk (26)
- # clojuredesign-podcast (3)
- # clojurescript (13)
- # core-async (5)
- # cryogen (3)
- # cursive (134)
- # data-science (8)
- # datascript (3)
- # datomic (32)
- # fulcro (24)
- # graalvm (2)
- # joker (5)
- # leiningen (5)
- # malli (18)
- # off-topic (14)
- # pathom (4)
- # re-frame (3)
- # reagent (11)
- # remote-jobs (3)
- # rewrite-clj (8)
- # shadow-cljs (47)
- # spacemacs (3)
- # sql (12)
- # vim (6)
Mornin'
Happy Monday 🙂
morning
måning
@mccraigmccraig: Do manifold/aleph provide any async wrappers to java.nio/nio2? Last I checked they depended on Netty’s underlying bytebuffer’s etc.
@rickmoynihan not that i know of - https://github.com/ztellman/manifold/blob/master/src/manifold/deferred.clj#L95
also relevant: https://github.com/ztellman/manifold/blob/master/src/manifold/stream.clj#L56
but obvs those are open sets, via the Deferrable
, Sinkable
, Sourceable
protocols
yeah that’s what I thought — my main issue with nio/async is using it in practice without accidentally blocking… e.g. our 3rd party data parsers and database drivers are all written to work with blocking I/O ; so whilst we could nio the interactions with services etc, we can’t nio the parsing/handling of the data.
Obviously we can pool the threaded/blocking sections and wait on them from deferred
’s or go
blocks, but it’d be a lot of work to validate it was correct, and there were no blocking actions in those stacks… e.g. calls through slf4j would need to be configured into async modes properly etc.
Incidentally does anyone know of any parser generators that either work with nio, or any examples where people target nio with a parser generator? Or do folk just hand roll parsers for everything? Or just convey the data from a blocking io thread onto a go
block or deferred?
yeah @rickmoynihan if you don't have non-blocking db drivers then async probably isn't going to work out very well - but for logging, why not just log to stdout and be done with it ?
Because despite the balkanised logging ecosystem, it’s actually really powerful and gives you a lot of useful power to configure logging in different environments.
Much better than println
, once you get slf4j setup with your logging backend.
swings and roundabouts i guess - we use stdout via timbre/tools.logging/slf4j with fluentbit/k8s log collection to elasticsearch which keeps the jvm/clojure side very simple
we actually do have some incidental blocking in our clojure - we aren't using nio for local-fs access, but we only rarely use the local-fs for temp storage, so i'm ok with wrapping it in a future :man-shrugging: (always deferred/future
, not clojure.core/future
)
@mccraigmccraig ok so you are using logging.
Incidentally isn’t println
blocking? I’m guessing you’ve set it to :async?
in timbre.
@U11EL3P9U: we’re not using SQL.
we're not using SQL either
yep, println
is blocking and despite timbre :async?
there's plenty of dependency code which calls tools.logging
or slf4j
method which will probably block (maybe - we're not using a standard appender, so i should look at that)
but it's not waiting for rust to spin or network, and our apis do saturate cpu under load so they aren't suffering from stdout
write backpressure