Fork me on GitHub
#off-topic
<
2022-05-07
>
adi11:05:43

I don't know who needs to hear this, but I got bitten by grep's stdio buffering behaviour today. Frankly I'm astonished it did not bite me all this time! Here is https://www.perkin.org.uk/posts/how-to-fix-stdio-buffering.html. TL;DR, copied from the post: This will apparently do nothing (really, it's just buffering)...

$ tail -f /var/log/foo | grep -E -v 'some|stuff' | sed | awk
Using stdbuf -o0 fixes it (presumably with performance penalty, because of force-flushing every line to stdout).
$ tail -f /var/log/foo | stdbuf -o0 grep -E -v 'some|stuff' | sed | awk

gratitude-thank-you 1
apiology15:05:49

If MODE is 'L' the corresponding stream will be line buffered.  This
       option is invalid with standard input.
I’ve never used this before, but based on the man page, I think -o0 means flush with every byte. I wonder if -oL would be a bit faster and still do what you intend.

Cora (she/her)18:05:27

there's also unbuffer, a cli tool that is part of TCL expect

👀 1
adi04:05:49

@U060KDHA6 thanks for the reference! Guess who read the blog post but not the manpage?

😂 1
apiology13:05:35

I’d guess it doesn’t make too much of a difference in practice, as the producer is probably producing things in larger chunks in most cases anyway.

👍 1
adi21:05:49

This minor snafu was on the way to getting a form of hot reload to work in the little static site generator I'm yak-shaving making for myself... And the reload is heating up! https://twitter.com/AdityaAthalye/status/1523776563241955328