Fork me on GitHub
#clojure
<
2022-12-30
>
pavlosmelissinos08:12:50

Hi! I've been facing an issue in a concurrent application (pedestal-based) where the value of a var set with binding sometimes isn't inherited by children threads. I've noticed that spawning threads with future (e.g. via pmap) does keep the entire context whereas Thread. loses it. I looked at https://github.com/clojure/clojure/blob/527b330045ef35b47a968d80ed3dc4999cfa2623/src/clj/clojure/core.clj#L7039-L7073 but I don't get what makes the difference. Any pointers? https://github.com/clojure/clojure/blob/38bafca9e76cd6625d8dce5fb6d16b87845c8b9d/src/clj/clojure/core.clj#L1971? The context is structured logging; I'm setting some attributes and expecting to have them available in all spawned threads. It's a web server so I don't (fully) control how/when the threads are spawned.

pavlosmelissinos08:12:38

Ooh, that makes sense

pavlosmelissinos08:12:30

Hmm, looks quite similar in spirit to bound-fn :thinking_face: I was going to take functions that are likely to be run in another thread (e.g. http handlers) and wrap them with bound-fn but this seems a bit better for the job. Still, something seems off, this can't be the best way to solve my original problem... edit: Darn, binding-conveyor-fn is private, didn't notice it before

phill10:12:25

Pedestal can do wonderful things with core.async... It seems to me that you might get (binding...) working case-by-case, but as a system-wide architecture it might be delicate. You might consider making the context explicit: Stuff it into the Pedestal "context", then within Pedestal interceptors either pass to further functions as a parameter, or partially-apply functions that must do logging, or put the logging context in closure with anonymous functions that write to the log. Doing so might be tedious, but it might also be better than wrong or misleading statements in the log.

👍 1
pavlosmelissinos11:12:33

Right, doing it explicitly would take care of logging that is owned by my application but a) like you said, it would be quite tedious and b) I think it wouldn't be able to handle logging from third party libraries (e.g. pedestal itself) because they wouldn't know what to do with the extra stuff. I could ignore those logs but then Datadog (which is what we're using as a log aggregator) would have "orphan" logs with missing attributes (e.g. when an HTTP request is made)

tatut12:12:13

Weird clojure.tools.logging effect, I’m seeing clojure.tools.logging$eval10882$fn__10887 invoke line before each message 🧵

tatut12:12:26

user> (clojure.tools.logging/info "foo")
Dec 30, 2022 2:34:53 PM clojure.tools.logging$eval10882$fn__10887 invoke
INFO: foo

tatut12:12:06

I have clojure.tools.logging and there’s slf4j and the JUL logging impl org.slf4j/slf4j-jdk14 in the classpath, so that should be the logging impl

lread15:12:01

Hi @U11SJ6Q0K, do you have a minimal deps.edn or project.clj? If so, can try on my dev box too.

hiredman17:12:25

That is the default unconfigured jul outout

gratitude-thank-you 1
hiredman18:12:33

The string with $fn in it is the name of the class (what is on the stack) when jul's logging method is called

hiredman18:12:26

And it is always a class in toos.logging when you use tools.logging

lread14:12:02

ah! makes sense!

tatut05:01:32

the 2 argument (source) for format patterns, so pretty useless (if you only have clojure things emitting logs)

escherize20:12:16

Dealing with a 400 line map with some very long and mostly short keys, and came up with.. something:

{:a #_-------------------------------------------------------------------------------------- 12
 :abcde #_---------------------------------------------------------------------------------- 10
 :b #_-------------------------------------------------------------------------------------- 13
 :abcdefghijklmnopqrstuvqxyzhijklmnopqrstuvwxyzabcdefghijklmnopqrstuvqxyzhijklmnopqrstuvwxyz 40}

p-himik21:12:38

Yikes. At the very least, Cursive aligns maps in the same way without the need for manual comments. Perhaps, zprint and/or cjl-format have some flags for similar formatting.

escherize21:12:13

yeah, they look like this:

{:a                                                                                          12
 :abcde                                                                                      10
 :b                                                                                          13
 :g                                                                                          13
 :e                                                                                          18
 :c                                                                                          13
 :d                                                                                          13
 :f                                                                                          13
 :abcdefghijklmnopqrstuvqxyzhijklmnopqrstuvwxyzabcdefghijklmnopqrstuvqxyzhijklmnopqrstuvwxyz 40}
Can you tell me which key maps to 18? Or do you find that difficult

p-himik21:12:12

This is how I see it. Visualizing whitespace helps. :)

1
p-himik21:12:52

And with long keys, I usually format maps like this:

{:a
 12

 :abcde
 10

 :b
 13

 :g
 13

 :e
 18

 :c
 13

 :d
 13

 :f
 13

 :abcdefghijklmnopqrstuvqxyzhijklmnopqrstuvwxyzabcdefghijklmnopqrstuvqxyzhijklmnopqrstuvwxyz
 40}

escherize21:12:57

that would work, might make it a little more difficult to sort though

escherize21:12:30

whitespace-mode is a nice trick — thanks

👍 1
Tiago Dall'Oca23:12:48

Good evening guys! I just https://clojureverse.org/t/any-examples-of-clojures-stm-being-used-in-the-wild/9665 to ClojureVerse asking for real life applications making use of STM, in which cases would it fit, patterns to spot, etc. 🧵

Tiago Dall'Oca23:12:51

I think it is really cool as a concept that I would like to be able to use more

Tiago Dall'Oca23:12:17

but wouldn't if it wasn't a good fit

seancorfield02:12:22

The reality is that atom is fairly widely used for controlled mutation but ref is very rare in the "real world".

1
seancorfield03:12:10

We have quite a few atom and agent calls in our codebase at work, but no ref calls:

Clojure build/config 22 files 406 total loc,
    177 deps.edn files, 3245 loc.
Clojure source 551 files 107736 total loc,
    4733 fns, 1104 of which are private,
    640 vars, 44 macros, 103 atoms,
    86 agents, 22 protocols, 64 records,
    820 specs, 25 function specs.
Clojure tests 565 files 26410 total loc,
    5 specs, 1 function specs.

Clojure total 1116 files 134146 total loc

Polylith:
   21 bases 434 files 75426 total loc,
   135 components 672 files 58281 total loc,
   21 projects 10 files 439 total loc,
  total 1116 files 134146 total loc.

Ed12:12:33

In the past I've worked on a system that matched the classic model of a bank transaction. Lots of entities that are created infrequently that have atomic changes that are made to small numbers of them concurrently. So, something like an atom containing a map of account number -> a ref of account balance. So transferring money between 2 accounts only locks those accounts, and you can use agents to record the side effects. It was more complicated than just having a single atom but it did allow a higher throughput of transactions for our scenario. But I think that's something you'd need to performance test for your own app.

phronmophobic18:12:19

This search has false positives, but you can find examples usages of ref in the wild at http://grep.app https://grep.app/search?q=%28ref&amp;filter[lang][0]=Clojure