I think we had to drop down to Java interop for this with log4j2 (and tools.logging). I'll try to remember to look in our code at work tomorrow.

OK, here's what we did for log4j:

(defn stringize-context
  "Given a logging context (that is hopefully a hash map), return a
  hash map with strings for keys and for values. If the context is
  not a hash map, return a hash map with a ctx string key and the
  context value as a string."
  [ctx]
  (if (map? ctx)
    (reduce-kv (fn [m k v]
                 (assoc m
                        (if (keyword? k) (name k) (str k))
                        (when (some? v) (pr-str v))))
               {}
               ctx)
    {"ctx" (pr-str ctx)}))

(defmacro with-log-context
  "Given a hash map and a body (sequence), add the hash map to the log4j2 mapped
  diagnostic context, and execute the body."
  [ctx body]
  `(with-open [_# (CloseableThreadContext/putAll (stringize-context ~ctx))]
     ~@body))

(with-log-context {:uid user-id :ip ip-address}
  ...)

I see… I was hoping for something like Pedestal log, where users will run e.g.

(log/info :message "hi" :foo "bar")

According to SLF4J's docs: "Note that at this time, only two logging systems, namely log4j and logback, offer MDC functionality."

But this is the API SLF4J offers: https://www.slf4j.org/api/org/slf4j/MDC.html

Earlier this year I made a variation of this https://github.com/clojure/tools.logging/blob/e9ac5b177985f1ecefd2ff3bddf8c723fe281bb5/src/main/clojure/clojure/tools/logging/impl.clj#L79-L80 such that ^String msg# (str msg#) was instead ^String msg# (to-json-str msg#). GCP, Datadog, and likely others recognise when output can be parsed as json, and then show it as data, when using their UI and querying capabilities. It was very nice IMO - one keeps using clojure.tools.logging's API with only relatively minimal changes underneath. For me it was preferable to reaching for a different library with its own conventions, and most importantly falling into the trap of "structured logging" which often means baby-sitting infrastructure.

Have you looked into mulog? It might fit your needs.

Cambium: https://cambium-clojure.github.io/

I have seen mulog and Cambium before. How do users of these libraries log then? Do they simply add the SLF4J bindings to their classpath and use the libraries directly instead of using clojure.tools.logging or SLF4J?

I was hoping that tools.logging could offer a way to use a “unified API” for structured logging. But if it doesn’t, then is there value in using tools.logging if I choose mulog or Cambium as my logging library?

Well the api of tools.logging isn't really data-oriented.

Log4j2 can log shallow maps as data rather than as a string. Constructing a MapMessage is straightforward: https://github.com/steffan-westcott/clj-otel/blob/master/examples/common/log4j2.utils/src/example/common/log4j2/utils.clj To emit the MapMessage as JSON in a log file, you can use JsonLayoutTemplate like this: https://github.com/steffan-westcott/clj-otel/blob/master/examples/microservices/manual-instrument/middleware/random-word-service/resources/log4j2.xml

Don't type-hint nil.

(let [^GrayU8 some-prop nil] (ConvertBufferedImage/convertFrom input some-prop)

It's needed for overloading.

you can put a typehint on a local

@U11BV7MTK thank you. Is there a more elegant way?

not that i know of

There will be in 1.12

We are adding some new arg tag metadata to explicitly specify overloads and that could be used in this situation

(^[BufferedImage GrayU8] ConvertBufferedImage/convertFrom input nil)

oooh... is that applicable to vararg calls too?

^[] will be new syntax mapping to a new meta attribute (like ^Class gets mapped into :tag meta) - it must explicitly narrow to exactly 1 overload

that's very cool

it will apply to varargs, but not sure yet if that will actually make that problem much better :)

😉 ... so I'll still have to say it`s an array at the end?

would it allow wildcards? (^[_ GrayU8] ConvertBufferedImage/convertFrom input nil) to just help the overload picker? Or does it have to fully specify every arg?

➜  ~ clojure
Clojure 1.11.1
user=> (defrecord _ [])
user._
user=>

...not sure it could

yes, it allows _ for slots that are unambiguous

@U064X3EF3 How would one describe an overload that takes primitive longs?

with primitive type hints

^[long] (Thread/sleep (/ 1 2)) ?

yes

well, no wrong position

(^[long] Thread/sleep (/ 1 2))

you're hinting the member symbol

nice

also works for instance methods and constructors

so, it's especially meant for "in-place" type hinting and not something that is going to be used for inference, right?

I guess it is in cases like:

(let [x (^[foo] (Static/method some-foo)]
  ;; type of x is known as a return type of Static/method overload
  (Static/otherMethod x))

well first

(let [x (^[foo] Static/method some-foo)]
  ;; type of x is known as a return type of Static/method overload
  (Static/otherMethod x))

I sometimes wonder if the compiler could have generated

(if (instance? Foo x) (Static/method ^Foo x) (if (instance? Bar x) (Static/method ^Bar x) ... fall back to reflection here ...)

it will be used for inference as knowing a method now would be used for inference

we considered something like this (with reified overloads) for method thunks but our general impression is that that is rarely want you actually want. almost always you want to use a specific overload.

How does lein checkouts work?

:local/root is pretty similar to Lein checkouts - or even better in a few aspects Note that changing a namespace in the dependency won't automatically reflect it in your consumer project. You still need to (require ,,, :reload) or some other method

i’m struggling to think of a way for this not to be the case. it’s essentially just putting those folders onto your classpath

it is exactly tha

example: {:deps {net.clojars.kwladyka/consistency {:local/root "../../consistency-clj"}}}

Oh thanks @U45T93RA6!

they can put them in their ~/.clojure/deps.edn file. in the readme of your project make a template that they can splat into their own home dir deps.edn. :project/my-name {:extra-deps {lib1 {:local/root "path to lib1…"}}}

I don't like the global config approach because it's outside of version control and generally detached from the project. Is there an advantage to it that I'm not seeing?

I should add that I'm quite happy with this approach, but haven't tried it on larger teams.

pardon my ignorance about this, but why would one developer have different dependencies from another? That sounds like a bad practice to me.

Personal tooling, editor extensions, etc. Plus different locations for local deps.

The system must build and pass without the personal deps.

Some of that personal tooling probably goes in the global deps, but sometimes it may need per-project config, etc.

oh, I’m new to clojure. I’m used to nodejs where you can put that all in devDependencies, and then all devs share that list. Is there such a thing as devDeps native to deps.edn?

because if not, then I think I’m catching up in why you’d have these separate lists for local dev this way

If devs want to share an alias, this convention would allow it, but I don't see a reason to force the issue. Anyway, Clojure Deps is flexible enough that shared aliases can be used in addition to personal ones.

My reason for posting was to see if existing teams may have experience with an idea like this they wanted to share. I haven't seen anyone using this or anything similar, but it's likely that these problems primarily show up in closed proprietary projects with teams working on them more often than in open source projects. Also, I thought it may be good to share it as an idea in case it may be useful to others.

Maybe it would work to save e.g. darrick.edn and pass it as an external source? https://clojure.org/reference/deps_and_cli#deps_sources

As I mentioned, I'm not looking for solutions to a problem here. I'm quite happy with this idea. I wanted to present an idea that seems to work well and seeing if others have experience or had run into gotchas that I wasn't seeing.

I think I should have made that clearer in my original post. Oops!

straightforward steps are to copy all of the deps from lein into the new format. then use https://github.com/babashka/neil to run neil add build. Work on getting the project to run locally and then getting the project to build as you used to.

https://github.com/borkdude/lein2deps

you'll probably want a :repl alias that provides nrepl dependencies if you used lein repl, and you'll probably want a :run {:main-opts ["-m" "YOUR-MAIN-NAMESPACE"]} alias if you used lein run

https://github.com/practicalli/clojure-cli-config seems a good place to start - worth checking out both the readme and implementation :)

Converting dependencies to deps.edn is the easy part. I recommend learning how tools.build works first, so that you can actually compile your code, run tests, and create uberjars.

https://clojure.org/guides/tools_build

code at rest is indeed static, clojure source, jvm bytecode, arm assembly

A running process is started by the code, but it's shape and structure is not reflective of the static structure of the code

but I've also seen that we can monkey-patch and update existing functions in the repl (until that program restarts, at least)? my initial mental model is Clojure -> Bytecode -> JVM, whats happening when I make an update like that?

Every expression you type in a REPL goes through the Clojure->bytecode step, i.e. the Clojure compiler compiles it to JVM bytecode.

This JVM bytecode is immediately loaded into the running JVM as a class on which methods can be called, without restarting the JVM. The JVM provides the ability to dyamically load classes while it runs.

The only caveat is direct-linking. Clojure itself is compiled with direct-linking so it's hard to redefine bits of it dynamically. If you AOT-compile code for an uberjar and use direct-linking, you remove a level of dynamism and that can make it harder to modify the running program with a REPL.

(if you redefine a function, you have to "redefine" all the callers, all the way up the chain, to guarantee your new definition takes effect fully)

Many other systems allow dynamic loading of code, too, but how often people take advantage of those capabilities depends upon the language/system and the developer. e.g. C programs on Linux can load dynamic libraries at run time, and call functions made available through them. If you use Linux or macOS (or Windows), this happens almost every time you run a program, most often used to reduce the size of the compiled binaries, not to dynamically create new code on the fly. But it can be used to create new code on the fly and load it into a running program.

That all actually makes sense! I never actually thought of linking in that way. I very much appreciate the education/de-magification ❤️

It's probably worth mentioning, if you have .class files on the classpath (either as individual files or as part of a JAR), you can still define/redefine stuff via the REPL -- see the tricks to improve REPL startup etc where you compile parts of your app locally and have classes on the classpath. First time you require a ns, it loads the compiled code but as you change source and eval code, that newer code is what is compiled and run.

I know this is a major necropost, but I think it might be important because I don't think this was directly addressed: The actual mechanism of updating code in a compiled jar is that we don't update code in the jar. What happens is that there is a new bit of code compiled that represents the new implementation of a function. Then once that code is loaded, it looks at a runtime object that sits inside the runtime representation of a namespace, that object being the var, and it takes a mutable pointer inside it and swaps it out to point at the new compiled code. All the old code that was compiled against the old function is actually compiled to take several steps: • on init, load up the var from the namespace and store a reference • at runtime, dereference the var to get an object • cast the object to something that implements the IFn interface • call .invoke() on that interface implementation This means that old code which refers to the var will find the new code the next time it dereferences the var. This process is a little different with direct linking turned on though. Instead of loading the reference on init and dereferencing at runtime, it loads and dereferences the var at compile time and encodes a hard reference to the implementation's method into the bytecode, which means that updating the var at runtime will do nothing to existing code compiled with direct-linking (it will be found by code not compiled with direct-linking, such as code from the repl)

maybe helpful to consider that this is functionally equivalent to "automatic rename" and look for that

I don't know of such a tool but seems like that's the hard part

fwiw, it's not that much code to write a printer that works exactly the way you want and it's often much better for your sanity than try to coerce pprint to a particular format.

> it's not that much code to write a printer that works exactly the way you want Writing a printer is actually just what I'm doing. 🙂 The annoyance here is that'd I'd like to test it by comparing my printer's output to that of pprint, but since my printer builds on top of pr, I'm getting different results.

I think in this case pprint is more logical and consistent than pr, but 🤷 .

my opinion is that screenspace is a better metric than print-level, but it depends on your use case.

I also think clerk's "viewer budget" is a neat idea (it's also similar to a "screen space" metric).

Yeah, are definitely better UI solutions to the problem print-level tackles. 👍:skin-tone-2: I’m just looking to make a (much) faster pprint here, though.

While I agree, it's not really a good reason to avoid starting a thread. I'd start a thread under the Alex's message while @-ing flowthing, the OP. Or at the very least, if you're posting in the main channel, IMO it's much better to post everything in a single message or edit the first message to add some details. I'm one of the "spam killers" here and I'm subscribed to many channels. While I don't receive any notifications, I still have an unread mark on the tray icon and usually react quite quickly to it - quicker than you were posting additional details, so I got to check Slack around 7 times more than needed.

sorry, my bad

> maybe pr doesn't apply print-level to scalars? It doesn't, as per the docstring, but map entries are colls, so it should apply to them.

if we talk about the print-level thing on this thread we will likely get told this is the wrong thread

You won't be told so, pinkie promise. :)

so as a mod what you are saying is you feel like you have to leap in to action for every public message

and messages in threads, although "public" are not moderated?

> you feel like you have to leap in to action for every public message No, I don't feel that way. I just think that monitoring things and acting when necessary is a good way for me to invest time and effort into this community. > and messages in threads, although "public" are not moderated? They are. Sometimes by mods checking the threads directly, sometimes by people reporting issues.

it seems like you are asking me to reduce the burden on the (volunteer) mods by being aggressive about using threads, so I am trying to understand what additional moderation burden arises from not using threads

It's not just moderation burden, and that part I have covered for at least myself personally - where I check Slack more often than really necessary. It's also burden on every other reader and writer here, apart from those directly involved in the most current conversation. It's been discussed before, both privately and publicly, plenty of times, the consensus is always the same for the most active/populated channels.

sure, and I get that people want threads, so I use threads, I've had that talk

Yeah, and rest assured that we value that. All I wanted to convey is that starting a thread doesn't have to be right under an OP.

but then as soon as there is a conversation that is not using threads, the community is very selective about who they decide needs to be reminded to use threads

Personally, I haven't noticed people being selective. But it might very well be because I almost always just start a thread whenever it's not worst than answering in the main channel.

In this particular case, the person that left the reaction is a regular participant, not a member of the mod/admin team.

sure, which is why I said "the community" and not "the mod team"

"regular" can mean many things

I say that because "regular participant" might mean someone who is not a mod, or someone who participates often

FWIW, I myself always let newcomers know when they post multiple consecutive messages that such messages shouldn't be separate from the main one - they should either be in a thread or a part of the main message. Out of curiosity, I just scrolled back #C03S1KBA2 all the way to June. There were around 5-10 instances of 2 messages with the second one being in the main channel - nobody got reminded about anything, as far as I can tell. The only 2 instances (that I noticed) of people sending more than 2 messages related to the same topic outside of a thread were made by you, where you weren't reminded of anything (here: https://clojurians.slack.com/archives/C03S1KBA2/p1693679194814309), and by another person, where they did get reminded here: https://clojurians.slack.com/archives/C03S1KBA2/p1689350153277869 Of course, two messages don't make a statistic. But I hope you can excuse me not wanting to go further than 2 months back at almost 1 am here. :)

please, don't feel compelled to do something you don't want to do on my account

I only want to make sure that we're on the same page.

does it matter if we are not?

my conversions with the mod team are what I believe ultimately got https://github.com/clojurians/community-development/blob/master/welcome-message.md made available somewhere so whatever "policy" exists on the use of threads can actually be read if you joined the slack so long ago that you don't ever remember getting the welcome message

so I believe I am aware of the general stance of the mod team

> does it matter if we are not? In all honesty - it does, but there are different kinds of "matter". If nothing changes, threads keep being created less frequently than reasonably useful, and you keep feeling that there's seemingly a witch hunt, then I'd like to understand what can be done to make it better for everyone, if anything reasonable can be done at all. If the visible behavior changes but you still feel that it's not fair or not right in some way, I would also like to change that to the better, but here I'd have much, much less leverage to do anything except for a vague discussion. > so I believe I am aware of the general stance of the mod team Now I'm starting to think that perhaps I'm not seeing something. If that's the case, why are we having this whole conversation, why did it get started, given that there's that third bullet-point?

you started the thread to talk about threads

As a response to your message about actively deciding not to use a thread, yes. As a point of reference: I have posted dozens, maybe hundreds, of messages asking people to reply in a thread. Pretty much always the reaction is the same - different varieties of "OK". Regardless of the circumstances - whether the person being asked is the OP or someone responding to them. You got asked for a thread very indirectly, by a non-invasive reaction that you can't even possibly be notified about (well, AFAIK - maybe Slack got changed again). For the first time in months, at least in this channel. And your reaction is "no, not happening". Why? IMO even not sending that particular message would've been better - the next person posting a reply would be much more likely to start a thread somewhere, now that there was a 🧵 reaction under your latest message.

I am not sure why there are quotes around "no, not happening" since that is not what I said

It wasn't a direct quotation, it was conveying of meaning.

meaning of the written word is famously ambiguous

I did not say I would never use a thread, I did not say I would not move to a thread

I said that due to the question asker and the first responder both being in the channel, but not in a thread, the way to ensure they both see a response is to also send to the channel

if I started a thread under the initial question, the first responder might not see it, and under the first responder the initial asker might not see it

"hey, the first person to respond wasn't in a thread, so the conversation is not in a thread"

I see, so it seems that I misunderstood that particular message. But it changes little, I think. Don't you agree that starting a thread under Alex's message and @-ing the OP would be the best way to respond to both people while also not polluting the main channel with multiple consecutive messages? If you don't, then why?

I reject the idea that messages about clojure in #C03S1KBA2 are pollution to start with

the entire point of the channel is for people to talk about clojure stuff, so the idea that people talking about clojure stuff is "cluttering up the channel" is absurd

Pollution comes in many forms, that's the issue. Thematically, they're not an issue of course. Organizationally, any unnecessary top-level responses are an issue that can easily be solved with a thread. As an analogy - you do use subdirectories in your file system, right? Even if files form different subdirectories thematically belong to the common parent directory, you still might want to separate those with higher granularity.

When I want to have a discussion about something Clojure-related, I want to have a discussion with an interested party. I don't want to shout out into the room with thousands of other people and concurrently hear shouts from other people from across the room, on a completely unrelated topic, albeit still about Clojure. And I feel it's a very common desire.

I might, or I might not

on my filesystem at home it doesn't effect others

Exactly. And even there you sometimes choose to use subdirectories when it makes sense. Here, there are many thousands of people. I can't see how it would make more sense to send a message to everyone when it could've been sent in a thread.

so I can very well grant that some people find #C03S1KBA2 more legible if it is chopped into threads

but they are also in #C03S1KBA2 by choice

if they are not interested in clojure, maybe they shouldn't be in there?

That's exactly what my next-to-last message starting with "When" talks about. Being interested in something is not orthogonal to wanting to participate in an organized chat. Many more things can be said about choice here. People that are here have chosen this particular Slack server as well, one that has threads and moderation. They didn't choose an IRC channel. And that 🧵 reaction is indicative of that.

@U0482NW9KL1 why the 🧵 ?

Even if you treating that reaction as ambiguous proves to be correct, you can still treat it as what I've described, because I was about to just ask you to use a thread anyway, but you responded to the reaction a tad earlier.

and why were going to ask me and not @U064X3EF3?

AFAICT, we never react to immediate replies. I don't remember seeing even a single instance of that. Your reply was the third one, and it consisted of multiple messages spread over an interval of a few minutes.

so you were more annoyed by my reply?

The internal reaction was manifold. 1. The threshold of 3 was reached. Not an explicit threshold set anywhere - just something I learned by osmosis here (for an example, see a link to a message I left earlier, when scrolling back to June) 2. Multiple messages were posted in a rapid fire manner - it's a subset of the above point, but it registers in a different matter. An implementation detail, if you will 3. There was indeed a slight feeling of annoyance. But I ignore that - otherwise, as you can probably imagine, I simply wouldn't be able to be so civil (as far as I can judge myself) and wanting to reach an understanding instead of dispensing a prescription

I'm all for you two hashing out your differences here, but let me take this moment to share that I feel you are both awesome, helpful and much-valued contributors to the clojure community. I am sending you both a .

> @U0482NW9KL1 why the 🧵 ? > I simply wanted to put it out there as a mild nudge, I know it was a few out-of-thread-replies deep, but, you know, "be the change you want to see in the world" 😄. I didn't mean any offense by it! I also wish to see and appreciate Clojure-talk in Clojure 😆, but I did also have himik's same experience of reopening the channel multiple times because I kept getting an 'unread'-mark I do wish Slacks faculties for organizing messages/conversations was less terrible, there really should be a tool (at least for admins) that lets you move a selection of existing messages into a thread, so that it doesn't have to be so manual/ad hoc.

you can also turn notifications off, perhaps my slack experience is so different because I don't have notifications for every new message turned on

> there really should be a tool (at least for admins) that lets you move a selection of existing messages into a thread That tool would indeed be fantastic to have.

I can't help pulling this, sorry

> I don't have notifications for every new message turned on It's great if it works for you just fine. And perhaps Samuel would also benefit from it, I can't tell. But unfortunately, just disabling notifications doesn't solve the issue for many people, it somewhat alleviates it at best.

no, not right now (and maybe not ever)

unclear what that would mean once you start combining them

there can be only one local repo from a Maven perspective so this is part of the top-level procurer config

I need to override the local-repo for a run in CI but perhaps I can just change the CI directory

Not anywhere close to a maven expert

So thanks!

if you want to set outside, you can also do clj -Sdeps '{:mvn/local-repo "/foo"}'

Would that override one if it existed in the deps.edn?

yes

the "extra" deps.edn provided on CLI is last in the merge chain so "wins"

Had to ask 😄 Thanks!

lol sorry I told you incorrectly @U05NZDGDYG3 !