Does this help?: https://clojurians.slack.com/archives/C03S1KBA2/p1672403773765659

Yeah I looked at the code of clojure.tools.logging and it seems to have no support for logging function and line, just the namespace at best. Certainly inferior library compared to timbre. Shame.

I mean - you should be able to configure java.util.logging to get that

the info is there, at least the "logging function" - error/warn/etc.

+ you can try a different appender

I don’t think that’s the case, that configuration can fix it. I’ve also seen comments in timber code where it has provider for clojure.tools.logging to the effect that the library can only take namespace information, not function or line information

if they wanted to do this, clojure.tools.logging would need to use LogRecord classes to log, but they don’t.

let me look at the code for a moment

ah - well I know how it could be done

you are right, looking at the code only these things are explicitly passed to the logging backend

(write! [logger level throwable message]
    "Writes a log message to the given Logger.")

but, there should be a way - either with MDC, dynamic vars, or scope locals - to get that info down there

We've used c.t.l and log4j for ages and we don't miss function/line -- we just use something descriptive in the message to make it easy to tie back to something specific in our source. We do use MDC but that's typically for stuff like user ID, IP of the request, etc. Using standard Java logging under the hood is important to us for observability tooling (New Relic), which is also why we use Jetty rather than http-kit.

as a hack you should be able to add line number with macro. So whenever you call log fn you can call your log macro which add line number under the hood. That is one of a few reasons when you can want to use macro. https://www.reddit.com/r/Clojure/comments/sxwmt9/how_to_log_line_numbers_with_toolslogging/ Something like what I found in google. I didn’t test.

If i will really need lines, I will use macro. I think so.

But as Sean mentioned. I don’t have lines in logs for my hobby projects and I even didn’t realise it until this moment.

@U04V70XH6 ok but you could also use Timbre and an appender that posts to java.util.logging . If you don’t need correct namespace or function then that’s fine, but I’d love to see support for that in c.t.l and I am not sure why it’s absent. As a library author I avoid putting any logging in my libraries specifically because I don’t want to force any user to use Timbre or whatever specifically. So I wanted to use c.t.l so library users could use whatever. However since the lines look like this:

Oct 23, 2023 2:15:14 PM clojure.tools.logging$eval3668$fn__3671 invoke

The only reason I would say avoid timbre is that it pulls in a bunch of stacktrace prettification libraries by default, which makes reading logs wretched

In my java libraries, i've started just using System.Logger . Means consumers need to pull in the slf4j adapter usually, but i don't have any consumers

@U66G3SGP5 We tried Timbre for a while and disliked it a lot for all sorts of reasons. We have everything going through log4j2 now -- every bridge-to-log4j -- and we can configure it centrally and override that as needed, it supports MDC, and works well with New Relic log forwarding. We have multiple log outputs (rolling detailed logs), plain console logs with less detail. And we have a bunch of smart email reporting wired into log4j2. I also have my dev setup with c.t.l wired into Portal so I can expect log messages in my editor too. I agree that it would be nice for c.t.l to have an optional way to add line numbers to the logging tho'. Someone should post a request on http://ask.clojure.org 🙂

> but seems I am using it How exactly are you using the result, without doall?

Just in case, how I'd write that code if I didn't need lazy collections at all:

(into []
      (comp (partition-all 5000)
            (mapcat (fn [users] ...)))
      users)

The code is in a function which I call to get the results basically, interesting mapcat is not something I use that often

It's flatten that you shouldn't be using that often. ;)

fair enough, still interested in why I need the doall with the above, I will change what I have to use mapcat and comp

> still interested in why I need the doall with the above It depends on how the data is used. I don't think there's anything that can be said without that detail.

Now, if you (count ...) it and get different numbers, then it's extremely suspicious.

so basically I am calling the above code, passing it through a function that maps it into another structure then I am pushing it into another function which inserts data

but I was calling the above with count to debug what was happening which is when I noticed I got less results

(defn fetch-data [users]
  (doall
   (->> (partition-all 5000 users)
        (map-indexed (fn [idx users]
                       (-> (sqlh/select [:email :email]
                                        :first_name
                                        :last_name)
                           (sqlh/from :accounts)
                           (sqlh/where [:in :%lower.email users])
                           (db/query))))
        (flatten))))
 
(def data
  (fetch-data [""]))
 
(count data)

Oh, one other thing - seems like db/query is using a dynamic variable to get the DB connection. Do not mix dynamic variables and lazy collections, you're gonna have a bad time unless you're extremely careful:

user=> (def ^:dynamic *x* 1)
#'user/*x*
user=> (let [c (binding [*x* 2] (repeatedly #(identity *x*)))] (take 1 c))
(1)

ah that will be it then cheers, now I need to understand why you should not mix them 🙂

The biggest problem is that by the time the lazy collection gets around to doing some computation, the dynamic binding will likely no longer be in effect.

@UU67HFS2X https://clojure-goes-fast.com/blog/clojures-deadly-sin/#dynamic-bindings

Thanks make sense now, I remember reading that article before but the refresher helped, probably forget again by time I hit the issue again 🙂

Actively avoiding laziness might be easier than trying to remember all the intricacies, you know!

https://stuartsierra.com/2015/08/25/clojure-donts-lazy-effects is a broadly supported idiom, regardless of broader opinions on laziness

Here’s a slightly earlier post where he shows a video of the way he’s using my stuff to synchronize his CDJs with tracks playing in Ableton Live: https://www.instagram.com/p/CyoXCTsB2H3/

Oh, and the reverse-engineering documentation I created to help other people build similar things in other languages uses sci to provide a nice domain-specific language for embedding byte-field diagrams inside AsciiDoc documents, which has proven to be an even more effective stealth avenue for bringing other kinds of people into the Clojure fold. 😆

That's awesome, once we see al DJs wearing t-shirts with parens, the Clojure community has won 😉

On a related note, I'm looking into Max/MSP's Java support but I'm not sure how the performance is going to be :thinking_face:

That is amazing. Congratulations.

> “a weird as :face_with_symbols_on_mouth: scripting lang” - Deadmau5 This should be somewhere!

@U0JEFEZH6 I had previously played a bit with ClojureScript + https://docs.cycling74.com/nodeformax/api/ and it worked well!

@UA2U3KW0L oh yeah, that's an option too - and wouldn't require the JVM to be installed, I'll look into that!

Nice one 💪 I invested pretty heavily into cdj and ableton stuff in the past, but sadly I don't have as much time/energy these days. Would have loved to hack on Deep Symmetry 😿

He adds "unfortunately" which is a bit of a letdown 😞

Yeah, I think that may be general JVM negativity, which does still exist out there!

@U0JEFEZH6 I did embed Afterglow, my Clojure live-coding lighting controller, in Max/MSP and it seemed to work fine, but the direction there definitely seems to have shifted to Node/JS lately and Afterglow is heavily dependent on a lot of JVM libraries so I have not explored working with ClojureScript there.

Nice, my needs are much simpler as I'm mostly looking into MIDI effects... but just as @U45T93RA6 there's just no time to look into this stuff

wow- very cool stuff @U0EHA00G1! I'm going to try really hard not to fool around with any of it for a few months (tight deadlines).

I have a project deps.edn with :dev alias, which sets up jvm-opts and extra-paths. Then in my ~/.clojure/deps.edn my :dev alias adds a dependency, a collection of REPL helpers, and it brings a couple of other things like Portal. The global dev alias uses :extra-deps to add it. Last thing is :nrepl alias in my project's deps.edn, which pulls in nREPL and uses main-opts to start it. The issue I'm running into is that when I run clj -M:dev:nrepl and try to require my helper library, I get an error that it's not found on the classpath. I read the guide an it's not obvious what's going on or how to debug it.

can you share the actual aliases from both deps.edn's?

if it's a lib issue, might also be useful to check your deps with clj -X:deps tree :aliases '[:dev :nrepl]'

Thanks, here are the relevant parts: https://gist.github.com/lukaszkorecki/7d3f4d2cf28dd18fc0b1c622a99586d6 When I run deps tree, I can see my helper lib being pulled in along with its dependencies

that doesn't seem exactly to match what you said above - it's including nrepl in you main deps (seems weird) and not in the :nrepl alias, but if you're seeing your helper lib being pulled in then it's probably something else. what does your helper lib deps.edn look like, and are you sure you've not mis-typed the namespace of your helper lib namespace?

can you load a repl in the helper lib and do the same require?

I guess it's just (require 'r) ?

Yeah, it should be (require 'r) - running that in the clj repl in my library works just fine. As for the nREPL dependency is also used by the application in production, I can connect to a live process Just In Case ™️

I tried just doing

% clj -Sdeps '{:deps {io.github.lukaszkorecki/rumble {:git/sha "45eac130737b8049821c45859d374c1db1d82471"} nrepl/nrepl {:mvn/version "1.0.0"}}}'
Clojure 1.11.1
user=> (require 'r)
> c > #'r/c > ([component-name])
Pul out a compont from a running system, pass keyword for the component name
...

mysterious, I wonder if I have some other deps.edn somewhere messing that up

is there a way to trace how deps.edn files are merged?

the tree thing you did above should be loading the same set of deps

if you did clj -Sverbose -M:dev:nrepl that should print a cp_file location in your .cpcache (that's the actual classpath used) and you can find the basis file (merged deps.edns) at the same path but .basis instead of .cp

I don't know if it's possible you have an old Clojure CLI, but you can check that with clj -version . Latest is 1.11.1.1413.

Looks like I have latest. I've created an empty project, just with the dev alias, and inspected the basis file. Merged dev alias looks like this:

:dev {:extra-paths ["dev-resources" "dev-data"] :jvm-opts ["-XX:-OmitStackTraceInFastThrow" "-Duser.timezone=UTC" "-Dfile.encoding=UTF-8" "-Djava.awt.headless=true"]}

Just to confirm one thing, when I add my library directly to the dev alias dependencies list in my project, it works as expected

@U02EMBDU2JU You said: > I have a project deps.edn with :dev alias, which sets up jvm-opts and extra-paths. Then in my ~/.clojure/deps.edn my :dev alias adds a dependency The same alias in both deps.edn files? The project :dev will hide the user :dev -- they don't combine. Am I misunderstanding you?

One reason I use qualified keywords for the user deps.edn is to avoid clashes with project xep.edn aliases that people create https://practical.li/clojure/clojure-cli/practicalli-config/

@U04V70XH6 oh, so I misread the merge section on this page? https://clojure.org/reference/deps_and_cli#_merging_deps_edn

I suppose it makes sense why aliases wouldn't be merged just from the sensible approach perspective - it would be a nightmare to debug, but the way that section is written it implies that everything is merged (except the keys mentioned). I guess my global deps.edn can just include with top level :deps list which does get merged from what I'm seeing but it was somewhat surprising that aliases do not work this way.

The contents of the set of aliases you provide are merged -- but that means that you merge system + user + project deps.edn as hash maps first, then get the (juxt :all :the :aliases :provided) -- in your case (juxt :dev :repl) -- and then that vector of data is merged using the deps-specific merge rules.

It would be bad if a project couldn't rely on its own aliases but instead they were modified (merged) with arbitrary data coming from a different user deps.edn on everyone's machine -- whereas if a developer says -M:my/dev:dev:nrepl and :my/dev is not in the project deps.edn, that's specifically up to the developer who knows what they are adding to the project deps etc.

Right, I get why it's not working that way - it was just not apparent from the reference, when I read merge... I thought it would truly merge everything. All clear now!

I guess you could create a GH issue for clojure-site to suggest that wording be clarified...

Yep, it's on my list 👍

common

cool beans thanks

common to all platforms