Clojurians Log v2

I do have

:source-paths ["src/clojure"]
  :java-source-paths ["src/java"]

So far the only thing that has worked has been to set up a :precomp profile like

:profiles {:precomp {:java-source-paths ["src/java"]}

lein with-profile precomp javac

the fundamental issue seems to be that if I dont do this, it tries to process clojure files first…but the clojure code depends on the java code so it fails. I’ve done similar things in the past and it just worked, so I am scratching my head as to what is wrong in this project.

Have you altered prep-tasks anywhere?

Not to my knowledge

I’d double check that.

Theres nothing in the project file…but what I dont know is if some plugin may alter it

Use whatever profiles you are using and lein pprint it

ok

Hmm

When I pprint it, it looks like its the default

:prep-tasks ["javac" "compile"],

$ lein --version
Leiningen 2.10.0 on Java 21.0.1 Java HotSpot(TM) 64-Bit Server VM

That seems fine to me then. javac is first.

Yeah…its confounding

You can do DEBUG=true ahead of your lein command to get more details of what it is doing.

ok., let me try that

ok, that helped move the needle a bit. I immediately started with “DEBUG=true lein uberjar” to narrow it down and it actually did the right thing, giving me a hint that its not so much the overall config, but my flow

I have a Makefile and in there are other tasks like lein eastwood and lein cloverage that run before the build

Yes. Progress. So then figure out what your flow task is running.

so my problem seems to be i need to hook those the right way

That’s a possibility.

I dont know if this is meaningful to you, but when I enable debugging for one of the broken tasks

$ DEBUG=true lein cloverage
Leiningen's classpath: /usr/local/Cellar/leiningen/2.10.0/libexec/leiningen-2.10.0-standalone.jar
Applying task cloverage to []
Applying task javac to nil
Running javac with [@/var/folders/bl/s_t9lqxx2glgf9dv7h7z54g40000gn/T/.leiningen-cmdline84724007947143382.tmp]
Warning: environ value /Users/ghaskins/.jenv/versions/system for key :java-home has been overwritten with /Library/Java/JavaVirtualMachines/jdk-21.jdk/Contents/Home
Warning: protocol #'manetu.policyengine.api/policyengine is overwriting function test
WARNING: test already refers to: #'clojure.core/test in namespace: manetu.policyengine.api, being replaced by: #'manetu.policyengine.api/test
WARNING: test already refers to: #'clojure.core/test in namespace: manetu.policyengine.core, being replaced by: #'manetu.policyengine.core/test
Reflection warning, protojure/grpc/codec/lpm.clj:184:5 - reference to field close can't be resolved.
Exception in thread "main" Syntax error macroexpanding at (manetu/lambda_service/subservice/executor/wasm/guest_context.clj:1:1).

Of course, if I run the precomp profile or build the uberjar first, javac runs in the right timing

$ DEBUG=true lein uberjar
Leiningen's classpath: /usr/local/Cellar/leiningen/2.10.0/libexec/leiningen-2.10.0-standalone.jar
Applying task uberjar to []
Applying task javac to nil
Running javac with [@/var/folders/bl/s_t9lqxx2glgf9dv7h7z54g40000gn/T/.leiningen-cmdline15031772485933302220.tmp]
Compiling 35 source files to /Users/ghaskins/sandbox/git/mcp-lambda-service/target/uberjar/classes
Note: Annotation processing is enabled because one or more processors were found
  on the class path. A future release of javac may disable annotation processing
  unless at least one processor is specified by name (-processor), or a search
  path is specified (--processor-path, --processor-module-path), or annotation
  processing is enabled explicitly (-proc:only, -proc:full).
  Use -Xlint:-options to suppress this message.
  Use -proc:none to disable annotation processing.
Note: /Users/ghaskins/sandbox/git/mcp-lambda-service/src/java/io/github/kawamuray/wasmtime/Func.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

Im basically missing this in the broken cases

Compiling 35 source files to /Users/ghaskins/sandbox/git/mcp-lambda-service/target/uberjar/classes

When you run cloverage - you aren’t adding your profile precomp

So it isn’t getting the :java-source-paths.

well, to be clear, I set that in the top-level config, too. I only added the precomp profile as a hack

Ok, at the top-level I’d expect it to work there

yeah…and ive done this a bunch of other times in other projects, so its super weird

I cant figure out what I am doing differently

cloverage fails and eastwood?

are these custom :aliases?

no, i only have one alias for clj-kondo and I am not using it in this flow

its only lein-plugins

the operative part is

:source-paths ["src/clojure"]
  :java-source-paths ["src/java"]

  :repl-options {:init-ns user}

  :eastwood {:add-linters [:unused-namespaces]
             :exclude-linters [:unused-meta-on-macro :local-shadows-var :unused-ret-vals]}

  :profiles {:precomp {:java-source-paths ["src/java"]}
             :dev     {:dependencies   [[clj-http "3.12.3" :exclusions [commons-io]]
                                        [org.clojure/tools.namespace "1.4.5"]
                                        [criterium "0.4.6"]
                                        [eftest "0.6.0"]
                                        [clj-kondo "2023.12.15"]
                                        [mockery "0.1.4"]
                                        [integrant/repl "0.3.3"]]
                       :resource-paths ["jna/target" "jna/target/release" "test/lambdas"]
                       :aliases        {"clj-kondo" ["run" "-m" "clj-kondo.main"]}}
             :uberjar {:aot                :all
                       :omit-source        true
                       :target-path        "target/uberjar"
                       :uberjar-name       "app.jar"
                       :uberjar-exclusions [#".*manetu.*\.clj[xcs]*"]
                       :jvm-opts           ["-Dclojure.compiler.direct-linking=true"]}}

where

:profiles {:precomp {:java-source-paths ["src/java"]}

Did you try lein clean before DEBUG=true lein cloverage

I had hammered it with rm -rf target, but let me do it the right way and see what happens

ah ok, probably that covered it - just making sure you didn’t somehow have old classfiles somewhere causing issue

and as expected, same behavior

its like it ignores the java-source-paths when the source-paths [“src/clojure”] is present

or, maybe it schedules java-source-paths as secondary, but we never live long enough to get there

The default prep-tasks should run javac first and javac explicitly compiles the :java-source-paths that it finds.

So it is weird. It’s as if they are missing during certain tasks to me.

Makes sense…and thats what I thought…so confounding

odd, you have issues for both eastwood and cloverage

i didnt exhaustively look..let me try some of the other pre-build functions

But did you have issues with anything else before you tried using the profile precomp?

You said lein uberjar works with no special :aliases right? So it’d just be based on the top-level project.clj merged with the :uberjar profile.

right…i didnt try any other flows…i have a top level makefile that does this (currently)

all: scan test bin security-scan

precomp:
	lein with-profile precomp javac

scan: precomp
	lein cljfmt check
	lein bikeshed -l false -n false
	#lein kibit
	lein eastwood

and so it was puking somewhere in the scan

at the time, I didnt realize uberjar would actually work (should have thought to try that)

(the precomp was added as a workaround for this, that wasnt there originally)

So everything there is a plugin basically

it works with the precomp

correct

thats why I am wondering if its somehow related to the way the plugins interact

Yeah, with the precomp done first - you leave the classfiles in the :target-path and the other tasks will get them on their classpath

yep, that much I understood

the part I dont get is where lein is making the decision to compile the java-source-paths in some cases but not others

I don’t understand how every plugin would be susceptible to the same problem though. It isn’t something I’ve heard of I think.

Heh…i love when I find weird problems

Well, I feel better knowing that you dont know either

If it was one plugin, I’d wonder if it perhaps altered prep-tasks or didn’t merge with your project top-level - but even then that’d be pretty odd and probably always broken.

i was banging my head on the wall

When I have these issues, I deal with it a bit of the tedious way.

in any case, its largely academic at this point: my precomp hack works

i was hoping to just learn a more lein-idiomatic way, but maybe its elusive

I’ll basically git clone the lein version you want to deal with. Then I’ll launch a repl directly in leiningen and try to recreate the project setup via something like leiningen.core.project/read and then make sure I get the same profile configuration state and then look for things like :java-source-paths there etc. I probably could write a tiny post on this at some point, but it’s not that elegant.

Yeah, in worst-case, you could just script that you manually ensure the javac is ran correctly before each plugin task - like you have.

so, it looks like cljfmt is fine, probably because it doesnt need the classfiles present…bikeshed, eastwood, and cloverage all seem to suffer the same fate

in any case, ty for talking it through with me. I at least discovered that the uberjar target works, which at least instills some confidence in me that the config isnt fundamentally broken

something else is going on, but my hack in the makefile is fine for the time being

Ok, sounds good. I’m making a TODO to see if I can recreate your issue with a tiny project with cloverage as an example - to see if I can understand better what happened. I probably wont’ get to that for a few days though. I’ll let you know though if I find out anything interesting there.

I appreciate your time, ty