@UQHJGGW86 How are you starting your REPL? Are you not using lein/`boot` or the new CLI with deps.edn?

Never mind, I just saw that you're calling Clojure from inside a Grails app running as a WAR on Tomcat. Certainly a very unusual execution environment.

It would be easier to help if you can reproduce a problem and error message with an unmodified Clojure 1.10.1

in REPL 1.10.1, I had tried the same and I did not get any error.

So you only encounter a problem when trying to load that code with your locally modified version of Clojure 1.10.1 ?

When I build my project, by calling compile.load using the project, I am getting the error

Out of curiosity, why are you using compile.load, instead of more common methods of loading or compiling Clojure code?

e.g. clojure.core/compile can also be used to AOT compile Clojure

and clojure.core/require or load-file can be used for loading it.

Maybe you are doing this because you want to load Clojure code and call it from a Java program, or some other JVM language?

Our project has been compiled in this mode since clojure 1.2

It was working fine till 1.8

We thought of migrating 1.10.1 (stable version).

started encountering this error.

Do you have a pointer to the compile.load method you are using, in the Clojure source code? I'm not familiar with it. Your best bet might be to ask in #clj-dev channel where folks knowing the APIs for invoking the Clojure compiler are more common, but if so, you should be prepared to provide as much detail as you can, e.g. code snippets showing exactly what calls were working in Clojure 1.8 and no longer working in 1.10

As you can see in the stack trace, 7648 | load in clojure.lang.Compiler is called

It is loading the lib.clj file and when lispReacer is processing the file, it is throwing the syntax error.

Is there a short Clojure expression you used that you can share, that leads to this stack trace?

I don't know what code you executed that caused this to happen -- I may have missed it earlier.

There are any of dozens of Java methods inside the Java implementation you might have tried calling directly, or something else. I looked back at earlier messages, and I do not think you have said yet how you are initiating this process.

I will be away from keyboard for next hour or so, but will look at any messages you have after returning. Again, if you can explain exactly what steps you follow that lead to your problem, with an unmodified Clojure 1.10.1, maybe someone in #clj-dev can help better than I can.

Sure Andy. Thanks for your support.

I am able to get the all the clj code in nlplabs.lib successfully in REPL with lispReader spitting the output

Here is the project environment:

we are building an aggregator which will access multiple job portals. we are using Grails framework to build the app and run under tomcat. We create a war file of the app and deploy under tomcat. The clojure files are also bundled in the WAR package. During the deployment, clj files are compiled and the app will be running under tomcat.

We are trying to migrate the app which has clojure component to be migrated to 1.10.1 from 1.8.0

Now, I was trying to deploy my app after copying necessary clojure 1.10.1 JAR files in the lib folder.

I am facing this problem.

Clojure 1.10.1 changed the RT initialization process - it's possible that this broke some assumption of how the Clojure runtime is being loaded

My impression from the stack above is that this code is not using any public documented API to initialize the runtime

so could easily be affected

if you could easily try with Clojure 1.10.0 (and it worked) then I would strongly guess this is the cause

but it could also be something to do with the addition of spec macro checking to the initialization path or something else. that GClojure class seems like a key piece of code to review

Alex, I just now brought up 1.10.1 in REPL mode and gave the first line (ns nlplabs.webfetch.lib (:refer-clojure) (:use nlplabs.lib)) and it went through fine

processed all the files in nlplabs.lib successfully

We are guessing that some part of your code is calling into some Java method in the implementation of Clojure that was never documented as "we promise not to change that", and it changed.

For example, this is the (very small) Java API that they promise to support: https://clojure.github.io/clojure/javadoc/clojure/java/api/Clojure.html

The lib.clj definition is as below(ns nlplabs.lib (:refer-clojure) (:use clojure.test) (:require (nlplabs.lib defs date file logging readermacro locals re signal)))

If you call some other Java methods, then from one version of Clojure to the next, they might change.

I don't suspect that ns form has any syntax errors, as I was guessing earlier, because I tried that on Clojure 1.10.1, too, and it gives no errors. The problem is in how you are invoking the Clojure compiler, is my current best guess.

I am copying the code snippet of calling clojure compiler.load public static void loadExtractorLib() throws Exception { synchronized (GClojure.class) { if (loadedExtractors) return; loadedExtractors = true; } load("nlplabs/webfetch/lib.clj"); } public static void load(String resource) throws Exception { InputStream stream = LFUtil.openResourceStream(resource); if (stream != null) { try { File f = new File(resource); Compiler.load(new InputStreamReader(stream, "UTF-8"), resource, f.getName()); f = null; } finally { try { stream.close(); } catch (IOException ignored) { } } } }

Your stack trace mentions a class com.nlplabs.util.GClojure. Is that code owned/maintained by your organization, or is it some third party library that you use?

Our code

I just copied the snippet from GClojure.java fiile which calls the compiler.load with nlplabs/webfetch/lib.clj

I see it, and anyone who can help you resolve the issue would likely want to see that. Sorry, but I don't know off top of my head the changes in this area that Alex Miller was referring to earlier.

Thanks Andy for your help & support.

I am wondering if I need to dig into Compiler.java code under Clojure.lang

As a guess, it might be that adding some call to some initialization method for the RT class somewhere in here might help, but any concrete suggestion from me like that would be guessing and poking at the machine, without full understanding, but there are folks in the #clj-dev channel who understand this better than I. They would want to see the code snippet above, and the stack trace you showed earlier, and that "it worked with Clojure 1.2 through 1.8, but something is different about Clojure 1.10 here"

FYI, I am using JDK 1.8

Do you know how to join the #clj-dev channel?

sorry, it is called #clojure-dev -- I keep making that mistake.

No andy.

I joined this forum after Alex pointing it to me.

Are you using Slack within a desktop browser? If so, you probably see a "Jump to..." box/link near the upper left of your window, just above "All Unreads"

yes

Click on that, type in clojure-dev, press enter

I did not see clojure-dev option there

When you click on the "Jump to", does a pop-up text box appear? If so, type the letters clojure-dev using your keyboard, then press enter

I searched and got clojure-dev

I am inside that

Join channel option is there.

And in the list of "Channels" on the left side of your window, there are separate lines for # clojure and # clojure-dev now?

Yes

Each channel is a separate 'discussion room' which can have different participants involved.

ok. got it.

#clojure-dev channel tends to have readers who know more about Clojure internals than average.

i.e. Developers of Clojure itself.

Sure. Thanks for the heads-up

I am also a beginner

I shall post this question on dev forum

Sure, you can let them know that, and they'll understand. The stack trace and Java code snippet you gave are key information for them, as is the "works with Clojure 1.2 through 1.8, but not 1.10"

Thanks Andy once again.

Perhaps they may have a suggestion of changing that Java code in some way.

I think the key here is that this code is just calling Compiler.load(), which is not a public entry point

the public way to do this would be to use the Clojure Java API and it's facilities to require clojure.core, then invoke the compile function

using the API will ensure that the RT (runtime) is properly loaded and initialized

Here is the import definition of Gclojure.java file import clojure.lang.AFn; import clojure.lang.APersistentMap; import clojure.lang.APersistentSet; import clojure.lang.APersistentVector; import clojure.lang.Compiler; import clojure.lang.Keyword; import clojure.lang.RT; import clojure.lang.Var; import com.nlplabs.lf.LFUtil;

yeah, none of that is public API

the public Java API is: https://clojure.github.io/clojure/javadoc/

Is anything changed from 1.8 to 1.10.1

All along the same code was working fine

that API has not changed, no. as I mentioned above, how RT is initialized has changed

ok. I shall have a look at the link that you have provided

what I'm saying is that your code uses internal APIs, which have changed in subtle ways. the public supported API has not changed and has been updated appropriately

since you are just loading a particular file, rather than compiling a namespace, you might need to go through a few more hoops to do the real equivalent of this

if you wanted to try just a small change, you could invoke RT.init(); before you call Compiler.load().

ok. I shall try that.

That call should exist on any version of Clojure but would just print an error to stderr for recent versions up through 1.10.0. As of 1.10.1, it's actually doing important parts of the initialization.

obviously, you can't initialize Clojure without the clojure.lang.RT class. This means that either the class is not on the classpath or that the classloader at this point is not able to access it. That's really a JVM/Java question for your application structure, not really a Clojure issue.

I would check that you have the clojure.lang.RT class on the classpath

and if so, then start investigating the thread context classloader chain at the point of the load

this is not right, but you get hte idea

I'm not sure if I find that more readable 🙂

my main point was really that you shouldn't have the macro emit a bunch of work in the expansion if you can do it in the macro at compile time

I'm not sure if I'm following your line of thought. What does it matter if I write this using for or map, it boils down to the same data at compile time?

I mean:

(defmacro foo [] `(def x '~(for [i [1 2 3]] (inc i))))

(defmacro bar [] `(def x '~(map inc [1 2 3])))

for vs map doesn't matter (I just prefer to view it as a data transformation pipeline)

yeah, I see what you're getting at, doing things at compile time is cheaper than doing something every time at runtime again and again, but afaik my macro doesn't do very much at runtime

(defmacro foo [] (let [data (map inc [1 2 3])] `[~@data]))

(macroexpand '(foo))   ;; [2 3 4]

in this particular case, maybe it doesn't matter, but I generally think minimizing the expansion and moving work to the macro is a good preference

prob matters even more in cljs

because it's less confusing than mixing syntax-quote and unquote multiple times you mean?

also that, although that's a separate point

if you separate out all the data transformation / construction at the top of the macro, you can more easily test it (and put it in a function, which often ends up turning into a non-macro variant that is separately useful)

I've seen that movie a few dozen times :)

true. the macroexpansion of my macro looks like this btw:

(do (def CANON_EQ java.util.regex.Pattern/CANON_EQ) (def CASE_INSENSITIVE java.util.regex.Pattern/CASE_INSENSITIVE))

yeah, that's what you want, not a bunch of for blather

this is what is generated by the for blather, I think that's where the confusion in this discussion comes from. the for is compile time.

if you go aot this, do you want it to happen once at aot compile or on every load?

I want to do as much as I can once at aot time

yes, me too. sooo?

so you want the for in the macro, not in the expansion

yes, that's what's already the case, I've tried to communicate that several times now 🙂

maybe we aree just in violent agreement then :)

lol 🙂

I thought above you were disagreeing but maybe I misinterpreted

no, I agree with what you say, but it seemed you were thinking the macro generated a runtime for

nope, I think we were just talking past each other

I was talking about the original code

yes, the original code also had that

https://clojurians.slack.com/archives/C03S1KBA2/p1573674970409600

ah, right - again, I have it stuck in my head that the for was in the expansion and not in an unquote

sorry, just my brain not working

so yes, the other point is that minimizing the nested syntax quote / unquote makes it more readable

no problem 🙂 nested syntax-quote / unquote can be confusing, that's why it's probably easier to read with minimal back and forth between the two

nice conclusion

sorry for the confusion!

np