I'm trying to use Clojure to build a plugin (distributed as a jar file) for an application. I'm having issues when this application loads the jar. It appears to do with this application messing with the Java classloader causing clojure.lang.RT.load to now be able to find clojure/core__init.class (which is in the jar).

Caused by: .FileNotFoundException: Could not locate clojure/core__init.class, clojure/core.clj or clojure/core.cljc on classpath.
        at clojure.lang.RT.load(RT.java:462)
        at clojure.lang.RT.load(RT.java:424)
        at clojure.lang.RT.<clinit>(RT.java:338)

@ghopper You said it's an AOT'd JAR? And you're loading it into... another Clojure app?

hi, my understanding is that when code is evaluated through a normal nREPL, any classes that are dynamically defined during evaluation will be stored in memory only, in a DynamicClassLoader. Is there a simple way to get those classes to actually be written to files?

The behavior in the REPL is how Clojure code is treated when a library is loaded too: source in the library is compiled when a namespace is loaded, one form at a time -- just like the REPL -- to classes in memory.

Only AOT compilation -- a separate, specific step, usually associated with creating an "uberjar" of a complete application -- are the compiled-in-memory classes also written to disk.

Nearly all Clojure libraries are distributed as JARs of source code, so the normal mode of operation is compile-on-demand, to memory.

Does that answer your question @brandon268?

Hmm, to be more specific, my goal is to open a REPL, type in some forms, and then be able to get the class files of any dynamically-generated classes. Something along the lines of REPL middleware.

The challenge I'm running in to is understanding the process for evaluating code in a REPL, and what customization I can apply to the class loader in that context, if any

What problem are you actually trying to solve?

Get the class files and send them to other machines

Why?

If you type the forms into a file, you can AOT it to get class files but that's not normally how Clojure programs are run or distributed.

Ya, not sure you can call the aot compiler on just a form. I remember it taking resources.

You might be able to have a middleware that writes the sent form to a file, compile it, and then sends you back the compiled .class

@brandon268 is this some apache spark - type usecase?

yes

there's already infrastructure in place to load dynamic class files on remote machines, the main challenge is coercing a Clojure REPL to produce actual class files

so far, I haven't had any luck with re-setting the class loader before evaluating a form

what infrastructure is already in place?

or rather, what was the usecase before

if a class files is placed in a certain directory, it becomes accessible to remote machines

I thought I saw a library announcement just recently that lets you use regular Clojure source with Spark?

this is pure evil, so I don't actually suggest it

but you can always dig into the internals of DynamicClassLoader

peek at whatever private fields the definitions are in

public class DynamicClassLoader extends URLClassLoader {
    HashMap<Integer, Object[]> constantVals = new HashMap();
    static ConcurrentHashMap<String, Reference<Class>> classCache = new ConcurrentHashMap();
    static final URL[] EMPTY_URLS = new URL[0];
    static final ReferenceQueue rq = new ReferenceQueue();

Oh, I was thinking of this (for Apache Beam): https://github.com/atdixon/thurber

I've created a "wrapper" class loader that takes a DynamicClassLoader and takes any new class, also dumps it to a file. But Clojure doesn't let me override the class loader, or I'm doing it wrong

I'm probably just doing it wrong 😛

You tried calling setContextClassLoader ?

On the thread?

(binding [*use-context-classloader* true]
  (let [original-cl (.getContextClassLoader (Thread/currentThread))]
    (try
      (.setContextClassLoader
        (Thread/currentThread)
        (FileOutputClassLoader.
          original-cl
          "./repl-classes"))
      ;; arbitrary code here
      (finally
        (.setContextClassLoader (Thread/currentThread) original-cl)))))

That didn't work?

You might alao need to bind clojure.lang.Compiler/LOADER

both? :thinking_face:

Normally I think the DynamicClassLoader is bound to that variable. And that's the one used to compile repl forms. The context class loader is used before, to load a namespace and compile Clojure core I think

I think only clojure.lang.Compiler/LOADER needs to be set to a DynamicClassLoader

For repl forms. But I would try all combination until it works :p

Hey, is there a channel where people talk about using redis from clojure?

Had a quick search, couldn't find anything but I might have missed it.

How do I track down where this exception is from?

Exception in thread "async-thread-macro-2" java.lang.NullPointerException
        at clojure.core.async$thread_call$fn__14495.invoke(async.clj:484)
        at clojure.lang.AFn.run(AFn.java:22)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)

@grounded_sage in some cases the JVM elides stack traces on NullPointers

There is a JVM flag to tell it not to do that

-XX:-OmitStackTraceInFastThrow

I could use some help with reflection. I have a Java .jar that I'm wanting to digest at runtime, and so far I've been able to find it and load the class. However, when I attempt to invoke a method, I'm running into issues.

When I call (.getMethods my-class) I'm getting back reasonable info: #object["[Ljava.lang.reflect.Method;" 0x6e321cee "[Ljava.lang.reflect.Method;@6e321cee"]

And I can getName from the method `

(.getName (first (.getMethods my-class)))

which gives me what I'm expecting

why are you reflecting? is the exact set of methods available unknowable at compile time?

But if I try to invoke the method (.invoke my-method)

I get No matching field found: invoke for class java.lang.reflect.Method

The methods are not knowable at compile time

you need to pass the object as the first arg

so the shortest possible call is (.invoke my-method my-instance) or (.invoke my-method my-class)

> If the underlying method is static, then the specified `obj` argument is ignored. It may be null. TIL - there still needs to be a placeholder there

also any args need to be packed into an object-array

So the method I'm trying to invoke is just a public static void main(String[] args)

then you can pass nil for the object, then an object-array where the first item is a string-array (optionally empty)

So I would that look like (.invoke my-method my-class nil)?

it would be (.invoke my-method (or my-class nil) (object-array (into-array String [])))

it might work with an empty last arg, not sure

but strictly speaking, main there has exactly one arg which needs to be an array of string

and the .invoke method takes all args in an object array

(.invoke my-method (object-array (into-array String []))) returned "No matching method invoke found taking 1 args for class java.lang.reflect.Method"

right, you missed the mandatory first arg

which can be nil or your class in this case

(.invoke my-method my-class (object-array (into-array String []))) says "wrong number of arguments"

same with (.invoke my-method nil (object-array (into-array String [])))

More specifically `Execution error (IllegalArgumentException) at jdk.internal.reflect.NativeMethodAccessorImpl/invoke0 (NativeMethodAccessorImpl.java:-2). wrong number of arguments`

wrong number off arguments to who? invoke, or the underlying method?

that is the underlying method

oh - I think you want (object-array [(into-array String [])])

that's my bad

hah! That was it

👏

so the semantics here are that you want one empty array of String

where before you had zero objects (coerced from an empty array of String)

gotta stack your empties correctly :D

heh, well thank you sir.

ok, I think I understand better now. In the method definition, it's looking for (Object obj, Object... args), so the Object... is the outer (object-array), and the first item in it was String[] created by calling (into-array String []).

right, exactly

it's easier to get confused by because main already acts like varargs by taking arguments in an array, so we end up needing the array of arrays

and object-array takes a collection to coerce, so it's easy to leave off a layer and get the wrong coercion for nested arrays