Fork me on GitHub
#clojure
<
2020-12-16
>
msolli06:12:39

I have a weird error in production sometimes. It’s an uberjar by Leiningen, and it’s a mix of .clj and .cljc files. Sometimes (but only sometimes!) after a deploy, the new instance will start reporting clojure.lang.Compiler$CompilerExceptions for a few forms in .cljc files. These are files that have been built and tested locally and on CI without error, and runs fine on the other instance. In other words, a moon-phase bug. 🙂 Restarting the app server (with the same code) always solves the problem. I don’t know how to even start to form a theory of what’s going on here. Here’s a sample excerpt of a stack trace (with names pseudonymized):

...
                                                                    clojure.core/require                       core.clj: 6007 (repeats 2 times)
                                                                      clojure.core/apply                       core.clj:  667
                                                                                     ...
                                                                  clojure.core/load-libs                       core.clj: 5969
                                                                  clojure.core/load-libs                       core.clj: 5985
                                                                      clojure.core/apply                       core.clj:  667
                                                                                     ...
                                                                   clojure.core/load-lib                       core.clj: 5928
                                                                   clojure.core/load-lib                       core.clj: 5947
                                                                clojure.core/load-lib/fn                       core.clj: 5948
                                                                   clojure.core/load-one                       core.clj: 5908
                                                                                     ...
                                                                       clojure.core/load                       core.clj: 6109
                                                                       clojure.core/load                       core.clj: 6125
                                                                    clojure.core/load/fn                       core.clj: 6126
                                                                                     ...
                                                                        foo.bar/eval3949                       bar.cljc:    1
                                                        foo.bar/eval3949/loading--auto--                       bar.cljc:    1
                                                                                     ...
                                                                    clojure.core/require                       core.clj: 6007 (repeats 2 times)
                                                                      clojure.core/apply                       core.clj:  667
                                                                                     ...
                                                                  clojure.core/load-libs                       core.clj: 5969
                                                                  clojure.core/load-libs                       core.clj: 5985
                                                                      clojure.core/apply                       core.clj:  667
                    java.lang.Exception: namespace 'foo.xyzzy' not found
2020-12-15T17:38:09.900000+00:00 i-002af1a1b62baa6d3 clojure.lang.Compiler$CompilerException: Syntax error compiling at (foo/bar.cljc:1:1).
    data: {:clojure.error/phase :compile-syntax-check,
           :clojure.error/line 1,
           :clojure.error/column 1,
           :clojure.error/source
           "foo/bar.cljc"}

Renato Alencar11:12:13

It looks like something that it's not being built or no being found. I would inspected the jar in order to see if the class generated for the namespace is there, it should be something like bar__init.class, if this is not being found it tries to compile it's local clj files on the class path.

noisesmith07:12:41

the first thing I'd look for is def calls with code that might throw exceptions

noisesmith07:12:03

in the namespace that isn't found

noisesmith07:12:20

also, be sure you don't have any code that uses a namespace without requiring it

msolli08:12:45

Thanks, @noisesmith! Yeah, I’ve checked those things. No side-effecting defs, and all namespaces are required before use.

msolli08:12:11

I think it might be a concurrency issue. I noticed it happens when two web requests are processing concurrently on the same instance, just after restart. There is a dynamic require for a view (the .cljc file) in there. I don’t know much about the inner workings of require, but might it be possible that some Vars that are being (re-)defined in one thread are “invisible” to other threads? The first CompilerException I get after a new instance launches is always with an error like this in the dynamic require call: “java.lang.RuntimeException: No such var: xyzzy/some-var”

andy.fingerhut13:12:43

The discussion at that link contains some approaches to mitigate this property of require

msolli14:12:07

Ah, this is exactly what I needed. Much obliged!

kwladyka09:12:23

tree src/api/logs
src/api/logs
├── CustomLayout.java
├── CustomMessage.java
└── core.clj
Any simple way to use .java with deps.edn and REPL and make .java files automatically reload / compile ? How do you deal with java files when coding in Clojure? Do you really make separate project for 2 files? I would like to keep Clojure experience in REPL even when add this 2 java files.

p-himik09:12:26

Although it's designed to work with Leiningen and Boot, you can still use it in any Clojure project - you just have to write some code for it. In my case, most of that code was just copied from virgil itself and then changed a bit.

kwladyka09:12:03

But it is strange there is no tool for that already. Maybe separate project is the right call then hmm.

kwladyka09:12:38

What about all dependencies and versions then between 2 files java project and whole deps.edn project?

p-himik09:12:22

It will become conceptually simpler but harder to implement, support, and use. And the hot reloading capability will be gone.

kwladyka09:12:15

on the other hand I don’t want to maintain my own personal virgil 🙂 It looks I will lose whatever path I will choose 🙂

p-himik09:12:15

It's not "your personal virgil" - I just copied 30 lines from it. And even those were because I'm using a newer JVM that's somehow incompatible with the way virgil does things (I had to change literally one line) and I wanted to provide an explicit set of Java files (virgil searches for Java files itself).

p-himik09:12:40

BTW "I don’t want to maintain my own personal virgil" is a great answer for why "there is no tool for that already" ;)

delaguardo10:12:48

there is another alternative that is working smoothly with clojure cli https://github.com/EwenG/badigeon

👍 3
delaguardo10:12:32

(ns ops.java
  (:refer-clojure :exclude [compile])
  (:require [badigeon.javac :as javac]
            [badigeon.classpath :as classpath]))

(defn compile [{:keys [source-path output-path aliases]
                :or {aliases []}}]
  (javac/javac source-path
               {:compile-path output-path
                :javac-options ["-cp" (classpath/make-classpath {:aliases aliases})
                                "-target" "8" "-source" "8"
                                "-Xlint:-options"]}))
and then you can run it using -X option
:compile-java {:extra-deps {io.xapix/ops {:local/root "../ops/"}}
                          :exec-fn ops.java/compile
                          :exec-args {:source-path "src/java"
                                      :output-path "target/classes"}}
extra-deps here point to location of above namespace

kwladyka12:12:49

thank you, I will take a look on this

kwladyka12:12:19

package api.logs;

import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.core.config.plugins.Plugin;
import org.apache.logging.log4j.core.config.plugins.PluginAttribute;
import org.apache.logging.log4j.core.config.plugins.PluginFactory;
import org.apache.logging.log4j.core.layout.AbstractStringLayout;

import java.nio.charset.Charset;

@Plugin(name = "CustomLayout", category = "Core", elementType = "layout", printObject = true)
public class CustomLayout extends AbstractStringLayout {

    private static final String DEFAULT_EOL = "\r\n";

    protected CustomLayout(Charset charset) {
        super(charset);
    }

    @PluginFactory
    public static CustomLayout createLayout(@PluginAttribute(value = "charset", defaultString = "UTF-8") Charset charset) {
        return new CustomLayout(charset);
    }

    @Override
    public String toSerializable(LogEvent logEvent) {
        return logEvent.getMessage().getFormattedMessage() + DEFAULT_EOL;
    }
}
` Unless you will say code above is easy to modify into Clojure :)

delaguardo13:12:41

imho, doesn’t worth a try) small helper like that is better to delegate to java

p-himik13:12:24

If that was a simple class, I would've definitely used proxy. But since it uses Plugin, it can end up being arbitrarily complex. But I have no idea how Java annotations work under the hood - maybe it's something that you can do with proxy or something else as well.

kwladyka13:12:05

thank you for feedback, I was looking confirmation it is bad idea to do java->clojure for this 🙂

kwladyka14:12:48

Do you know master of Java interops in Clojurians? 😉

kwladyka14:12:46

ech there are still issues around this, logging in structured JSON format in Clojure is a pain 🙂

Dennis Tel10:12:02

Hi, I’m trying to call Clojure from Java/Kotlin and a function requires a map. Is there a list/place where the interfaces for these objects are defined?

p-himik10:12:30

FWIW I just use Clojure's source code as such a place.

Dennis Tel10:12:15

Do you by any chance know if you can directly use the Class clojure itself uses? or is it common to define your own implementation to the interface?

p-himik10:12:27

I have seen both. Depends on what you need. If you just need a Clojure map then reusing the existing classes makes sense. If you need custom behavior then a custom implementation makes sense.

valerauko10:12:28

You can, probably, but I think you won't need to use PersistentHashMap just use whatever Map you use in Java and Clojure will work its magic when you call the function

👍 3
Dennis Tel10:12:48

sounds good, thanks 🙂 !

Renato Alencar11:12:30

Clojure implements Java native interfaces, such as Map, Iterable and Callable. So most of the time if you want use any interfaces of the java.lang package, Clojure implements it. Clojure runtime source code it's extremely simple to read, so you just directly read the implementation to understand how it works. https://github.com/clojure/clojure/tree/master/src/jvm/clojure/lang Also, Clojure has some shortcuts for the Java Reflection API, you could just call ancertors in a class in order to see its parent classes and implemented interfaces. For instance (ancestors (class {}))gives me this:

#{clojure.lang.IPersistentCollection clojure.lang.MapEquivalence clojure.lang.IMeta clojure.lang.ILookup clojure.lang.IEditableCollection java.util.Map clojure.lang.IMapIterable java.lang.Iterable java.util.concurrent.Callable java.io.Serializable clojure.lang.IFn java.lang.Object clojure.lang.IKVReduce clojure.lang.Seqable java.lang.Runnable clojure.lang.IHashEq clojure.lang.Associative clojure.lang.Counted clojure.lang.AFn clojure.lang.APersistentMap clojure.lang.IObj clojure.lang.IPersistentMap}

👍 6
Dennis Tel11:12:37

ah thats a nice trick to know!

Alex Miller (Clojure team)14:12:50

You can invoke the Clojure.read() function to read a literal map or Clojure.var() to look up Clojure vars and invoke them

Alex Miller (Clojure team)19:12:54

has anyone tried Clojure 1.10.2-rc1? Any feedback good/bad?

lukasz19:12:41

All tests pass in our biggest Clojure service, no issues whatsoever

lukasz19:12:54

Will probably ship it to production next week

hiredman19:12:47

not to steal sean's thunder but we've got it in production on one service that got deployed yesterday, it should go out to the other services whenever they are next deployed

dharrigan19:12:27

been using it in our test environment, nothing unusual to report.

dharrigan19:12:59

yes, it works 🙂

flowthing20:12:11

Been using it in one of the apps I work on (not yet in prod, though) — everything’s been working as expected so far.

seancorfield20:12:20

What hiredman said 🙂 We're on Alpha 4 for all our other production services so the differences are minimal for us.

seancorfield20:12:36

(also as a comparative data point: we have 1.10.1.754 CLI on our production servers and 1.10.1.763 on dev and QA so that will also go to production with our next full deployment)