Fork me on GitHub
#clojure
<
2021-11-13
>
João Galrito03:11:38

i'm kind of new to Maven, and I'm trying to setup my current Java project to also compile some .clj code on src/clojure

seancorfield03:11:14

Do you have some specific :gen-class stuff?

João Galrito03:11:14

do I have to add gen-class? I only want to have the namespace on the classpath, as it is only called by other clojure code (that is dynamically loaded)

João Galrito03:11:29

I don't need to call it from Java

seancorfield03:11:55

If the src folder is on the classpath, Clojure code can require it.

seancorfield03:11:01

No compilation needed.

seancorfield03:11:58

The Clojure code you are calling from Java is already on your classpath, right?

seancorfield03:11:41

You only need :gen-class in Clojure to create Java classes that you're going to call from Java. As long as you're using the Clojure Java API, you don't need :gen-class and you can just require code.

João Galrito03:11:42

ok so I just need to add it as a resources on pom.xml?

João Galrito03:11:52

the clojure source folder

seancorfield03:11:14

If you know the root folder in advance, it can go in pom.xml to be on the classpath when you build/run.

seancorfield03:11:39

I thought you said you didn't know that folder until runtime tho?

João Galrito03:11:11

I will have 1 clj file bundled in the .jar

João Galrito03:11:23

that the dynamically loaded ones will call

seancorfield03:11:02

OK, so you're constructing a classloader with that folder on it?

João Galrito03:11:16

so to add the folder I do it as a resource?

<resources>
        <resource>
            <directory>src/main/clojure</directory>
        </resource>
    </resources>

seancorfield03:11:17

I don't know Maven well enough but I think that will only affect building not running?

seancorfield03:11:06

At work, we construct a classloader in our legacy app and add all of the Clojure code/libs to it. Not sure if it will help you -- since it is not Java -- but it might give you some guidance: https://github.com/framework-one/cfmljure/blob/master/framework/cfmljure.cfc#L138-L150

João Galrito03:11:03

that part I got working

seancorfield03:11:03

OK, so I'm not sure what else you need?

seancorfield03:11:36

That classloader is going to contain the folder(s) that Clojure will load .clj files from.

João Galrito03:11:14

I think it's finding the file but I'm getting an error

João Galrito03:11:15

Unable to resolve symbol: .iterator in this context

João Galrito03:11:43

the dynamically loaded .clj uses a macro defined in the packaged .clj that uses .iterator

seancorfield03:11:59

You'll need to share more detail about it.

seancorfield03:11:17

.iterator as a symbol doesn't sound right -- that's meant to be Java interop, calling the iterator method, yes?

seancorfield03:11:32

So maybe your problem is that your Clojure code isn't actually valid in the first place?

João Galrito03:11:40

the code dynamically loaded at runtime (test_plugin.clj)

(ns test-plugin
  (:use [plugin-tools]))

(feed SomeStream
      (map dostuff))

João Galrito03:11:28

the code packaged (plugin_tools.clj)

(ns plugin-tools
  (:import (some.java.namespace DataSource)
           (java.util.stream Stream)))

(defn seq1 [i]
      (lazy-seq
        (when (.hasNext i)
              (cons (.next i) (seq1 i)))))

(defmacro feed [input & body]
          `(with-open [stream# (-> ~input
                                   DataSource/getInstance
                                   .create)]
                      (->> stream#)
                      .iterator
                      seq1
                      ~@body
                      dorun))

João Galrito03:11:16

when the plugin_tools were on the same folder as test_plugin it was working ok

João Galrito03:11:00

ok, there's actually a typo

João Galrito03:11:25

ok, everything working now 🙂

João Galrito03:11:32

thanks and sorry

João Galrito03:11:28

also, thank you for your contribution to this community!

João Galrito03:11:53

using some of your stuff at work

seancorfield03:11:06

Glad you got it working.

seancorfield03:11:45

One thing I will caution about: don't use single segment namespaces. Namespaces compile to Java classes and Java has some weird restrictions on class names without packages.

seancorfield03:11:19

Better to use qualified namespaces (and therefore nested folders and files).

seancorfield03:11:39

One of the really nice things about loading Clojure into an existing/legacy apps is that you can specify a JVM option at startup and the Clojure runtime will spawn a Socket REPL, so you can connect your editor into your running legacy app and inspect it (and modify the Clojure code and maybe the global state)... magic!

João Galrito18:11:09

hm, how does that work? what JVM option?

seancorfield18:11:15

You can specify that JVM option for any JVM process that causes Clojure to be loaded and when Clojure loads, it will start a Socket REPL.

João Galrito18:11:33

ah cool, I'll try that out

João Galrito18:11:06

right now I'm fighting a weird bug, when the code reloads after I make a change, it complains about not finding a random symbol that is actually part of a string inside the code

João Galrito18:11:37

(ns sample-feed
  (:use [tcp-feed-tools])
  (:import (my.namespace RedisPro)
           (my.namespace MergedV4FeedObject)))

(def registrations #{"Bar" "Baz"})

(def in-reg-list? #(registrations (.getReg ^MergedV4FeedObject %)))

(feed MergedV4FeedObject
      (filter in-reg-list?)
      (map #(.toString %))
      (RedisPro/send "feed-merged"))
The only thing I did was add "Foo" at the beginning of the registrations set, and it complains with Unable to resolve symbol: ged in this context

João Galrito18:11:47

which coincidentally are the last 3 characters of the string at the end

João Galrito18:11:05

so maybe it's the macro's fault, but it's weird because the macro itself isn't being reloaded

João Galrito18:11:26

and it works fine on the first load

João Galrito22:11:05

hmm, it seems I can't get the socket REPL to work, not sure when it is supposed to boot up (when I load clojure code for the first time?)

seancorfield22:11:27

How did you test it?

seancorfield22:11:23

If you've provided that JVM property the REPL should start when the Clojure runtime is initialized.

Jacob Rosenzweig03:11:05

Is there a benefit to using the component library if none of my components are stateful (yet)?

seancorfield03:11:47

If your components have any dependencies on each other, it's worth doing. And, yes, definitely worth doing for DB stuff.

🎯 1
seancorfield03:11:05

next.jdbc has built in support for Component for pooled datasources.

seancorfield03:11:18

It supports both HikariCP and c3p0 out of the box.

Jacob Rosenzweig04:11:17

Is there an example for postgres? I don't think it has a clazz with a setJdbcUrl()

seancorfield04:11:03

I don't understand that question. There's nothing DB-specific about the component function. You provide a db-spec hash map, just like you would to get-datasource (except for the :user/`:username` weirdness with HikariCP).

seancorfield04:11:03

(just use component instead of ->pool to get a Component).

seancorfield04:11:57

There's an example of using component further down in that section with HikariCP.

Jacob Rosenzweig04:11:03

(connection/->pool com.zaxxer.hikari.HikariConfig ... I have to specify the driver class, no?

seancorfield05:11:12

:dbtype "postgresql" :dbname "..." :username "..." :password "..." -- a db-spec.

seancorfield05:11:17

No driver class needed.

seancorfield05:11:27

You only ever need to specify the driver class if next.jdbc doesn't know about your database type.

vemv14:11:37

> Is there a benefit to using the `component` library if none of my components are stateful (yet)? Predictably you'll have a number of IO-heavy components: db, cache, email, http... one of the main selling points is being able to mock out those (whereas with-redefs is far from being a clean alternative) Not sure if those count as 'stateful', guess it depends on who you ask :)

vemv14:11:43

Also, protocol extension via metadata deserves a honorable mention, this way you can code impls with defn instead of defrecord methods. Less nesting, more specing

Jacob Rosenzweig20:11:43

hikari isn't a database driver, it's a jdbc connection pool. I get it now.

Jacob Rosenzweig03:11:28

Well, actually, I guess the DB connection has to be stateful as it needs to persist (they're expensive).

anisoptera20:11:49

can confirm, i thought i was making my life easier with :dynamic vars that held a bunch of connection context. it's now impossible for me to handle a dropped backend connection without restarting my entire app and fixing that is probably an entire day of work. just make a context map and some helper methods to generate a new one, you'll like it a lot more in the end

anisoptera20:11:20

:dynamic is an even bigger trap than singletons, lol

João Galrito20:11:13

I'm loading some .clj code at runtime from a Java application

(ns sample-feed
  (:use [tcp-feed-tools])
  (:import (my.namespace RedisPro)
           (my.namespace MergedV4FeedObject)))

(def ids #{"Bar" "Baz"})

(def in-ids-list? #(ids (.getId ^MergedV4FeedObject %)))

(feed MergedV4FeedObject
      (filter in-ids-list?)
      (map #(.toString %))
      (RedisPro/send "feed-merged"))
the first time it loads it works fine, however, when I add "Foo" to the ids set, on reload I get an error Unable to resolve symbol: ged in this context Which, coincidentally, are the last 3 characters of the string in the last line. It only happens when I'm adding stuff, which leads me to believe when I call clojure.core/require from the Java side it's expecting the same file length as before

João Galrito20:11:23

but on the next try it is able to load it

João Galrito20:11:40

this happens even if the change to the file happens way before it tries to reload it

hiredman20:11:50

That is common when the file contents change while it is being read

hiredman21:11:40

The way loading works is a top level form is read, compiled, then run, then the next is read compiled and run, etc etc

hiredman21:11:57

So if you have a top level form which does some long running thing, load the file, it won't read the rest of the file until that long running whatever completes

hiredman21:11:47

So if you modify file while the long running thing is running, up until the long running thing executes, you are reading the first version, and then afterwards you are reading the second

hiredman21:11:26

Which can lead to all kinds of weird behaviors (reading starting the middle of a form, etc)

hiredman21:11:07

I've played a lot of games with starting repls in weird places in code, and hit this a lot (popping a repl that hangs and waits for input is the equivalent of a long running process)

hiredman21:11:49

This is one of the reasons why you shouldn't have top level forms that do stuff, they should just define functions that do stuff, so you load the file in one quick go, then look up defined functions and call them after loafing

João Galrito21:11:26

alright I'll give that a try, thanks

João Galrito22:11:10

I'm starting my Java application with -Dclojure.server.repl={:port 5555 :accept clojure.core.server/repl} but I can't connect to the repl

dpsutton22:11:00

how are you attempting to connect to the repl?

hiredman22:11:11

If you haven't caused the clojure runtime to load, it won't be loaded and won't start the repl

João Galrito22:11:18

I'm loading the clojure runtime via the java clojure api

João Galrito22:11:36

Clojure.var("clojure.core", "require")

João Galrito22:11:58

and then trying telnet 127.0.0.1 5555

hiredman22:11:43

Are you passing -D directly to java or through some intermediary launcher script?

João Galrito22:11:28

intellij with maven runner

João Galrito23:11:01

"C:\Program Files\Java\jdk-17\bin\java.exe" -Dmaven.multiModuleProjectDirectory=D:\code\dataprocessor "-Dclojure.server.repl={:port 5555 :accept clojure.core.server/repl}" etc.

João Galrito23:11:31

if I remove the " from the configuration and put escaped quotes around the map

hiredman23:11:37

Wild west, who knows what those flags are actually getting passed to, and what kind of escaping is needed

João Galrito23:11:37

it errors out saying it cant find class 5555

hiredman23:11:12

Maybe you need quotes around the map, not around the whole thing

hiredman23:11:45

Like, look at the -D example on the page

João Galrito23:11:23

tried all manners

João Galrito23:11:30

no quotes, quotes on the map, quotes on the whole thing

João Galrito23:11:41

quotes on -D and the map

hiredman23:11:42

The -D flag itself cannot be quoted, otherwise it won't be interpreted ass flag

hiredman23:11:51

I guess that actually depends on how the process is being launched, which again, wild west who knows

João Galrito23:11:09

"C:\Program Files\Java\jdk-17\bin\java.exe" -Dmaven.multiModuleProjectDirectory=D:\code\dataprocessor "-Dclojure.server.repl=\"{:port 5555 :accept clojure.core.server/repl}\"" "-Dmaven.home=C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2021.2\plugins\maven\lib\maven3" "-Dclassworlds.conf=C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2021.2\plugins\maven\lib\maven3\bin\m2.conf" "-Dmaven.ext.class.path=C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2021.2\plugins\maven\lib\maven-event-listener.jar" "-javaagent:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2021.2\lib\idea_rt.jar=56643:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2021.2\bin" -Dfile.encoding=UTF-8 -classpath "C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2021.2\plugins\maven\lib\maven3\boot\plexus-classworlds-2.6.0.jar;C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2021.2\plugins\maven\lib\maven3\boot\plexus-classworlds.license" org.codehaus.classworlds.Launcher -Didea.version=2021.2 -Dexec.workingdir=D:\code\dataprocessor "-Dexec.args=-classpath %classpath com.dataserver.Main" "-Dexec.executable=C:\Program Files\Java\jdk-17\bin\java.exe" exec:exec

João Galrito23:11:26

pretty much all the options are quoted

hiredman23:11:12

Maven is likely launching your project in a second jvm too

hiredman23:11:40

So -D flags for the maven jvm don't pass through

hiredman23:11:12

You likely need to add extra layers of quoting and pass it through exec.args

João Galrito23:11:37

so these are options for maven runner JVM

João Galrito23:11:40

not the JVM used for the application

hiredman23:11:46

I mean, I dunno, I haven't seriously used maven in maybe 8 years, and never used intellij, but that is what all those command line options look like

hiredman23:11:43

That is why java.exe is mentioned twice, once as the java used to launch maven and a second time as the java used to launch your project code

João Galrito23:11:30

still not the correct place

João Galrito23:11:49

that's argv for the program, not VM options

hiredman23:11:06

Yeah, in that big list of stuff you pasted there is a -Dexec.args

Jacob Rosenzweig22:11:42

com.zaxxer/HickariCP has a default logger that is preferred by compojure and unfortunately it fails with a NOOP logger (see https://stackoverflow.com/questions/31371993/basic-logging-in-clojure-web-service-not-appearing-on-console). Does anyone have an idea of how to ix this for HickariCP in particular? If I switch to the logback-classic package, I get this very overzealous logger that prints a lot more than what the default compojure logger does.

seancorfield22:11:32

Logging in Java is a giant mess. After a lot of messing about we've gone to log4j2 with everything else bridged into it and then we can control it easily via a Properties file (and we can select different files at startup via a JVM property.)

1
☝️ 1
kulminaator11:11:19

sort of on the same boat. we are using log4j, but barely. we actually build our own json payloads and log4j just provides the tooling to dump those into files that are rotated based on datetime and log levels

deactivateduser21:11:31

Ditto, though I ended up standardising on using tools.logging from my own code, with SLF4J (and logback-classic) as the backing implementation. This is mostly because I found that (at least circa 2014 when I last spent appreciable time on this) SLF4J had the most comprehensive support for forcing the dumpster fire of Java logging facades and implementation calls into using a single implementation (of my choice) at runtime. This is as simple as https://github.com/pmonks/futbot/blob/main/deps.edn#L25-L27, executing https://github.com/pmonks/futbot/blob/main/src/futbot/config.clj#L35-L37, and https://github.com/pmonks/futbot/blob/main/resources/logback.xml`logback-classic`https://github.com/pmonks/futbot/blob/main/resources/logback.xml`SLF4J`https://github.com/pmonks/futbot/blob/main/resources/logback.xml.

kulminaator10:12:54

well that blew in our faces really hard 😄

seancorfield19:12:47

I was glad we keep dependencies up to date and were on 2.14.1 so we could apply the JVM option to disable lookups and restart everything, to tide us over to our next deployment (which has 2.15.0).

seancorfield22:11:00

I'm on my phone so I can't share the deps we use for that.

Jacob Rosenzweig22:11:19

It's no rush, I'll just deal with this verbose logger for now. Better than no logging!

seancorfield22:11:47

And we use tools.logging and a JVM option to tell it to use log4j2 as well