Clojurians Log v2

How exactly are you running the Polylith tests in CI? (and I suspect this belongs in #polylith but I'll try to answer it here)

We run test on github action with following command:

AWS_REGION=eu-west-1 AWS_CBOR_DISABLE=true clojure -M:poly test :project since:release

And how are you running the tests locally?

I run test in the repl.

Well, that's not the same environment...

Does clojure -M:poly test since:release fail locally from a terminal?

(not sure why you have :project in there if you're not specifying a particular project to use?)

I will launch test locally when I will be in front of my laptop (evening here in France)

Follow-up in #polylith since this is a classpath dependency issue due to something in your Polylith structure, not a SQL issue, as far as I can tell. In your REPL, you'd be in the :dev project with all bricks on the classpath and all of their dependencies. When you run clojure -M:poly test it is running in the project context, with the classpath constructed from each individual project.

Ok, i will ask there.

Thanks for you help

A last question @seancorfield: What do you mean by :dev project? In Polylith projects folder, we have all our business project but no dev project. When we launch test in repl, we use the same test-system as the one used in CI test environnement. This test-system is still for the moment the test-system of one of the bases , not move yet to a specific component (legacy from before Polylith).

I moved my question to #polylith in the mean time.

The :dev alias represents the development project, intended for use with the REPL. Maybe you need to have another read of the Polylith docs? It sounds like you don't have a grasp on some of the basics yet...?

(and there's also a :dev command-line argument for some poly commands that adds the development project into the scope of the command -- e.g., you normally test via projects but that doesn't include development, unless you add the :dev command-line argument... not to be confused with using :dev as an alias with the Clojure CLI when you are starting a REPL)

Hello @seancorfield, sorry for late reply, we were focused on this issue. True, I am quite new to polylith. As I work on an already setup project, I used usual command like create a new component and so on. Now, I read the documentation and understand better how it works. Anyway, sound our issue is not a polylith one. In order to validate that the driver is well in the classpath, I have add this line in a test namespace: (prn (Class/forName "com.mysql.jdbc.Driver")) With this line, there is no more exception.

So we have the feeling that the jdbc driver loading is lazy and not realized without the prn . Just a feeling. Do you have an idea?

Can you share the code of how you set up the HikariCP connection pool?

Hi Sean! (I work with rpatrelle) Basically this is how our component looks like https://github.com/patinside/sql_classpath_issue/blob/main/components/sql/src/beop/sql/core.clj

OK, so that would attempt to load com.mysql.cj.jdbc.Driver first and if that failed it would attempt to load com.mysql.jdbc.Driver -- because MySQL's driver has changed the class name.

You're using a pretty old driver mysql/mysql-connector-java {:mvn/version "5.1.49"}

Is it possible you have both the old and the new driver on your classpath?

How did you know that com.mysql.cj.jdbc.Driver is loaded first? It's Hikari which import this?

connection/jdbc-url will construct the :jdbcUrl from a db-spec and as part of that it will try to load (and cache) driver classes that correspond to the :dbtype.

For MySQL -- and MySQL alone -- two possible driver classes exist: the new one (with .cj) and the old one, and next.jdbc tries them in a specific order (new, then old).

Thanks for the clarification! When I run -Stree I see only the old driver.

How are you running -Stree -- Polylith uses different classpaths for dev (REPL) and for tests (projects)

like this: clojure -Stree -M:dev

You really need to run clojure -Stree inside each of the projects folders

The :dev alias is for the REPL, not the tests (although you can ask poly test to also include development when running tests -- but the default is per-project)

Each projects only have the old mysql version

Is that github repo an actual repo of the problem? Should I expect to see the driver not found error if I clone and run it?

Unfortunately we don't reproduce the behaviour in the public repo.

That is why I still think this is a Polylith-related problem in your actual repo 😕

If you can repro it in a public repo, I can try to help.

That public repo "works" (modulo failing to find the expected rows in the database) and uses the old driver -- so that shows there isn't a problem with next.jdbc -- and at work we use it with the new driver and Polylith and that also works.

When it fails with No suitable driver do you get a full stack trace?

Is not next related indeed. In fact, the error occurs when other projects are tested. If the ci test the project alone, no error.

Not sure what you mean by "If the ci test the project alone" -- how exactly are you running those tests?

We launch the poly command: clojure -M:poly test :project since:release we get the projects to test. If the culprit project is the only to test, no problem. The full exception thrown:

clojure.lang.ExceptionInfo: Error while evaluating form (do (clojure.core/use (quote clojure.test)) (clojure.core/require (quote beop.export-api.modules.schedule-test)) (clojure.test/run-tests (quote beop.export-api.modules.schedule-test))) in class-loader. Cause: java.lang.reflect.InvocationTargetException; java.lang.RuntimeException: Failed to get driver instance for jdbcUrl=jdbc:; java.sql.SQLException: No suitable driver {:form (do (clojure.core/use (quote clojure.test)) (clojure.core/require (quote beop.export-api.modules.schedule-test)) (clojure.test/run-tests (quote beop.export-api.modules.schedule-test)))}
	at polylith.clj.core.test_runner_orchestrator.core$__GT_eval_in_project$fn__27578.invoke(core.clj:110)
	at polylith.clj.core.clojure_test_test_runner.core$run_test_statements$fn__41996.invoke(core.clj:59)
	at polylith.clj.core.clojure_test_test_runner.core$run_test_statements.invokeStatic(core.clj:58)
	at polylith.clj.core.clojure_test_test_runner.core$run_test_statements.invoke(core.clj:52)
	at polylith.clj.core.clojure_test_test_runner.core$create$reify__42023.run_tests(core.clj:98)
	at polylith.clj.core.test_runner_orchestrator.core$run_tests_for_project_with_test_runner.invokeStatic(core.clj:88)
	at polylith.clj.core.test_runner_orchestrator.core$run_tests_for_project_with_test_runner.invoke(core.clj:77)
	at polylith.clj.core.test_runner_orchestrator.core$run_tests_for_project.invokeStatic(core.clj:159)
	at polylith.clj.core.test_runner_orchestrator.core$run_tests_for_project.invoke(core.clj:129)
	at polylith.clj.core.test_runner_orchestrator.core$run.invokeStatic(core.clj:214)
	at polylith.clj.core.test_runner_orchestrator.core$run.invoke(core.clj:197)
	at polylith.clj.core.test_runner_orchestrator.interface$run.invokeStatic(interface.clj:5)
	at polylith.clj.core.test_runner_orchestrator.interface$run.invoke(interface.clj:4)
	at polylith.clj.core.command.test$run.invokeStatic(test.clj:11)
	at polylith.clj.core.command.test$run.invoke(test.clj:5)
	at polylith.clj.core.command.core$execute.invokeStatic(core.clj:118)
	at polylith.clj.core.command.core$execute.invoke(core.clj:89)
	at polylith.clj.core.command.interface$execute_command.invokeStatic(interface.clj:5)
	at polylith.clj.core.command.interface$execute_command.invoke(interface.clj:4)
	at polylith.clj.core.poly_cli.core$_main.invokeStatic(core.clj:33)
	at polylith.clj.core.poly_cli.core$_main.doInvoke(core.clj:7)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.lang.Var.applyTo(Var.java:705)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.main$main_opt.invokeStatic(main.clj:514)
	at clojure.main$main_opt.invoke(main.clj:510)
	at clojure.main$main.invokeStatic(main.clj:664)
	at clojure.main$main.doInvoke(main.clj:616)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.lang.Var.applyTo(Var.java:705)
	at clojure.main.main(main.java:40)
Caused by: java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at polylith.clj.core.common.class_loader$invoke_in_STAR_.invokeStatic(class_loader.clj:31)
	at polylith.clj.core.common.class_loader$invoke_in_STAR_.doInvoke(class_loader.clj:27)
	at clojure.lang.RestFn.invoke(RestFn.java:494)
	at polylith.clj.core.common.class_loader$eval_in_STAR_$print_read_eval__3887.invoke(class_loader.clj:49)
	at polylith.clj.core.common.class_loader$eval_in_STAR_.invokeStatic(class_loader.clj:51)
	at polylith.clj.core.common.class_loader$eval_in_STAR_.invoke(class_loader.clj:45)
	at polylith.clj.core.common.class_loader$eval_in.invokeStatic(class_loader.clj:61)
	at polylith.clj.core.common.class_loader$eval_in.invoke(class_loader.clj:60)
	at polylith.clj.core.common.interface$eval_in.invokeStatic(interface.clj:30)
	at polylith.clj.core.common.interface$eval_in.invoke(interface.clj:29)
	at polylith.clj.core.test_runner_orchestrator.core$__GT_eval_in_project$fn__27578.invoke(core.clj:108)
	... 30 more
Caused by: java.lang.RuntimeException: Failed to get driver instance for jdbcUrl=jdbc:
	at com.zaxxer.hikari.util.DriverDataSource.<init>(DriverDataSource.java:114)
	at com.zaxxer.hikari.pool.PoolBase.initializeDataSource(PoolBase.java:326)
	at com.zaxxer.hikari.pool.PoolBase.<init>(PoolBase.java:112)
	at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:93)
	at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
	at next.jdbc.connection$make_connection.invokeStatic(connection.clj:452)
	at next.jdbc.connection$make_connection.invoke(connection.clj:436)
	at next.jdbc.connection$eval30353$fn__30354.invoke(connection.clj:481)
	at next.jdbc.protocols$eval30083$fn__30084$G__30074__30091.invoke(protocols.clj:25)
	at next.jdbc.result_set$eval31056$fn__31064.invoke(result_set.clj:934)
	at next.jdbc.protocols$eval30115$fn__30116$G__30106__30125.invoke(protocols.clj:34)
	at next.jdbc.result_set$eval31095$fn__31100.invoke(result_set.clj:1023)
	at next.jdbc.protocols$eval30115$fn__30116$G__30106__30125.invoke(protocols.clj:34)
	at next.jdbc$execute_BANG_.invokeStatic(jdbc.clj:251)
	at next.jdbc$execute_BANG_.invoke(jdbc.clj:238)
	at beop.export_api.test_utils$with_test_system.invokeStatic(test_utils.clj:43)
	at beop.export_api.test_utils$with_test_system.invoke(test_utils.clj:35)
	at clojure.test$compose_fixtures$fn__9850$fn__9851.invoke(test.clj:694)
	at clojure.test$default_fixture.invokeStatic(test.clj:687)
	at clojure.test$default_fixture.invoke(test.clj:683)
	at clojure.test$compose_fixtures$fn__9850.invoke(test.clj:694)
	at clojure.test$compose_fixtures$fn__9850.invoke(test.clj:694)
	at clojure.test$test_vars.invokeStatic(test.clj:731)
	at clojure.test$test_all_vars.invokeStatic(test.clj:737)
	at clojure.test$test_ns.invokeStatic(test.clj:758)
	at clojure.test$test_ns.invoke(test.clj:743)
	at clojure.core$map$fn__5935.invoke(core.clj:2772)
	at clojure.lang.LazySeq.sval(LazySeq.java:42)
	at clojure.lang.LazySeq.seq(LazySeq.java:51)
	at clojure.lang.Cons.next(Cons.java:39)
	at clojure.lang.RT.boundedLength(RT.java:1790)
	at clojure.lang.RestFn.applyTo(RestFn.java:130)
	at clojure.core$apply.invokeStatic(core.clj:669)
	at clojure.test$run_tests.invokeStatic(test.clj:768)
	at clojure.test$run_tests.doInvoke(test.clj:768)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.core$eval73541.invokeStatic(NO_SOURCE_FILE:0)
	at clojure.core$eval73541.invoke(NO_SOURCE_FILE)
	at clojure.lang.Compiler.eval(Compiler.java:7194)
	at clojure.lang.Compiler.eval(Compiler.java:7184)
	at clojure.lang.Compiler.eval(Compiler.java:7149)
	at clojure.core$eval.invokeStatic(core.clj:3215)
	at clojure.core$eval.invoke(core.clj:3211)
	at clojure.core$eval73461.invokeStatic(NO_SOURCE_FILE:0)
	at clojure.core$eval73461.invoke(NO_SOURCE_FILE)
	at clojure.lang.Compiler.eval(Compiler.java:7194)
	at clojure.lang.Compiler.eval(Compiler.java:7149)
	... 45 more
Caused by: java.sql.SQLException: No suitable driver
	at java.sql/java.sql.DriverManager.getDriver(DriverManager.java:298)
	at com.zaxxer.hikari.util.DriverDataSource.<init>(DriverDataSource.java:106)
	... 91 more

Without access to your source code, I really can't help.

What is the code around these lines:

at beop.export_api.test_utils$with_test_system.invokeStatic(test_utils.clj:43)
	at beop.export_api.test_utils$with_test_system.invoke(test_utils.clj:35)

(but I'll probably need to keep asking to see more code and more deps.edn files)

Just a function and a call to jdbc/execute! to create test table. I will continue the debug tomorrow, I will reach you if I find something interesting. Huge thanks for your support and help!

> Just a function and a call to jdbc/execute! to create test table. This is not a useful answer. I would need to know exactly what the arguments were and how they were constructed.