I’m running tests of my student’s submitted code in a docker image (so not interactively). I’m using lein test homework.foo-test and catching the stdout and stderr.
What I’d rather do, but not sure whether it is possible is first run lein just to compile the student code. and if there are compilation errors I can report them in a meaningful way back to the student. And only if there are no compilation errors, THEN run the tests.
What’s the best way to do this?
what I've used in the past is a small shell script that iterates every file under a given directory, and attempts to load it into clojure, writing a list of files that didn't load successfully to stderr. any clojure build tool can spit out a classpath for a project, and that can be used with java and clojure.jar to load each file.
this is equivalent to lein check except it doesn't do reflection warnings and can scan directories that aren't in your project src paths, and of course it doesn't require any specific build tool
lein check will run a check over the source code but not tests -- and it will also check for reflection warnings (which you may not want?).
There doesn't appear to be any task/function for similarly checking tests -- other than trying to run them (via lein test).
I tried just lein run homework.test-hello which is the name of the namespace which contains one particular test case for a student assignment.
But lein exists with status=1 with the error message
No :main namespace specified in project.clj.
I tried with the command line lein run -m homework.hello-test and I get the following errors
lein run -m homework.hello-test
WARNING: ignoring checkouts directory /Users/jnewton/Repos/courses/dir-clojurein/clojurein/clojurein-source-code/checkouts/README.md as it does not contain a project.clj file.
Execution error at user/eval138 (form-init13231397332125612410.clj:1).
Cannot find anything to run for: homework.hello-test
Full report at:
/var/folders/c1/sqxjhjm15gdcyr49z7r4k1mr0000gn/T/clojure-7180557784506930078.ednDo I need to put a -main function in all of my test case files? won’t that interfere with the normal testing with lein test … ?
OK, i’m giving it a try, and I think I’m ALMOST there.
I’m in the directory where the project.clj is found. And I call the following shell command:
lein run -m homework.test-hello
lein gives me the following error
lein run -m homework.test-hello
lein run -m homework.test-hello
WARNING: ignoring checkouts directory /Users/jnewton/Repos/courses/dir-clojurein/clojurein/clojurein-source-code/checkouts/README.md as it does not contain a project.clj file.
Can't find 'homework.test-hello' as .class or .clj for lein run: please check the spelling.
Execution error (FileNotFoundException) at user/eval138$fn (form-init13258461395224719360.clj:1).
Could not locate homework/test_hello__init.class, homework/test_hello.clj or homework/test_hello.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.
Full report at:
/var/folders/c1/sqxjhjm15gdcyr49z7r4k1mr0000gn/T/clojure-4052305510252037654.ednI would like it to find the file test/homework/hello_test.clj which defines the namespace
(ns homework.hello-test
(:require [homework.hello :as sut]
[clojure.string :as string]
[common.util :refer [testing-with-timeout]]
[clojure.test :refer [deftest is]]))Do I have to do something special to tell lein to add test to the search path?
Because when I just run lein test it finds the file without any problem.
hmmm. I asked chatgpt and it gave me a solution which seems to work. (still testing though).
It said to add the following to my project.clj
:profiles
{:check-before-test {:resource-paths ["test"]}}
then to test with a command line like
lein with-profile +check-before-test run -m homework.hello-testit is unsettling that chatgpt found an elegant solution.
I hope the fact that chatgpt finds such solutions does not mean that people will stop talking to each other. community is important.
I think you could have used the existing :test profile instead of creating a new one.
Well there is already something in the :test and since I don’t really understand what it is doing, I didn’t want to tempt it.
:profiles {:test {:plugins [[lein-test-report-junit-xml "0.2.0"]]
:test-report-junit-xml {:output-dir "."}
}
;; chatgpt suggested adding the following line :check-before-test
;; this allows the test directory to be findable by the command line
;; lein with-profile +check-before-test run -m homework.hello-test
;; This allows us to try to run the -main function at the bottom of the test
;; files before actually launching any tests. If the file fails to load
;; then we report a more meaning error message.
:check-before-test {:resource-paths ["test"]}
:uberjar {:aot :all
:jvm-opts ["-Dclojure.compiler.direct-linking=true"]}}if I change the :resource-paths can I be sure I’m not overriding an existing implicit :resource-paths or whether the testing enviornment would be confused by my changing the resource paths?
on the other hand, probably you’re right.
Try it and see 🙂
lein run requires a -main function, yes, but it won't interfere with lein test, so you could append (defn -main [& args]) to each test file fairly safely I think.