Fork me on GitHub

Is there a recommended tool for compiling *.java files in a deps.edn based project?


@smith.adriane I tend to just shell out and run javac 🙂


I guess I haven't run javac in a really long time. I just need clj to generate a class path to compile? To package the jar, I guess there's an option to tell depstar where to find the *.class files?

Alex Miller (Clojure team)00:02:38

clj -Spath will give you the cp

👍 3

I've found an interesting example of a macro-generating macro in Joy of Clojure, 2nd ed (p. 360). I'm not completely sure why unit-of-distance has to be a macro and not just a function. Is the primary reason here performance?

(defmacro defunits-of [name base-unit & conversions]
  (let [magnitude (gensym)
        unit (gensym)
        units-map (into `{~base-unit 1}
                        (map vec (partition 2 conversions)))]
    `(defmacro ~(symbol (str "unit-of-" name))
       [~magnitude ~unit]
       `(* ~~magnitude
           ~(case ~unit
              ~@(mapcat (fn [[u# & r#]]
                          `[~u# ~(relative-units units-map u#)])

(defunits-of distance :m
  :km 1000
  :cm 1/100
  :mm [1/10 :cm]
  :ft 0.3048
  :mile [5280 :ft])
;; => macroexpands to:
#_(defmacro unit-of-distance [G__21971 G__21972]
    (list '*)
    (list G__21971)

(unit-of-distance 1 :m)
;; => 1

;; look how simple the expansion is!
(macroexpand '(unit-of-distance 1 :cm))
;; => (clojure.core/* 1 1/100)


Where can I find more examples of macro-generating macros and what other use cases are there apart from moving the computation to compile time to achieve faster execution?


macros generating macros is sometime used in cljs to work around not being able to programmatically add definitions to namespaces

Alex Miller (Clojure team)03:02:53

Spec 2 has some macro generating macros for creating custom spec ops (which are macros)


Hmm. More articles should use phrases like "custom spec ops". :D


Are there any general rules relating to defrecords vs deftypes when the fields are mutable? I have a thing that is a grouping together of a bunch of promises representing potential future states. Should I expose the promises with a protocol and make the thing via a deftype? Or use a record and expose the promises via it's keys (and potentially also a protocol?). A third option is to use a map, but since I do need this to participate in other protocols, I'm thinking that might be ruled out first.


defrecord will also implement the protocols that enable assoc .based off your description, an assoced value might break some invariances. it seems like deftype might be preferable so you can limit which protocols are implemented to the ones that make sense


that makes sense, thanks!


I have another really odd bug where it works at the REPL but fails when compiled.

❯ java -jar target/uberjar/enki-master-SNAPSHOT-standalone.jar
Using database: /home/dsp/.enki.db
Exception in thread "main" java.lang.NullPointerException: Cannot invoke "clojure.lang.IFn.invoke(Object)" because the return value of "clojure.lang.IFn.invoke(Object)" is null
        at enki.db$create_all_tables$fn__15909.invoke(db.clj:50)
        at clojure.core$run_BANG_$fn__8790.invoke(core.clj:7715)
        at clojure.core.protocols$iter_reduce.invokeStatic(protocols.clj:49)
        at clojure.core.protocols$fn__8140.invokeStatic(protocols.clj:75)
        at clojure.core.protocols$fn__8140.invoke(protocols.clj:75)
        at clojure.core.protocols$fn__8088$G__8083__8101.invoke(protocols.clj:13)
        at clojure.core$reduce.invokeStatic(core.clj:6828)
        at clojure.core$run_BANG_.invokeStatic(core.clj:7710)
        at clojure.core$run_BANG_.invoke(core.clj:7710)
        at enki.db$create_all_tables.invokeStatic(db.clj:50)
        at enki.db$create_all_tables.invoke(db.clj:46)
        at enki.db$delete_and_recreate_database.invokeStatic(db.clj:63)
        at enki.db$delete_and_recreate_database.invoke(db.clj:58)
        at enki.core$_main.invokeStatic(core.clj:47)
        at enki.core$_main.doInvoke(core.clj:43)
        at clojure.lang.RestFn.invoke(
        at clojure.lang.AFn.applyToHelper(
        at clojure.lang.RestFn.applyTo(
        at enki.core.main(Unknown Source)
❯ lein repl
OpenJDK 64-Bit Server VM warning: Options -Xverify:none and -noverify were deprecated in JDK 13 and will likely be removed in a future release.
nREPL server started on port 44969 on host - 
REPL-y 0.4.4, nREPL 0.8.3
Clojure 1.10.1
OpenJDK 64-Bit Server VM 15+36-Ubuntu-1
    Docs: (doc function-name-here)
          (find-doc "part-of-name-here")
  Source: (source function-name-here)
 Javadoc: (javadoc java-object-or-class-here)
    Exit: Control+D or (exit) or (quit)
 Results: Stored in vars *1, *2, *3, an exception in *e

enki.core=> (-main)
Using database: /home/dsp/.enki.db
I am using run! as recommended previously to force side effects on a seq, but I guess I must be doing it wrong? funcall that blows up:
(defn create-all-tables
  "Create database tables. Expects each table listed in the `tables` set
  to have a corresponding create-`table`-table function defined."
  (run! #((-> (str "create-" % "-table") symbol resolve) x) tables))
I trimmed -main down to:
(defn -main
  [& args]

  (println "Using database:" (:subname db))
For full ref:
(defn delete-and-recreate-database
  (let [database-file (io/file (:subname db))]
    (when (.exists database-file)
      (.delete database-file)))
  (create-all-tables db)
  (create-all-indexes db)
I am sure I must be doing something stupid.


@dsp Are all of those create-*-table functions in the same ns as create-all-tables?


Yes, but not in the same ns as core


I have [enki.db :refer :all] at the top.


Prior to the create-all-tables defn, I do:

(def tables #{"nodekeys"

(def indexes #{"data"})

(def tables-loaded (into #{} (map #(let [p (str "enki/db/sql/" % ".sql")]
                                     (hugsql/def-db-fns p) %)


I know, kinda esoteric. I want to programmatically define the hugsql functions just by crawling the dir.


Ah, I think I found the culprit.


create-all-tables is in enki.db -- and so are all the create-*-table functions, yes?


Map again. Doh.


I probably need to force that one. I keep tripping up on these old bits of seq code around the place...


Likely the def-db-fns isn't being evaluated except at REPL.


Thanks for pointing my head in the right direction.


So, apparently Clojure is not a "real lisp" because I can't metaprogram at run-time and the code is recompiled on the fly. Is this because of the jvm as a base?


Clojure is what it is. There's no universal definition for what a lisp is and then people argue based on their own definitions. I've seen someone claim Clojure isn't a real lisp because symbols are globally interned. Some people claim that if you can't evaluate code from 1962 you aren't a real lisp.


It seems like the assertion that clojure can't be meta-programmed at runtime is wrong. Am I missing something?


> you never stop the running program, the running program itself is a set of objects in memory that you manipulate the same way you manipulate any other object, while it is running. working on my program while it's running is a big reason I enjoy clojure


I have no idea what they are talking about

😌 3

i've never seen one of these discussions benefit anyone in any way. arguing what is a lisp on the internet is just a dumpster fire

😆 3

This is very much one of those "how many angels...?" type questions (and belongs in #off-topic at best since it is not a technical question that can be answered as asked).


I had no idea it is an idealogical thing. I was surprised to read that clojure can't be changed at run-time because it has to be recompiled. At the moment I take the recompilation as an "inbetween" step to be back again and then it's run time again. when it's quick it doesn't make a difference to me.


As others have noted, the question and most of the answers there are... "opinions"... and many of those folks choose their definitions in such as a way as to support the points they want to make so their definitions are also... "opinions"...


(most "opinion-based" questions on StackOverflow get shut down pretty quickly by the moderators so I'm a bit surprised something like that was allowed to "live" in the first place... but that probably says more about the Stack Exchange moderators than it does about the merit of the question 🙂 )


I didn't try to dig into that StackOverflow discussion in depth, but at least from a first reading of what they were claiming about Clojure, it appears some of the opinions expressed about Clojure were factually incorrect. You definitely can build a Clojure program from the ground up in a running JVM, function by function, without ever having to start the JVM over again -- exactly as many other Lisps allow you to do in a REPL.


All Lisps allow you to stop that running program and start a new one using the same source code (if you have saved that source code somewhere accessible to the next running process).


I feel like they are leaning towards the idea of a LISP Machine when they talk about it that way. It's a technique I think which would be interesting to explore, but I'm sure there is plenty of feedback as for why this is a bad idea. Instead of .clj files on a filesystem, you'd essentially program into a REPL and use ns to basically note that you're writing to a different file. The file would be archived in the environment, either spooled to a disk or even a database. From the REPL you could "open" a namespace and it would provide the user with a source code file where you could add comments, examples, etc. As you use different dependencies a graph could be kept with respect to what require/use statements are necessary, and the actual mapping would be applied 👋automagically:wave:. The source would be kept under version control in the system, so you could recall earlier versions of a namespace as well. My deeper vision is that this could build upon what @U017QJZ9M7W is working on so that the published form can be literal programming documentation. Maybe instead of a LISP Machine, it would be a Clojure Machine?

🎉 3
Sam Ritchie23:02:41

it does seem that the deps.edn machinery and namespaces give us the primitives to at least see what that would feel like


@U017QJZ9M7W, The biggest obstacle I can see is with respect to classpath imports. I'd also like to talk with you some other time about the SCIM and literal programming that you're looking at.

Sam Ritchie16:02:21

@U01NAG0N7FS let’s do it! shoot me a DM and we can find some time


into with map should be eager -- but it is kind of dangerous to have a top-level def with side-effects since that will run when the ns is loaded which will also happen during compilation (likely when you build the uberjar).


Yep, doing doall didn't solve it.


The only effect should be function definition I think? Is that dangerous?


What I am essentially trying to accomplish is to programmatically do what the hugsql docs give as an example:

;; The path is relative to the classpath (not proj dir!),
;; so "src" is not included in the path.
;; The same would apply if the sql was under "resources/..."
;; Also, notice the under_scored path compliant with
;; Clojure file paths for hyphenated namespaces
(hugsql/def-db-fns "princess_bride/db/sql/characters.sql")
but instead of using manually-inserted paths, have them constructed from a set, and each table SQL definition file has a create-X-table (where X is the name of the table) function, and then have a function that can re-initialise the table by iterating through the set and calling the related definition. That it works at the REPL, but not when compiled, is perplexing me, esp as you say into should be eager. I also think it previously worked... If there is nothing obvious, I'll try ripping everything out and trying to pinpoint where things went wrong. If I am doing something really wrong or dangerous though, it's a good learning experience.


@dsp "at the REPL, but not when compiled": what exactly do you do outside the REPL to kick this off?


Literally just

❯ java -jar target/uberjar/enki-master-SNAPSHOT-standalone.jar
which I would expect to just enter at -main in enki.core (and confirmed by the println). (-main) works at REPL.


Effects like this should probably kicked off from your -main and not from the top level of your namespace


I had wanted to keep db funcs in the db namespace for cleanliness, but maybe there is a way for me to do definitions there from core.


It'll be exposed via HTTP API, so has to be accessible through the regular execution chain.


main functions need to return an integer right?


(System/exit n) is what is the return code


or an exception will also trigger a non-zero one


Am just trying to figure out exactly what's going wrong and why. When expanding -main like so:

(defn -main
  [& args]

  (println "Using database:" (:subname db))
  (let [database-file ( (:subname db))]
     (when (.exists database-file)
       (.delete database-file)))
  (println tables-loaded)
  (println 'enki.db/create-data-table)
  (create-all-tables db))
I can see the following:
❯ java -jar /home/dsp/Development/enki/target/uberjar/enki-master-SNAPSHOT-standalone.jar
Using database: /home/dsp/.enki.db
#{contactkeys authtokens incoming outgoing data nodekeys mykeys}
Exception in thread "main" java.lang.NullPointerException: Cannot invoke "clojure.lang.IFn.invoke(Object)" because the return value of "clojure.lang.IFn.invoke(Object)" is
       at enki.db$create_all_tables$fn__15912.invoke(db.clj:50)
       at clojure.core$map$fn__5866.invoke(core.clj:2755)
       at clojure.lang.LazySeq.sval(
       at clojure.lang.LazySeq.seq(
       at clojure.lang.RT.seq(
       at clojure.core$seq__5402.invokeStatic(core.clj:137)
       at clojure.core$dorun.invokeStatic(core.clj:3133)
       at clojure.core$doall.invokeStatic(core.clj:3148)
       at clojure.core$doall.invoke(core.clj:3148)
       at enki.db$create_all_tables.invokeStatic(db.clj:50)
       at enki.db$create_all_tables.invoke(db.clj:46)
       at enki.core$_main.invokeStatic(core.clj:52)
       at enki.core$_main.doInvoke(core.clj:43)
       at clojure.lang.RestFn.invoke(
       at clojure.lang.AFn.applyToHelper(
       at clojure.lang.RestFn.applyTo(
       at enki.core.main(Unknown Source)


So it looks as though the functions are indeed being defined in the right namespace.


Indeed, a direct call in -main to that create-data-table function that is defined programmatically works. So I suspect it's related to the symbol resolve?


That's why I asked which functions are in which namespaces... but you would likely be safer to specify the namespace explicit in the symbol before calling resolve.


Yep, that is definitely the issue.


Strange that it works at the REPL. In db namespace I did:

(defn show-all-resolved-symbols
  (map #(-> (str "create-" % "-table") symbol resolve) tables))
and in -main in core I call it like
(println (into #{} (show-all-resolved-symbols)))
At REPL it is:
enki.core> (show-all-resolved-symbols)
but when compiled it is: #{nil} I don't know why, but I will try explicit namespace references, I imagine that will work


dev=> (doc resolve)
([sym] [env sym])
  same as (ns-resolve *ns* symbol) or (ns-resolve *ns* &env symbol)
^ *ns* behaves differently in the REPL to in an uberjar is the TL;DR.


If you do (-> (str "enki.db/create-" % "-table") symbol resolve) I think it'll solve your problem @dsp


or resolve from the other ns using ns-resolve - should yield the same (`(ns-resolve 'enki.db %)`)


(defmacro create-all-tables
  "Create database tables. Expects each table listed in the `tables` set
  to have a corresponding create-`table`-table function defined."
  `(run! #((ns-resolve (find-ns 'enki.db) (-> (str "create-" % "-table") symbol)) ~x) ~tables))


This seems to work. Thanks so much! I hope one day I more fully understand the pitfalls of relying on REPL behaviour as being akin to the endproduct.


Gone midnight, bedtime for me. But glad to squash this bug that's had me pulling at my hair for the last hour or two 🙂 Appreciate the help folks.