Fork me on GitHub
#clojure
<
2021-11-09
>
emccue03:11:45

:repositories ^:replace {"jcenter" {:url       ""
                                      :snapshots false
                                      :releases  {:checksum :fail :update :always}}
                           "maven"   {:url      ""
                                      :releases {:checksum :fail :update :always}}
                           "clojars" {:url      ""
                                      :releases {:checksum :fail :update :always}}}

emccue03:11:53

translating this leiningen to deps edn

emccue03:11:14

there is no documentation of the :releases key - is that important?

emccue03:11:02

its not popping out in the docs and its an extra key so i don’t imagine anything is gonna complain if its not supported

qqq09:11:32

Is there a guide for running Clojure on a cluster? Say you have 20 machines fired up on EC2, all with ssh access and jvm + clojure installed. What is the standard way for running Clojure on these machines? (In Elixir / Erlang, one would setup distributed Erlang, and have them all join the same dist Erlang cluster).

Linus Ericsson10:11:20

One could argue that the JVM (and in some ways maybe Clojure) is not built with clustering as the primary goal (as Erlang and the OTP is). There is nothing wrong with the networking in the JVM, but it's not OTP, and the networking model is not as elegant as the one in erlang (with pattern matching etc). Clojure could do really cool things in clustering mode. There are some things built: One notable historical thing originally built in clojure is apache storm. http://storm.apache.org/ Another thing to checkout is probably OTP-like https://github.com/suprematic/otplike There is also some scope capture things that can migrate environments between processes, and a lot of tools just wait to be put into a clustering thing. I don't know if you've seen: https://www.clojure-toolbox.com/ but it lists at least some of the more canonical libraries for clojure.

emccue12:11:27

i mean, depends what you need the clustering behavior for. You can set up clojure w/ akka and i’d imagine it would work the same as erlang

Ben Sless14:11:04

Would you expect those machines to communicate with each other in some way? how? You can always handle deployment and management with things like K8S where your final artifacts are docker images

Narendra11:11:31

This is interesting. I wonder how clustering mode would look like for Clojure if built using the concepts that we already have. I can imagine having distributed core.async channels that have some way of managing their identity across JVMs.

Fahd El Mazouni14:11:28

Hi ! say my code contains a (def toot (SomeClass.)) would the jar contain a byte code representation of the SomeClass instance that was created during clojure compilation ? any help understanding this would be much appreciated

p-himik14:11:06

The constructor will be called at run time, not compile time.

Fahd El Mazouni14:11:11

thanks ! how is it different than an expression doing x and returning a value (42 for exemple) ?

p-himik14:11:49

Not sure what you mean by "doing x", but e.g. with (def x (+ 1 2)), the (+ 1 2) part will also be executed at run time. Maybe there are exceptions to that, but I'm not familiar with them.

jkxyz14:11:42

The reference page on compilation may be interesting to you: https://clojure.org/reference/compilation The compiler generates an init class for the namespace which evaluates those top-level expressions at runtime, when you load the namespace.

Fahd El Mazouni14:11:09

ok so if I understand correctly, the evaluation happens before compilation and is only used to expand macros and such, and whatever expressions are used to initialize vars will be evaluated at runtime as one would expect, right ?

p-himik14:11:37

The (+ 1 2) part is not evaluated. It's read and analyzed. If such an expression contains macros, they're expanded. And then the result is compiled.

Fahd El Mazouni15:11:08

I'm not trying to be annoying @U2FRKM4TW 😛 thanks for your patience,

(ns toot.core (:gen-class))

(def toto (do (println "hello") 42))
(defn -main "I don't do a whole lot." [] (println "Hello, World!"))
running lein uberjar on this prints the hello, the hello is also there when you run it as you'd expect

p-himik15:11:11

Wasn't suggesting that you're annoying. :) I'm not a native English speaker and my written speech can be more terse than needed. That's an interesting example. Perhaps I'm wrong then.

p-himik15:11:10

I see now that clojure.lang.Compiler/compile1 has this block:

Expr expr = analyze(C.EVAL, form);
objx.keywords = (IPersistentMap) KEYWORDS.deref();
objx.vars = (IPersistentMap) VARS.deref();
objx.constants = (PersistentVector) CONSTANTS.deref();
expr.emit(C.EXPRESSION, objx, gen);
expr.eval();
Notice that there's eval at the very end there. No clue why.

Fahd El Mazouni15:11:58

thanks ! maybe it's how the compiler catches errors

p-himik15:11:01

@U064X3EF3 Could you please help us understand the intent of the last line in the code block above? Why does compilation also need to evaluate something, while also ignoring the return value? I very vaguely remember cautionary tales about side effects at the top level of namespaces being problematic for compilation, but I can't recall the specifics and can't find anything related in the documentation.

Ed15:11:40

if you (def toto (do (println "hello") 42)) at the top level in a namespace, that will be executed at load time. This happens both when you AOT a ns and when you require it. All def's are side effecting forms because they change the current namespace, usually defining a function or whatever, but if you have arbitrary side effects that will happen both when you gen-class and when you load the file. Does that make sense?

p-himik15:11:19

> that will be executed at load time But I'm talking about compile time. > All `def`'s are side effecting forms because they change the current namespace At run time that makes sense. But why would they need to be evaluated at compile time?

Ed16:11:23

I think that loading is part of both compiling and requiring. Before the compiler can spit out any bytecode it will ensure that the ns loads, so it can do things like macro expansion, etc. Consider the case where we have a (def x (throw (Exception. "err"))) the code isn't going to load. I think without a type system to help decide what side effects need to be run in each phase, this is probably the best that can be done.

Alex Miller (Clojure team)16:11:59

yeah, that's not a good explanation

Alex Miller (Clojure team)16:11:39

"load" is the primary operation here which means to ensure that a namespace and its code is loaded in the classloader

Alex Miller (Clojure team)16:11:01

require triggers a load (if needed), load may load either existing bytecode (aot compiled source) or source. in the case of the latter, compilation happens to make the bytecode. If we are also compiling (this is just a dynvar), then the bytecode is also written to disk.

Alex Miller (Clojure team)16:11:42

I don't think this is directly related to the original question though

Alex Miller (Clojure team)16:11:16

if the question is why eval - b/c that's how effects happen. in the case of a top-level def for example, you need to def the var or later top-level forms that use it can't be compiled

Alex Miller (Clojure team)16:11:06

loading, while triggered by the load of a namespace (usually), is really a form-by-form operation where each form is compiled and evaluated in order.

Alex Miller (Clojure team)16:11:22

compile time is not a thing separate from loading; it is a side effect of loading

Ed16:11:40

apologies for the bad explanation ... thanks Alex

p-himik17:11:17

> you need to def the var or later top-level forms that use it can't be compiled But why would you need the value of a var, assuming there's no direct linking? In other words, from the perspective of the compiler compiling some ns a that uses b/thing, would it make any difference if b had (def thing (println "thing")) as opposed to just (def thing)?

Alex Miller (Clojure team)17:11:41

How would the compiler know to magically do one thing but not the other?

Alex Miller (Clojure team)17:11:55

Compiling is not a separate thing, it is a subset of loading

Alex Miller (Clojure team)17:11:27

Loading is also evaluating

p-himik18:11:52

Hmm. It still doesn't sit right in my head so I probably don't have the right model of the whole process. But it's not that important, and eventually I'll probably end up compiling Clojure myself and experimenting a bit. Thanks!

Alex Miller (Clojure team)18:11:07

the key to everything is that Clojure rotates around the evaluation of forms - the REPL is not just an interface, it is Clojure - read/eval, read/eval load is just evaluation of forms read from a file, one by one compilation is an implementation detail of eval

p-himik18:11:55

Alright, that makes sense. But what about explicit compilation done by calling compile? Doesn't it make the compilation a thing in and of itself, not just an implementation detail?

Alex Miller (Clojure team)19:11:55

I know that sounds like a separate thing but what I'm saying is that calling compile merely means writing some of the intermediate implementation parts of load/evaluation to disk

👍 1
Alex Miller (Clojure team)19:11:30

"remember this stuff I compiled while evaluating and write it down"

Alex Miller (Clojure team)19:11:12

which is a totally different idea than say, the Java compiler

p-himik19:11:21

Huh, alright, something clicks now. And it means that one can't possibly compile CLJ code in an environment where it can't be run (missing run time requirements, like files, env vars, etc), right?

Alex Miller (Clojure team)19:11:36

well I'm not saying it's not possible but it would be different :)

👍 1
Fahd El Mazouni20:11:05

thank you guys !

emccue15:11:15

Execution error (ExceptionInfo) at clojure.tools.build.tasks.uber/explode (uber.clj:172).
Cannot write license/LICENSE from xml-apis/xml-apis as parent dir is a file from another lib. One of them must be excluded.

emccue15:11:52

getting this error building an uberjar - not sure the right path to debug

Alex Miller (Clojure team)15:11:07

you have both a license (or maybe LICENSE) and a license/LICENSE file in your exploded uber jars. license can't be both a file and a directory at the same time, so you need to add an exclusion on one of them

emccue15:11:50

If i add exclude license it won’t accidentally ignore com/company/thing/license.clj right?

Alex Miller (Clojure team)15:11:49

for matching the full path

👍 1
dpsutton15:11:06

when i first debugged this issue it was helpful to run it from a repl and then go inspect the temp directory it builds up to create the jar. I don’t remember how much information was in the error message but it could be helpful to add some logging to which jar it was as well

Alex Miller (Clojure team)15:11:03

xml-apis/xml-apis is the lib the second file is found in

Alex Miller (Clojure team)15:11:30

the lib the original file is from is no longer known at the point when the error occurs

dpsutton15:11:14

correct. but being able to see which license it is was helpful for me. so having the temp dir still around made it easy to see “this is a license for X” and then being able to choose which i wanted to exclude rather than just excluding the second because some other library had already claimed it

Alex Miller (Clojure team)17:11:29

this might be better in #tools-build btw

emccue17:11:38

i’ll move it

ribelo23:11:54

Is it possible to similarly access metadata in a function as in a macro? for a macro i can use (meta &form)

ribelo23:11:36

I mean is it possible to do such a thing for a function?

^{:some-key :some-value}
(myfn)

Abhinav04:11:24

&form is a special variable only available inside a macro. could you give an example of what you want to accomplish?