Fork me on GitHub
#beginners
<
2020-12-22
>
Harley Waagmeester00:12:57

Would using def as a mutation mechanism cause any compiler or execution anomally ? i.e. (def a [1 2 3 4]), and then later on mutating a with (def a (conj a 5))

phronmophobic00:12:33

not sure what you mean by anomaly. would the following work for your use case?

(def a (atom [1 2 3 4]))

;; later on
(swap! a conj 5)

Harley Waagmeester00:12:15

I'm sure that would work. It is more idiomatic for sure.

noisesmith01:12:03

def is thread safe, swap! is safer because it retries, to answer your question, no there are no compiler or execution anomolies caused by def (other than side effects inside def, which run while compiling, a common gotcha for making packages for deployment)

noisesmith01:12:48

so for example someone runs $ lein uberjar and it never returns because it starts a server or something while compiling your code

👆 3
raspasov03:12:49

@U051SS2EU that’s gotta be one of the leading beginner gotchas 🙂 I remember running into that back in 2013 when I first tried making an uberjar

andarp07:12:00

Are there any good books, guides, videos or other resources that focus on teaching the ecosystem surrounding Clojure? As someone who has barely touched Java I feel a lot more confused about maven and POM and deps and uberjars and nREPL etc etc than about LISP syntax or functional programming. I feel like a lot of the literature is aimed at people who are already familiar with the Java landscape, or it doesn’t touch the practicalities of running and deploying Clojure at all. I’ve read two Clojure books and I still feel like I don’t really understand how the REPL interaction with a running program works, or how I am supposed to deploy a Clojure program in a somewhat professional manner. In some ways I feel like starting out with Cursive has hampered my learning as the IntelliJ environment sets up a lot of facilities for me.

jumar08:12:35

(n)REPL stuff is really orthogonal to the Java/JVM ecosystem. For books: • IIRC Clojure Applied has some good practical stuff. • Web development with Clojure has the "Deployment" chapter too. But I don't think you really need books for this. It comes to identifying which pieces confuses you and reading the docs or online guides (e.g. for maven deps or uberjars). I also barely understand nREPL and deal with the REPL just fine 🙂. When it comes to Java interop it's fine to learn as you go - you can read javadoc or try stackoverflow every now and then (but in my experience it's not needed that much; however, I do have a java background) One piece I find difficult is Java/JVM concurrency - if you need that read "Java Concurrency in Practice" or similar

noisesmith19:12:08

clojure itself is the REPL, and happens to have functions that third party tools use to run in other ways nREPL is a tool that allows multiple parallel networked "sessions" talking to one clojure, and forwarding results to queries from each client, while also bolting on various convenient features (usually for editor integration) in a running program, you can (if you choose) serve up the singleton clojure instance via the network (with nREPL or the simpler socket REPL that clojure natively provides), this is optional and often a bad idea outside local debugging purproses

noisesmith19:12:40

the tl;dr on maven: clojure programs are java programs, like all java programs they access runtime resources through the classpath, the dominant way of managing libraries via the classpath is maven which uses jars (zip files with a specific structure) and pom files (metadata about what is provided by the jar, and the dependencies it needs (other jars...))

noisesmith19:12:20

you don't need a deep understanding of repls (outside clojure itself, which you are already learning and is a repl) - either you want the tooling (and therefore follow the recipe) or don't (and can ignore the recipe) you might end up needing a better understanding of deps (especially as they can conflict and break one another), but as a beginner you can ease yourself into this by avoiding external libraries not provided by some recipe on project creation

andarp19:12:46

Thanks a lot @U051SS2EU, this really helps. I know about JARs in general, but are jars used together with maven essentially "packages and a system to manage these packages"? And pom files are manifests for the "packages"?

noisesmith19:12:58

right, packages and manifests precisely - the special thing is that it's kept as a versioned cache, rather than global system or user level "installs" - so two clojure projects can each use conflicting versions of the same library

noisesmith19:12:12

it's all under ~/.m2/ by default

andarp19:12:20

That's great to hear. Lots of languages have had to solve that issue over the last decade or so.

noisesmith19:12:49

right, clojure itself has never needed to care as it used java, which already solved the issue (very elegantly IMHO) via classpath and maven

andarp19:12:23

And is there some sort of de facto standard "feed" for maven packages?

andarp19:12:03

I put this in a deps.edn file and it works, but I have no idea where it's downloading it all from:

{:deps {org.clojure/data.csv {:mvn/version "1.0.0"}}}

noisesmith19:12:06

maven repositories, we use http://clojars.org and http://maven.org by default with all mainstream clojure package managers, and those can be queried via web UI

andarp19:12:37

So clojars is the place to put clojure packages and http://maven.org is "everything Java"?

noisesmith19:12:02

almost - clojure.core and all the important clojure libs are on maven - clojure is a java library

3
noisesmith19:12:38

clojars is very clojure specific, and leiningen (once the only good clojure package management choice) makes it easy to put your code on clojars

andarp19:12:20

And uberjars - is that creating a single .jar file containing all dependencies?

noisesmith19:12:22

clojars has looser rules about package naming, versioning, etc.

andarp19:12:29

Except a java runtime, I would assume?

noisesmith19:12:31

right, the dependencies are all resources (.clj files, class files, etc.) and clojure itself is the java program which will end up using those resources, but only rarely is the vm itself packaged with all that

noisesmith19:12:48

it is possible to make an "executable jar" containing vm plus deps, but most people don't need or want that

noisesmith19:12:28

@U01GZD00HA8 a fun thing is that you can use clojure to examine the resources on your classpath (demo in a moment)

👀 3
noisesmith19:12:08

(cmd)user=> +
#object[clojure.core$_PLUS_ 0x54336c81 "[email protected]"]
(ins)user=> ;; from the above, we can get a file name: clojure/core
(ins)user=> (io/resource "clojure/core.clj") ;gets us the handle to read it
#object[java.net.URL 0x3fcdcf "jar:file:/home/justin/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar!/clojure/core.clj"]
(ins)user=> ;; note that above you can see the path, plus the identifier of a resource inside that file
(ins)user=> (def source-code (slurp *1))
#'user/source-code
(ins)user=> (println (subs source-code 10000, 10200))
       m (if (map? (last fdecl))
                  (conj m (last fdecl))
                  m)
              fdecl (if (map? (last fdecl))
                      (butlast fdecl)
                      fd
nil
(ins)user=> ;; source-code contains the full text source of clojure.core
so features you need tooling for in other languages (finding the source for some library etc.) can be done (somewhat tediously) in clojure's repl

noisesmith19:12:38

I've often solved strange bugs by verifying the contents of what was packaged in some artifact

noisesmith19:12:06

(which sometimes doesn't match what I'm seeing in git or my local filesystem for for complex reasons...)

andarp19:12:20

Hah, that's beautiful! Also reminded me to start using the * repl shortcuts. I keep defing stuff just to use it on the next line...

noisesmith19:12:02

also, any good programmer's editor can open a jar file as if it were a directory for exploring

noisesmith19:12:43

it's a hobby horse of mine that clojure's tooling is not as needed as tooling for most languages, and that there's an advantage to learning the clojure introspection facilities that a lot of tooling is built on (especially when brittle tooling fails and you need to know what's actually happening)

practicalli-john13:12:06

@U01GZD00HA8 I have online books and videos that cover the use of core tooling, especially using Clojure CLI tools which has helped me understand the REPL more as well as the ecosystem. There is still lots of content in that area to add too. I've learnt a lot about deployment, specifically when deploying web applications https://practicalli.github.io/

andarp14:12:48

Thanks a lot @U05254DQM - will check out!

holymackerels08:12:11

a few random questions: 1) does (last (take 1000 (some-infinite-lazy-seq)) end up being the same as (nth some-infinite-lazy-seq 1000) because of the laziness? or do the extra 999 from take occupy some space/time? 2) if I annotate values with type metadata, like ^long, do I need to annotate each fn that those values will travel through in order to take advantage of any potential performance gains? or just the place where they initially come into be? like if I annotate the return from a fn, do I also need to annotate functions where I'm passing that result or is just one place enough? 3) is there an idiomatic value to use for the placeholder in as->? I've been using _ though I see a lot of $. I'll probably keep using _ because I like it, but just wondering if there's convention there

futurile11:12:38

3) replacing unused function/collection elements with _ is idiomatic. I see people use $ a lot for as->. I can't see anything in https://guide.clojure.style/

futurile11:12:30

I don't think anyone will come after you with pitch forks either way!

simongray11:12:54

Yeah, like @UV1JWR18U says, _ would indicate that something that is unused, so other people reading your code might get confused.

simongray11:12:04

I just use $, personally

andarp11:12:04

Regarding 1, while far from comprehensive, this should give you an indication:

(dotimes [_ 5] (time (nth (repeat 1) 1000)))
"Elapsed time: 0.095773 msecs"
"Elapsed time: 0.069648 msecs"
"Elapsed time: 0.071448 msecs"
"Elapsed time: 0.068614 msecs"
"Elapsed time: 0.076651 msecs"
=> nil
(dotimes [_ 5] (time (last (take 1000 (repeat 1)))))
"Elapsed time: 0.243615 msecs"
"Elapsed time: 0.217264 msecs"
"Elapsed time: 0.208162 msecs"
"Elapsed time: 0.213417 msecs"
"Elapsed time: 0.206169 msecs"
=> nil

andarp11:12:53

I'm not sure how nth works with lazy sequences, and I'm a Clojure beginner. But I would assume that last+take both have to traverse the elements in some way. nth should only have to do that once in this scenario.

andarp11:12:41

If you expand the example to take the 10,000nd element the performance difference is a lot bigger in favour of nth.

andarp12:12:47

Regarding 2: “Once a type hint has been placed on an identifier or expression, the compiler will try to resolve any calls to methods thereupon at compile time. In addition, the compiler will track the use of any return values and infer types for their use and so on, so very few hints are needed to get a fully compile-time resolved series of calls.” https://clojure.org/reference/java_interop#Java%20Interop-Type%20Hints

alexmiller13:12:06

1 - yes they take space to allocate and gc as the take seq nodes (which nth will not) 2 - it depends. If the value is passing through multiple functions, each arg has to be marked as long or it will be converted to a boxed value 3 - don’t use _, but otherwise doesn’t matter

holymackerels22:12:52

:thumbsup: thanks!

Janne Sauvala10:12:48

Is there a way to update/replace value in lazy seq? All my attempts have failed to complains that seq is not an associative structure. When I change my lazy seq to a vector then I can do it with (assoc my-seq 1 new-element)

valerauko10:12:40

I don't know of a single function for this, but it can be achieved with take and nthrest:

(defn update-seq [target i elem]
  (let [pre (take i target)
        post (nthrest target (inc i))]
    (concat pre [elem] post)))

valerauko10:12:15

user=> (take 10 (update-seq (range) 5 "hello"))
(0 1 2 3 4 "hello" 6 7 8 9)

Janne Sauvala10:12:32

Thanks, Balint. I wonder is there some reason why core functions doesn’t support this out of the box

jumar11:12:51

There's replace if you really to to "update/replace value"

alexmiller13:12:52

Replacing nth value in an immutable list/seq is a slow (linear) op so there is no function for it.

👍 3
alexmiller13:12:30

Generally we try to think in terms of whole seq modifications, using map etc anyways

Janne Sauvala13:12:19

Thanks Alex, that makes sense

Janne Sauvala16:12:56

@U06BE1L6T I tested this but realised that it will replace elements by comparing the actual value but not by index. So if I have several elements with the same value I’ll be replacing them all

alexmiller16:12:42

I think generally if you get to the point of asking this question, you are probably going about something the wrong way

Janne Sauvala17:12:03

Agree. I ran into these situations while I’m solving advent of code problems. Now the easiest fix for the solution would have required me to just replace an element in a lazy-seq. I haven’t run into this before and wanted to confirm if this is even possible 🙂

Frederik11:12:44

Leiningen question: using lein uberjar to create a standalone jar for deployment, but for some reason it's taking ages. It takes 20min to end up with an uberjar of only 140M (So not exactly massive?). Any directions on how to improve this? Should I look into leiningen, or are jvm options to blame, or .... ? I have no proper Java experience, so often feel in the dark with these kind of Clojure ecosystem problems.

jumar11:12:41

Are you sure you aren't doing any "real work" when compiling? In particular, if you use AOT compilation all the top level vars are evaluated. Sometimes this might mean that connections to external resources (like DBs) are established and so on.

Frederik12:12:41

No top level vars in my code. Also, the class generation part and the non-standalone jar are created in ok times, but the standalone one takes ages. Does this mean a dependency is the culprit or are there other potential reasons?

alexmiller13:12:17

One possible reason is you are including the uberjar in itself

Tim Robinson12:12:05

Hi all, I've written a function called map-in that lets you transform a single element of a (possibly) nested structure like this: (defn map-in [m ks f] (assoc-in m ks (f (get-in m ks)))) so let's say I want to increment the age of the first person in the list: ==>(map-in [{:name "John" :age 35},{:name "Sue" :age 29}] [0 :age] inc) [{:name "John", :age 36} {:name "Sue", :age 29}] Thing is, I don't want to invent the wheel and I'm sure this must be already in core or some common library, can anyone point me at it?

flowthing12:12:39

Vectors are associative, so if you know that m is a vector, you can simply use update-in:

user=> (update-in [{:name "John" :age 35},{:name "Sue" :age 29}] [0 :age] inc)
[{:name "John", :age 36} {:name "Sue", :age 29}]

flowthing12:12:48

> if you know that m is a vector Which I guess you're already relying on since you use get-in in your implementation.

Tim Robinson12:12:14

Yes thanks update-in is exactly what I was looking for but just couldn't find the right words to search for it. actually m will usually be a map but update-in works with both

bigos12:12:46

How do I create a clojure app if clojure -X:new create :name myname/myapp gives me this error? Unqualified function can't be resolved: create

flowthing12:12:53

Which version of the clojure CLI tool do you have installed? You can use clojure --help to check. That command should also work.

bigos12:12:56

clj -X:new :template app :name myname/myapp is the answer

bigos15:12:45

where I can find documentation for running main function? https://github.com/bigos/report-generation/blob/main/src/cls/report_generation.clj I get all sorts of errors when I try examples found on the web

bigos15:12:42

I would never guess reading the output of clj --help

bigos15:12:00

thank you very much, I will try now]

bigos15:12:38

clj -M -m cls.report-generation Syntax error compiling at (cls/report_generation.clj:12:3). Unable to resolve symbol: simple-body-page in this context Full report at: /tmp/clojure-2027952924787189784.edn

clyfe15:12:59

There's a bug, move the routes after the 2 next fns used.

bigos15:12:11

still the same problem

bigos15:12:39

thank you very much for helping me with this frustrating problem

clyfe15:12:52

ah, I see now routes are twice there, just remove the first defroutes

bigos15:12:06

Hurray, the browser says hello world. thank you for your patience and help!

👍 3
Tim Robinson18:12:29

try using some kind of linter that integrates with your editor. I found it difficult to set up but once it's working it's so much easier to find problems

andarp19:12:02

Sorry to ping you directly @alexmiller, but I'm curious about Clojure Applied. Is it still mostly relevant and useful stuff, or have parts of it gotten outdated enough that you'd recommend something else? Or just some complementary reading? It's hard to know how much five years really is for a book when you're new to a community. And I figured you'd be the best source to ask here...

alexmiller19:12:21

I'd say it's still 80% relevant

alexmiller19:12:43

I am starting to look at 2nd edition, but it's likely to be at least a year before such a thing exists

andarp19:12:54

Great! I'll read it with that in mind. Thanks a lot for answering. I'm on a Clojure reading binge at the moment (as most of my current free time is reading in bed while my daughter is sleeping, can't actually program there...).

alexmiller19:12:11

we call that "hammock time" :)

andarp19:12:20

Yes, apart from reading I'm also looking at the massive backlog of talks, Hammock Driven Development being the one I just started.

andarp19:12:17

Surprisingly I've found that reading about or formulating a problem early in the day and then having it simmer in my mind while I work, pick up groceries and so on usually lets me discard at least one or two false starts that would waste a few hours of coding...

Michael Stokley20:12:15

i find that some of the pretty printing doesn't return a value that can be read/evaluated. specifically, vectors become lists with a literal (). i often want to replace the data with a transformed version, and iterate on that. are there strategies or other pretty printing folks know of?

noisesmith21:12:28

> vectors become lists with a literal () this is not true, you are changing your data type somewhere or are misusing the term "vector"

alexmiller20:12:28

if you quote those, you can read them as data

Michael Stokley20:12:53

with the repl... it feels like this should be or could be programmatic

alexmiller20:12:15

I don't find it to be so, depends on how you're using it

alexmiller20:12:56

if you def the result (or use *1 etc) you are still working on the data

dpsutton20:12:59

i just want to point out that your specific complaint doesn't seem true. Vectors do not become lists. (clojure.pprint/pprint [1 2]) yields [1 2] not (1 2)

alexmiller20:12:16

I'm assuming he's seeing things like (map inc [1 2 3])

alexmiller20:12:33

applying a seq function to a vector will return a seq that prints as a list

alexmiller20:12:57

it is in fact very consistent if you understand the model :)

Michael Stokley20:12:31

maybe it's as simple as switching to mapv

didibus21:12:24

The problem is, you start with a vector, but then are using sequence functions on it. Those will automatically convert the vector into a sequence and from that point on, you have a sequence which prints as a list

didibus21:12:11

Instead, you need to use vector functions, those will keep things as a vector. The downside of this is that there's a lot less of them. You have mapv and filterv and that's about it

didibus21:12:43

Another approach in those cases where you really care about the type of the collections is to use transducers instead

didibus21:12:12

There's a lot more transducer functions then there are vector functions (though still not as many as their are sequence functions)

didibus21:12:37

With transducers, you just tell it what type you want the resulting transformation to be in

🙌 3
🙏 3
didibus21:12:25

For example this:

(->> [1 2 3 4 5]
     (filter odd?)
     (map inc)
     (take 5))
Takes a vector but then uses sequence functions on it, so it'll return '(2 4 6) You can switch to transducers and using into chose your output type:
(into []
   (comp (filter odd?)
         (map inc)
         (take 5))
  [1 2 3 4 5])
Now that will return [2 4 6]

didibus21:12:35

And this is an example where you could have used filterv and mapv, but there is no takev, so ya transducers to the rescue!

Michael Stokley21:12:26

very slick, this should prove helpful to me

holymackerels22:12:29

does using into with a transducer/xform give runtime benefits over doing it like the first example:

(->> [1 2 3 4 5]
     (filter odd?)
     (map inc)
     (take 5)
     (into []))
or using vec instead of the into?

didibus22:12:11

Yes it does. Using transducers will be the fastest and use the less memory. Using sequence will be faster than using vector functions, at least for large collections, maybe not for small ones. But it will use more memory. Using vector functions will be slowest, especially for large collections, might be faster than sequence for small ones, won't be faster than transducers though.

didibus23:12:19

The way to think of this is that, with vector functions like filterv and mapv, you are looping over the whole collection at every step. So first filterv everything, then mapv everything. So you're unnecessarily looping more than once over the collection. With transducer, you are looping only once, and applying the full chain of transformation each iteration. So you loop only once of each element, and filter map and take in one go. With sequence, you will be looping only once on the collection as well, like transducer, but after each iteration, you will create a remainder sequence, the creation of this intermediate sequence after each iteration is why it is slower than transducer. But this is also what makes it lazy, as the remainder sequence won't start the next iteration immediately, it'll wait for someone to request the next element from it to do so. Finally, calling vec after a sequence will cause a second loop, so the sequence chain will loop once and filter, map and take on each iteration (creating an intermediate sequence at each step), and finally when that is done, vec will loop over the result one last time to convert the sequence back into a vector. That is one extra loop over using transducers directly.

🙏 3
holymackerels00:12:17

thanks, that was super helpful