Fork me on GitHub
#beginners
<
2022-07-25
>
xbrln08:07:35

What is the difference between lein install and lein deps ? Am asking to find out which one should I run in my CI/CD test pipeline before running lein test This is my CI/CD pipeline for test,

test:
  stage: test
  image: 
    name: clojure:lein-2.9.8-alpine
  script:
    - lein install
    - lein test

delaguardo08:07:00

lein deps download and examine dependencies lein install is for installing jar and pom files generated from your project into local maven repository. As a side effect lein install will download dependencies as well as lein deps but it also does unnecessary (at least for CI/CD) jobs.

delaguardo08:07:06

btw. lein help %command% will give you better explanation

delaguardo08:07:11

also in most cases you don’t need any command before running lein test

xbrln08:07:26

thanks a lot @U04V4KLKC 🙂

lispyclouds09:07:45

one place you can use lein deps is to download the dependencies and let the CI cache them. Next time if the dependencies don't change the downloads wont happen again. But I'd also do this if and when the download takes longer. Often its a premature optimisation 🙂

xbrln09:07:26

I was thinking of a way to cache dependencies, since lein test runs on every push to remote and dependencies are not changing that often. Will try lein test after lein deps and see if it reduce the time it takes to run tests. Thank you for the tip @U7ERLH6JX

lispyclouds09:07:47

so the general flow would be • expand cache • lein deps • cache ~/.m2 path using the hash of project.clj as the key • other lein commands ...

sarna09:07:51

how to type hint rest arguments? (defn foo [& ^bytes byte-arrays] ... - is that correct? update: okay, [^bytes & byte-arrays] seems to work. nevermind :)

delaguardo09:07:37

that doesn’t work. [^bytes & byte-array] will add typehint for symbol & and will have no affect later in the code.

(set! *warn-on-reflection* true)

(defn foo [^bytes & bs]
  (String. bs))
this snippet will give you a reflection warning
(defn foo [& bs]
  (let [^bytes bs bs]
    (String. bs)))
so to make it right you can add a let expression (as in the snippet above ^ ) with proper type hint

sarna09:07:33

oh damn haha. I didn't get a warning in the repl so I assumed it's okay. thank you!!

sheluchin12:07:47

Sometimes my JVM seems to be working very hard and I'm not doing anything in particular - haven't executed anything in the REPL recently and didn't start any long-running tasks. Is there some way to see what it's up to, or is a simple JVM restart the best approach if it only happens on rare occasion?

Ed12:07:40

you can attach something like https://visualvm.github.io/ (which depending on the version you are using may be bundled with the jvm). It'll let you monitor the gc process (the usual suspect) and which threads are running. If you plan on running the application in production, it may be useful to learn how to debug these sorts of things, but if it's just an intermittent problem on your machine, restarting the jvm is often way easier 😉

sheluchin12:07:37

@U01AKQSDJUX thanks, I've seen that around but haven't tried it yet. Got it up and running now. Looks like a good tool to get familiar with.

Ed12:07:59

You often have to pass some options to the jvm when you start it to allow visual vm to attach to it and collect data. You can also send it signals to get it to spit out a thread dump (basically a bunch of stack traces and thread statuses) or a heap dump (which you can analyse with tools like https://www.eclipse.org/mat/ ) to work out what's taking up all your memory 😉 ... and if you want track what the gc process is upto, you can enable gc logging ... but visual vm is a great place to start and lets you track this data over time and draw some heap usage graphs and so on 😉

sheluchin13:07:39

It didn't come bundled with my JVM but I also didn't have to start the JVM with any extra flags, maybe because :jvm-opts ["-Djdk.attach.allowAttachSelf"] is all that's needed and I already have it for the clj-async-profiler to work. I'll check out MAT and your pointers are a good starting point to do some more reading about this.

👍 1
plexus14:07:54

it's also good to know about jcmd and jstack, the first lists all the running JVMs with their process id, the second prints out the stack traces of all threads based on a process id. If a JVM is really starting to eat up all the CPU then it might not be feasible to still start visualvm and attach it, but jcmd/jstack should still work and can give you a rough idea of what it's doing.

plexus14:07:17

also super useful for the opposite problem: my JVM seems to be stuck, what is it blocking on?

sheluchin15:07:09

@U07FP7QJ0 Good point, there might not be enough CPU left to start up a heavy inspection tool. Thank you for mentioning those alternatives!

Ben Lieberman17:07:16

ok so I have this ~/.m2/repository/org/clojure/core.async/1.5.648/core.async-1.5.648.jar in my classpath but when I run the REPL and do

user=>(:require [clojure.core.async])
I get a ClassNotFoundException

phronmophobic17:07:24

that should probably be:

(require 'clojure.core.async)

Ben Lieberman17:07:18

what's the difference between :require and require, I know the former is a keyword but when do I need that and when not

phronmophobic17:07:25

the (:require [clojure.core.async]) form is used when it's part of an namespace declaration. For example:

(ns my.ns
   (:require [cljoure.core.async]))

phronmophobic17:07:17

so the short answer is :require when it's inside of an (ns ...) expression, and (require ...) otherwise

Ben Lieberman17:07:42

thanks @U7RJTCH6J I got it working

👍 1
NoahTheDuke17:07:47

How do I use a library/dependency from the command line when using clojure cli? Never mind, answered!

dpsutton17:07:49

❯ clj -Sdeps '{:deps {org.clojure/core.match {:mvn/version "1.0.0"}}}'
Clojure 1.11.1
user=> (require 'clojure.core.match)
nil
user=>

NoahTheDuke17:07:31

ah, -Sdeps. i see now where that is on the Deps CLI page. I think that needs a couple more examples. Thanks so much

bortexz20:07:02

Does anybody know if delay ’s body resources can be garbage collected after the delay body is finished executed and the result is cached? I have a use case for ‘chained’ delay’s where a delay forces the previous delay, and was wondering if this could potentially create a memory leak if I only keep around the latest delay (whose body refer’s to the previous one, but would be the only reference)

dpsutton20:07:34

I’m not expert on GC related issues but if you have an explicit reference to essentially a linked list of delays I doubt they will be reclaimed/collected

ghadi21:07:53

the function that a delay delays is collected after it is invoked. docstring says: > will cache the result and return it on all subsequent force > calls. if you are hanging onto a realized delay, what it caches is strongly reachable

ghadi21:07:04

if you are hanging onto a realized delay, what it caches is strongly reachable

ghadi21:07:23

not eligible for garbage collection

bortexz21:07:23

thanks @U050ECB92 , I think it is fine for my use case then, as the reference to the delay is inside the function of another’s delay (and nowhere else)

bortexz21:07:16

hmm doing a quick test, I thought this wouldn’t increase the memory much, but it does (looking at VisualVM), and once is resetted to nil then it goes down (by performing GC)

bortexz21:07:15

(comment
  (def state_ (atom nil))
  (dotimes [_ 10000000]
    (swap! state_ (fn [d]
                    (delay
                     (force d)
                     (range 1e10)))))
  (reset! state_ nil)
  
  )

bortexz21:07:48

every ‘spike’ is an execution of the dotimes expr

phronmophobic21:07:06

Did you confirm the garbage collector was run at any point?

bortexz21:07:27

Yes, I manually clicked ‘Perform GC’ and memory was not going down

bortexz21:07:05

By adding a force to outside the delay though, it seems to not create this effect and memory doesn’t increase

(dotimes [_ 10000000]
    (force (swap! state_ (fn [d]
                           (delay
                            (force d)
                            (range 1e10))))))

bortexz21:07:11

Ah, I think I see the problem in my first example, I was never forcing any of the delays, so the memory increase might be because of all the chained non-realized delay fns

👍 1
1