a delay will calculate the value at most once, while a future will calculate the value even if it's not needed. If you look at https://github.com/clojure/core.cache/blob/63aeed08302e2b2c81faf59adade4ead82c87969/src/main/clojure/clojure/core/cache.clj#L57, it seems that the delay is only deref 'd if there's a cache miss.

A future also caches the value, right?

Takes a body of expressions and yields a future object that will
invoke the body in another thread, and will cache the result and
return it on all subsequent calls to deref/@.

oke, thanks for clarification!

yes, a future will also only run the calculation once, but it's eager not lazy

The other difference is that futures are executed in another thread (which is the basis for allowing a timeout on them). Delays execute in the current thread when/if they’re needed (as mentioned).

@seancorfield do you use #lsp and #clj-kondo on those big projects? Do they still cope with large codebases okay?

@U01LFP3LA6P Yes, I use VS Code/Calva + Clover for the REPL and that uses LSP and clj-kondo under the hood continuously. Every now and then, LSP spins up the CPU and the Clojure features in the editor "hang" for a few seconds but it's fairly rare (maybe once a day?) and I think the LSP team are tracking down a few possible causes.

The benefits of LSP/clj-kondo far outweigh the inconvenience of occasional "hangs" like that in my opinion.

I'm asking because on one of my customers' project (classical stack of java, spring, hibernate etc.), IntelliJ IDEA is often on the edge (or behind) of usability. I often find myself waiting for Idea to keep up with my code changes. For example Idea is showing some syntax/compile error in my code, I fix it.. and it takes up to 10-15 seconds before Idea finally figures out that the code is now valid. :man-facepalming: That's on project with ~7000 java files, 1000 of which are generated. So I'm wondering how usable large clj(s) project are with calva/kondo/lsp. Good to know that it's usable :thumbsup: .

Btw yeah, I know about one of the "CPU spinners" as well. But hopefully those minor glitches will be solved soon. https://github.com/BetterThanTomorrow/calva/issues/1250

Well, I don't know how VS Code would deal with such a large Java project -- I have the Java extensions installed but rarely work on any Java code -- but of course Clojure projects are always going to be smaller than their Java equivalent 🙂

Not just smaller.. not just smaller 🙂

That ticket doesn't apply to my setup because I'm not using Calva/nREPL for evaluation. And I keep Clover's REPL output trimmed when I'm working (and mostly hidden). I use Reveal to visualize all my results.

Yeah, I'll probably look into possibility of similar setup myself. Not sure if there's need of Clover in order to be able to do that (I hope not, but am afraid that it is). I like the idea of to using either reveal or portal to output every result of evaluation, but didn't have time to try it yet.

There's middleware for nREPL I believe that tap>'s every result but we don't use nREPL at work, just Socket REPLs.

Yeah that's what I suspected. With Ruby/Node.js I was fine with a free AWS micro instance with 1GB, so this is a huge change.

You can run clojure app on that - at least medium sized web app that doesn’t do heavy processing

@jakub.stastny.pt_serv To be clear, that node configuration is for the app running in a production service that serves several thousand requests per second.

And they're over-provisioned to absorb large intra-day swings in traffic.

I recall having some issues with this as well, but it was many years ago now. In principle, https://clojuredocs.org/clojure.spec.alpha/*recursion-limit* should help, but I seem to remember that it doesn't always. I think in my case I ended up using s/gen to attach my own generator, written using gen/recursive-gen: https://github.com/clojure/test.check/blob/master/doc/intro.md#recursive-generators

thanks @U7ZL911B3, gen/recursive-gen was what I needed. I was confused because this process is automated, eg, in Haskell, and I was trying to understand if someone had an automatic way of doing it

https://clojurians.slack.com/archives/C03S1KBA2/p1628263100420500?thread_ts=1628260780.412300&cid=C03S1KBA2

it is macOS bigsur.

which JVM?

[email protected] Repos % java -version
java -version
openjdk version "11.0.2" 2019-01-15
OpenJDK Runtime Environment 18.9 (build 11.0.2+9)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+9, mixed mode)

ok, then it should work. Could you try latest version of clj-async-profiler? https://github.com/clojure-goes-fast/clj-async-profiler from here it is 0.5.1

the reason I asked about OS and JVM is that it wont work on Windows

OK, I updated my project.clj to have the line [com.clojure-goes-fast/clj-async-profiler "0.5.1"] , and restarted my repl. I still get what appears to me to be the same error.

haven't verified every line.

is there a way to ask at the repl, which version of a dependency was loaded?

you can use lein deps :tree if you are using lein to start the repl

but not from the repl

you need to allow attachment

see here https://github.com/clojure-goes-fast/clj-async-profiler#jvm-options

> JDK9+: you must start the JVM with option `-Djdk.attach.allowAttachSelf`, otherwise the agent will not be able to dynamically attach to the running process

The exception is Can not attach to current VM, that's likely the cause

also > Ensure that `perf_event_paranoid` is set to 1 or less

last one is for linux, should not be a problem on mac os

yeah but -Djdk.attach.allowAttachSelf is critical

great! that seems to work well. It's amazing which random things you people know 🙂

At this point I probably went through all the JVM profiling tools, feel free to ask weird questions

besides yourkit

Hi Ben, do you have much experience looking at flame graphs. I'm trying to look at one and understand what it's telling me. or at least what it I should infer to make my program faster.

This is just a sample. I have not yet investigated whether it is a typical run. Is it telling me the number of times functions are called, or is it telling me the amount of time spent in functions? it's call cpu-flamegraph so I'd guess it probably is trying to imply something about total cpu time.

one thing I notice about the flame graph created by clj-async-profiler is that it is difficult to understand what is happening (profiling-wise) when a multi-function gets called. i.e., how much time is spent in any of the various multi-methods.

there is a trick I used to identify which method of defmulti been called

(defmulti foo identity)

(defmethod foo 1 one [_] "one")

(defmethod foo 2 two [_]
  (throw (ex-info "From two" {})))

(foo 1) ;; => "one"

(foo 2) ;; => Exception with :at [spapi.client$eval6527$two__6528 invoke "client.clj" 331] in stacktrace

the idea is the same as having the “name” for anonymouse function defined with fn

(fn foo [x] (+ x 1))

ahhh, that's clever!

not ideal thou, because the name will be decorated with generated suffix ( so it will take some time to get used reading traces

Okay, flame graphs and you, and short intro

Flame graphs are two dimensional representation of % of time spent doing something on the X axis, and how deep you are in the stack (and where) on the Y axis

Thus it is a sort of histogram

Think of an entire stack as a bin, and stacks with similar roots are joined

thus the stack X->Y->Z: 3 X->Y->W: 2 Will be joined and displayed as: X->Y (5) -> {Z(3), W(2)}

What you see and how to read it

In clj-async-profiler, stack traces related to Clojure are in pretty Clojure colors, i.e blue and green

Native stuff is in red:

In unappealing yellow you'll see the JIT compiler and GC

looking at your graph, it's mostly compiler work

This means you need to let the JIT have more time to work

Assuming you profiled you function like so: (profile my-expr) let the JVM burn some CPU getting to know it better: (dotimes [_ 10] my-expr) Then try profiling again Also, I usually use dotimes when profiling to get more samples for more accurate results

Reading the actual results: I start by clicking the base of the pyramid of Clojure colored stacks, on java.lang.Thread.run, which zooms me in to that stack

From looking at these stacks, it looks like you ran your expression only once

Am I correct?

I suggest letting the code warm up then rerunning with plenty of samples

yes ran the expression only once.

indeed

but it worked over a pretty large data structure which contains lots of redundant code paths. at least that was my intent

BTW about the flame graph. If the same function is called from two completely different call trees, will it appear once or multiple times in the graph? I think it will appear multiple times, once per call-tree. that is of course sometimes what you want. but sometimes, you just want to see the functions independent of their call trees.

Multiple times

Joining is from the root of the call tree

From looking at the flame graph the data is insufficient. Let it run for at least one minute

BTW I was wrong about the create-or function. Actually I have two functions of the same name in two different name spaces. it was the other one which apparently appears in the flame graph. 😞

but that other one is very similar to this one, it works on a different type of data structure, but its structure is very similar.

(def rte/create-or
  (fn [operands]
    (cond (empty? operands)
          :empty-set

          (= 1 (count operands))
          (first operands)

          :else
          (cons :or operands))))

This will probably be useful in profiling the expression for an arbitrary duration:

(defmacro profile-for
  [ms & body]
  `(let [a# (atom true)
         p# (future (prof/profile (while @a# [email protected])))]
     (Thread/sleep ~ms)
     (reset! a# false)
     @p#))

is there something funny about profiling a test case using clojure.test? I mean does the test driver artificially re-trigger the compiler?

Don't think so, but it's one-shot. Profiling is empirical, you need lots of samples

yes I ran the test case 1000 times (previously only 1 time) and I still see lots of compiler overhead.

maybe I’m reading the graph wrong. let me study it a bit more before complaining too much

I just do it interactively

I also don't know how you run the tests

but generally, I wouldn't tie profiling and benchmarking to my tests, too different

Here is what I’m typing into the repl:

clojure-rte.rte-core> (prof/start {})
"Started [cpu] profiling\n"
clojure-rte.rte-core> (doseq [r (range 1000)]
                        (clojure.test/test-vars [#'clojure-rte.rte-canonicalize/t-canonicalize-pattern-714]))
[:count 100]
[:count 200]
[:count 300]
[:count 400]
[:count 500]
[:count 600]
[:count 700]
[:count 800]
[:count 900]
[:count 1000]
nil
clojure-rte.rte-core> (prof/stop {})
#object[java.io.File 0x64837b2 "/tmp/clj-async-profiler/results/05-cpu-flamegraph-2021-08-09-14-03-04.svg"]
clojure-rte.rte-core> 

(clojure.test/test-vars …) calls a test case by name

right, let me poke at it

I could of course copy the code out of the test into a normal function and see if the result is the same.

you don't even have to use testing

(prof/profile (dotimes [_ 1e3] (clojure-rte.rte-canonicalize/t-canonicalize-pattern-714)))

tests are just functions with extra metadata

Are there some fixtures you need to run?

by fixtures do you mean setup and tear down?

yes

no. hey, when I copy the test code into just the repl, I see very different results from the profiler. bizare.

I wasn’t expecting that difference

by the way, how long does profiling take? Should be at least 30 seconds

no, its about 10 secs

I can up it by 4.

Thanks. Yeah I'm hoping I could make it work on 1GB RAM on a free AWS micro instance.

Could you consider using ClojureScript? Node is less hungry for memory

What I said in the other thread- you should be able to do that.

@U6N4HSMFW this is not simple as "x is better than Y" Both #graalvm native-image tool and node can have faster startup times and less initial memory footprint But if you are processing a lot of data, JVM will be faster and consume less memory than the other alternatives, in the most cases. for sure you can craft a problem where @jakub.stastny.pt_serv to run in smaller machines, I recommend you to create an AOT Uberjar to have better startup times. Also, avoid "huge horizontal scaling". As JVM uses JIT and Clojure do structural sharing, 2x 4Gb machines will perform way better than 8x 1Gb machines. But for sure, 2x 1Gb machines is safer than 1x 2Gb machines

Sure it depends on what kind of work the app is doing among many other things. However in my experience node is less hassle on machines with limited amount of memory compared to jwm .

Damn 24GB is quite a lot, when GC happens it must be quite the GC.

not necessarily, GCs happen quite regularly. there are 'minor' and 'major' ones will depend on the GC of choice and its params of course

That was going to be my follow up question, which GC are you using? Because I'm pretty sure when I'm using G1 and set the memory tad too high, then the GC pause is very annoying.

G1 and these settings https://github.com/reducecombine/.lein/blob/00468aefbfdfb3b4c9f4eec44a318dc3c3db377e/profiles.clj#L53-L57 don't recall a GC STW pause over the 2-3 years I've used it still I'm looking forward to try out the newer GCs :) promising sub-ms GC

Nice, I guess tuning the params matter as much.

Going to try these settings out

It did.

That is just a 'pretty-printed' version of the same data

type {:a/b "foo"} in your repl

#:a{:b "foo"} is shorthand for "every key in the map has a ns of as"

I see, thank you

You can (set! *print-namespace-maps* false) to disable this btw

Thanks!

You can use https://github.com/circleci/bond to spy on a fn. This example shows the spying on a fn and getting count for fn calls.

(ns test.foo
  (:require [bond.james :as bond :refer [with-spy]]))

(defn foo [x]
  (let [shaken (with-out-str (prn :martini))]
    [shaken])

(defn bar [y]
  (foo y))

(deftest foo-is-called
  (with-spy [foo]
    (bar 2)
    (is (= 1 (-> foo bond/calls count)))))

I'd rather keep it vanilla clojure for now. Thank you though I'm sure that will be useful in the future.

Ya I was going to recommend Bond as well. You can do something with with-redef, which would basically be re-implementing a lesser variant of Bond. But I'd say using an existing library would be better. ALTERNATIVELY I will say don't bother unit testing this. If b is pure, test b and test call-b. If b is impure, write an integ test

Does bond spy and mock? because I don't want foo to be mocked as a no-op. Only interested in what my code is calling it with.

If you're curious about with-redef, I've done this before:

(deftest call-b
  (let [b-spy (atom :not-called)
        orig-b b]
    (with-redefs
      [b (fn [& args]
           (reset! b-spy args)
           (apply orig-b args))]
      (is (do (call-b 123)
              (= [123] @b-spy))))

There are two main offerings of bond - 1. Spy - this will just spy on target function and won’t interfere with function definition. 2. Stub (mock) - you can use it to redefine a function. Additionally This also provides spy benefits

Bond uses with-redefs internally to implement these functionalities. If you want to stick with Vanilla clojure. You will have to repeat the effort.

Actually Bond had a commit a while back where it no longer use with-redefs, it uses something that is thread local and thread safe instead, so it's even better

There is one major drawback of with-redefs. It redefines the function globally. This means that another test running in parallel will be able to see the new definition. To solve this problem, Helpshift has added new functions over bond to provide local-stub and local-spy . These fns will be mocked in the current thread only and this redefinition wouldn;t interfere with another thread. https://github.com/helpshift/bond

https://clojurians.slack.com/archives/C03S1KBA2/p1628281124460900?thread_ts=1628279527.444400&cid=C03S1KBA2 I think we are talking about the same thing here.

Oh haha, ya I'm talking about: https://medium.com/helpshift-engineering/a-study-in-parallelising-tests-b5253817beae#:~:text=Although%20it%20is%20a%20very,is%20visible%20globally%20across%20threads. Wait so I thought it had been merged upstream, did it not?

Unfortunately the bond repo maintainer decided to not merge the request for code complexity/maintenance reasons. Checkout this https://github.com/circleci/bond/pull/47

Ah, too bad. I'll keep an eye out for your fork.

aside, I'd be suspicious of tests that make assertions about implementation details of the thing tested (did some f get called) as opposed to testing input / output values. in functional code you can reify the selection of function / arg as its own function (whose input / output you can test) if that's a property that's worth testing

There's likely a typo in your ns form, feel free to post it if the error persists

(ns hotline.core (:require [twilio.core :as twilio]) (:gen-class))

reading the error message more carefully, it might be a syntax error in their lib. It's not that rare; the ns form rules were strengthened in recent Clojure versions I'd check twilio/core.clj and raise an issue etc In the meantime you can disable spec-checking at macroexpansion time

How do I disable spec-checking ?

https://clojure.org/releases/devchangelog#_1_9_0_beta3_oct_25_2017

I almost pulled up the quote I saved from the clojure discourse of yours 🙂

I second the sentiment but never mixing pure and impure: is badly formulated. It would be unsustainable for an impure function not to be able to use inc or assoc Alt formulation: decouple pure from impure parts whenever possible

The "whenever possible" is to the discretion of the developer. But I don't think it's that unsustainable to do so. Why would you need to do inc or assoc in an impure function? I can see assoc maybe like to construct the return value, but inc I can't see why.

Perhaps moving on from a less fruitful discussion, is this a pattern you've used at scale? Especially considering never which is a quite unmistakable rule. As mentioned I like the overall intent but at scale, in a team it can predictably fall under its own weight. Especially when no tooling exists (e.g. a framework for functional architectures, or a linter for side-effect segregation)

I get that "never" is too strong, we probably try to follow it as much as we can, and call it out on CRs when we spot it not followed. But generally I'd say we've done this at scale. The only difference is that you end up needing more than a single giant top level orchestrator function, you can create sub-orchestrators, this does make things clearer. But its trying to split between: 1. Only IO 2. Only pure 3. Only orchestration I have to explain 3 a bit, but basically 3 ends up just as a way to connect the input/ouput of other functions. So say you have a business process that's like modify the input somehow, then query the db for something, and using what you queried finish modifying the input to return a result. You have pure -> impure -> pure. You'd have a function that's literally:

(defn foo-process
  [input]
  (-> (pure/logic input)
      (impure/io)
      (pure/more-logic input)))

(defn -main
  [input]
  (let [foo (foo-process input)
        bar (bar-process foo)]
    {:a (:result bar} :b (:result foo)}))

I wrote an example of this recently here: https://gist.github.com/didibus/ab6e15c83ef961e0b7171a2fa2fe925d Its a fake example, but I was trying to show a non-trivial use case. Now the process-bar would be top-level API, in this case I'd normally break it down into some sub-process, and also the "retry mechanism" I am using in real code would be hidden behind a macro (which you can find here: https://stackoverflow.com/a/68515868/172272), but still in my opinion, it is all quite readable, though feedback welcome 😄

so yeah we think pretty similarly :) we agree this is a best-effort thing Normally in a code review (at least in a few teams) indeed we'd suggest to decouple IO, so that one could have a unit test and a more heavy test. I've never gone as far as having a concrete notion/architecture of 'orchestrators'. I'd say that the main difficulty lies in: • conveyance - how do you know a given defn is pure / impure? It's rarely as easy as eyeing one defn - defns can call each other, recursively. naming conventions can help but have their limits • enforcement - how do you know the codebase-wide segregation effort is veridic? Without enforcement there could be hidden impure corners which could foil the whole effort / might be hard to revert etc

Perhaps an interesting read: https://github.com/clj-kondo/clj-kondo/pull/751 we left the effort as there were some pretty undecidable cases

If I were to go the linter route, I.dont think I'd use the shebang, since it doesn't really indicate side-effect, more like warns you of caveats, which aren't always side-effectful. I think I'd go for a new meta tag, like ^:pure and assume all things not tagged with it are impure. And I'd have the linter make sure a ^:pure tagged function isn't using anything not tagged ^:pure. And I'd let people configure things to be pure in a config as well if you want to backfill libraries like clojure.core. But I'd also have it work like type hints:

(defn foo
  ^:pure [a b]
  ^:pure (+ a b))

Also interesting to look at ghostwheel: https://github.com/gnl/ghostwheel