Fork me on GitHub
Jim Newton07:10:31

I always forget where the documentation for the parameter list of functions, (fn, defn, defn-, etc) is. Its not in the of fn 😞


it is in documentation

(fn name? [params*] exprs*) (fn name? ([params*] exprs*) +)

Jim Newton07:10:07

This doesn't explain keyword arguments.


keyword arguments?

Jim Newton07:10:34

yes some functions can be called like (f a :x 100 :y 200)

Jim Newton07:10:57

there's a way to define these giving default values of x and y


it is a sequence of args

Jim Newton07:10:06

the descructuring rules for let are different from those of fn . Is this really not explained anywhere?


they are the same

Jim Newton07:10:00

no, I cannot define a function like this, for example. (fn [a & others :as all] ...)

Jim Newton07:10:24

nor can I define a function like this (fn [[[x]]] x)

Jim Newton07:10:56

they are very similar but not the same.

Jim Newton07:10:29

wow. I didn't know that one. cool.

Jim Newton07:10:37

but :as doesn't work as far as I know


(fn [& [a & other :as all]])

Jim Newton07:10:26

but not (fn [a & other :as all] (list a other all))

clojure-rte.rte-core> ((fn [a & other :as all] (list a other all)) 3)
Syntax error macroexpanding clojure.core/fn at (clojure-rte:localhost:59787(clj)*:17647:24).
(:as all) - failed: Extra input at: [:fn-tail :arity-1 :params] spec: :clojure.core.specs.alpha/param-list
a - failed: vector? at: [:fn-tail :arity-n :params] spec: :clojure.core.specs.alpha/param-list


because this form is violating a spec for arglist

Jim Newton07:10:14

exactly, the format for arglist is different than the format for destructuring. But I don't find an explanation of this difference in the documentation anywhere


note that [ and ] are not a part of arglist, it is a form that is inside of brackets

Jim Newton07:10:58

BTW, thanks for the link. I found this useful example>

Jim Newton07:10:26

exactly, and what is inside the brackets is different than destructuring rules.


(fn name? [params*] exprs*)
here params* is a list of destructuring forms representing arglist [ and ] are not included into destructuring forms

Jim Newton07:10:58

Let me try again. I can destructure with [a], [a & others] , and [a & others :as all]

Jim Newton07:10:10

but I cannot define functions those three ways.

Jim Newton07:10:49

I see what you're saying though.

Jim Newton07:10:11

you're claiming that the arglist is not destructuring, but each variable in the argument list can be replaced by a destructuring form.

Jim Newton07:10:06

and you cannot use :as to get the entire value of the argument list, only the entire value of any element of the argument list.

Jim Newton07:10:51

I suspect this is really a bug in clojure core which has simply become a feature. Imagine writing a function which allows :as in destructuring forms, but explicitly disallows it in argument lists. My vague suspicion is that this is really just an overzealous syntax checker on fn.


bold assumption

Jim Newton10:10:54

otherwise what is the motivation for intentionally making a function's argument list processor, simply a destructuror?

Jim Newton10:10:19

I agree, it is a very bold assumption on my part


> otherwise what is the motivation for intentionally making a function’s argument list processor, simply a destructuror? I guess to simplify reading and compiling processes. As an example: (fn ([x]) ([x :as y]) …) not valid because those forms of the same arity but to throw there compiler need to know about destructuring


compare with [param*] where each param can be a simple symbol or destructuring form

Jim Newton07:10:06

question about destructuring:

(defmethod rte-match :sequential
  [pattern items & {:keys [promise-disjoint]}]
  (rte-match (rte-compile pattern) items))
what is the correct way to make the recursive call to rte-match passing the given keys? I.e., I want to pass :promise-disjoint nil if :promise-disjoint nil was specifically given at the call site, but not if the value is nil simply because it's missing from the map. given that I cannot use :as all in the arglist of rte-match

Jim Newton08:10:53

I think this works.

(defmethod rte-match :sequential
  [pattern items & {:keys [promise-disjoint] :as keys}]
  (apply rte-match (rte-compile pattern) items keys))


(defmethod rte-match :sequential
  [pattern items & {:keys [promise-disjoint] :as opts}]
  (apply (partial rte-match (rte-compile pattern) items) opts))

Jim Newton08:10:44

I didn't use partial , is it necessary?


i want to test if the length of a lazy-seq is larger than some value, without exhausting the seq by using count


what’s the proper approach ?


(take n) and (count) ?


(seq (drop n l-coll))


maybe (bounded-count n coll)


bounded-count fits here. but the function itself looks twisted to reason


What do you mean by that? the implementation of bounded-count is twisted?


(defn count< [n coll]
  (< n (bounded-count n coll)))


I mean the meaning of the return value of bounded-count.


ah, I think of it as a sort of (min n actual-count)

Jim Newton10:10:59

I'm looking to name a parameter to one of my functions, and the semantics are potentially difficult for the caller to understand, but could nevertheless have a big impact on the performance of the function. The function has two modes, 1) it can internally lazily-compile the user's data structure into a form which is very fast. Unfortunately the compilation is slow. 2) it can use a semantically equivalent interpreter which is slow. #2 is better if the user intents to only do it once. But if the user intends to call the function multiple times with different arguments, it may be better to use option #1. What could I call the parameter to make this more apparent to the caller?

Jim Newton10:10:41

I was thinking of calling it :hot-spot , implying that the more you run it the faster it will become.


split into two functions with proper documentation for each “mode”

Jim Newton11:10:58

Here's the function. I don't see how splitting it into two functions solves the problem. Since the function is pretty large, the common code would be factored anyway into a common function, and there'd still need to be a hot-spot argument. Right?

Jim Newton11:10:58

(defmethod rte-match :Dfa
  [dfa items & {:keys [
                       ;; if the caller promises that never are two transitions in the Dfa
                       ;;   labeled with intersecting types, the use
                       ;;   :promise-disjoint true, in this case rte-match
                       ;;   can be more efficient and can assume that the
                       ;;   clauses can be tested in any order.  If the transitions
                       ;;   are not guaranteed disjoint, then rte-match must
                       ;;   build new type designators each one containing an and-not
                       ;;   of the previously seen types. 
                       ;; hot-spot = true -- lazily compile the type checks into Bdds
                       ;;    which is slow-going but becomes faster the more often you
                       ;;    re-use the same pattern, either because of loops in the
                       ;;    Dfa, or when the same Dfa is used to match different
                       ;;    input sequences.
                       ;; hot-spot = false -- always interpret the type checks rather
                       ;;    than converting them to Bdd.  This option is probably faster
                       ;;    if there are few loops in the Dfa, or if you only use the
                       ;;    pattern once to check a single input sequence.
  (let [state-vec (:states dfa)
        sink-states (set (dfa/find-sink-states dfa))]
    (if (empty? sink-states)
      (rte-match (dfa/extend-with-sink-state dfa) items :promise-disjoint promise-disjoint)
      (let [sink-state-id (:index (first sink-states))]
        ;; There are two possible transition functions
        ;;   slow-transition-function -- this is faster if the caller intends to match
        ;;       the pattern only once.   The pattern is matched by an interpreter,
        ;;       and it is possible that the same type predicate will be tested multiple
        ;;       times on the same candidate objects.  If one of the type predicates
        ;;       is (satisfies slow-predicate) then that slow-predicate may be called
        ;;       multiple times, resulting in poor performance, especially if the
        ;;       pattern is used to test multiple sequences.
        ;;   fast-transition-function -- this is faster if the caller intends to match
        ;;       the pattern multiple times with different input sequences.  The
        ;;       pattern is *compiled* into a form where type-designators are converted
        ;;       to Bdds thus each type check guarantees to never check the same
        ;;       type predicate multiple times, and sometimes not at all.
        (letfn [(slow-transition-function [transitions]
                  (fn [candidate sink-state-id]
                    (some (fn [[type next-state-index]]
                            (if (gns/typep candidate type)
                (fast-transition-function [transitions]
                  (dfa/optimized-transition-function transitions promise-disjoint sink-state-id))
                (transition-function [transitions]
                  (if hot-spot
                    (fast-transition-function transitions)
                    (slow-transition-function transitions)))
                (consume [state-index item]
                  (let [state-obj (state-vec state-index)]
                     ((member state-obj sink-states)
                      (reduced false))
                     (((transition-function (:transitions state-obj)) item sink-state-id))
                     (:else (reduced false)))))]
          (let [final-state (reduce consume 0 items)]
            ;; final-state may be integer desgnating the state which was
            ;;  reached on iterating successfully through the input
            ;;  sequence, items.  Or final-state may false, if the
            ;;  iteration finished without iterating through the entire
            ;;  sequence, either because we found ourselves in a
            ;;  sink-state, or we encountered a item for which no transition
            ;;  was possible.
              (= false final-state) false
              (:accepting (state-vec final-state)) ((:exit-map dfa) final-state)
              :else false)))))))


i’m not suggesting duplication of code but making two “public/interface” function for each corresponding mode (rte-match-simple, rte-match-full as an example) each should prepare and set as a default value for that argument

Jim Newton11:10:42

then we have the same question, right? that argument needs a name.


no, you can change the names of functions to describe what they are doing instead

Jim Newton11:10:07

but the functions call a common function, and pass an argument. that argument needs a name. Or did I misunderstand?


details of implementation doesn’t need to have “absolutely meaningful” names as long as they hidden from the caller and properly documented for maintainance reasons

Jim Newton10:10:01

Another question about Destructuring and keyword arguments. My experimentation shows that Destructuring works a particular way, which I like, but I don't know whether this is accidental or by design. If a function is defined as follows:

(defn foo [& {:keys [x y]
              :or {x 12 y 13}}]
  (list x y ))
then I can call the function using apply like this
(apply foo some-map}
However if I want to override some value in the map, I can insert them into the apply call site.
(apply foo :x 300 some-map)
this calls foo with x=300, rather than whatever is in some-map . This is what I'd expect as a Common Lisp user. Is it dependable?


some-map can not be a map in that case


(apply foo :x 300 {:x 200})
Execution error (IllegalArgumentException) at user/foo (REPL:8).
No value supplied for key: [:x 200]
map in clojure is a sequence of key-value pairs

Daniel Stephens11:10:45

it doesn't error if there are an even number of keys in the some-map, but it's not doing what you want

(do (defn foo [& {:keys [x y]
                  :or {x 12 y 13} :as m}]
      (println x y m))
    (apply foo :x 1 {:x 2 :y 3}))
=> 1 13 {:x 1, [:x 2] [:y 3]}
my understanding is apply treats the last arg as a sequence, and map as a sequence is a sequence of key-value pairs


> map in clojure is a sequence of key-value pairs


the fact that map with even number of keys is not throwing exception doesn’t mean the logic of function is correct)

👍 3
Jim Newton11:10:55

ahh, I'm glad I asked. This means the note that I added to fn documentation some time ago is actually wrong.

(defmethod foo :a [a b & {:keys [x y z] :as all-keys}]
  (apply foo (f a) (f b) all-keys))

Jim Newton11:10:08

So what's the correct way to call a function, re-passing the keys which were received, or overriding some of them?


do not use “keyword arguments” maybe?

(defmethod foo :a [a b {:keys [x y z] :as all-keys}]
  (foo (f a) (f b) all-keys))

Jim Newton11:10:23

I'm tempted to write my own version of apply which has more reasonable semantics with optional arguments. couldn't be so hard actually. It's just lisp.

Jim Newton11:10:27

hmmm it seems clojure doesn't allow me to list the same key multiple times in a call to foo 😞

Jim Newton11:10:54

I'd expect a call like (foo :x 100 :y 200 :x 300) cause x to be bound to 100 within foo, instead it throws an exception.

Execution error (IllegalArgumentException) at clojure-rte.rte-core/eval8637 (form-init3719502167719786565.clj:93).
Wrong number of args passed to keyword: :x


1. why 100 and not 300? 2. it should working normally


(defn foo [& {:keys [x y]
              :or {x 12 y 13}}]
  (list x y ))

Jim Newton12:10:11

I'd expect leading values to override later values.

Jim Newton12:10:57

that way when dealing with argument lists programmatically, you can simply prepend overriding values without having to copy the entire list.

Jim Newton12:10:28

That means if you have a list of the optional args like my-optional-args, I can call the function as (apply foo my-optional-args) but you would also be able to override some values like this (apply foo :x 200 my-optional-args)


I would rather prepare my arguments before the call

Jim Newton12:10:46

with my approach no preparation is necessary. The approach is used very often in Common Lisp, just another surprise that something I thought of as fundamental is different in clojure. Not wrong, I suppose, just different.


I think wrong is thinking that Common List is “more fundamental” than clojure ) you could also replace with any other pair of programming languages in the sentence above )


I think the mismatch here has to do with the fundamental reliance in Common Lisp on the concrete cons cell - that is why prepending arguments seems more "natural"

Jim Newton12:10:59

probably right. if the cons cell is primary, than adding to the beginning is efficient, and becomes commonplace.


whereas in Clojure you might use vectors instead to store arguments, and conj to the end

✔️ 3

(also note that structures in Clojure are persistent and immutable, so the distinction of "copying the entire list" doesn't really apply)

✔️ 3

The idiomatic way to override a value would be (apply foo (assoc some-map :x 300))

Jim Newton12:10:10

can someone explain what destructure does? Its is not so illuminating. neither does there seem to be a useful comment in the code where it's defined.

Lennart Buit12:10:16

Destructuring is the process of extracting values from collections. Say you have a vector of two elements, [1 2], you can use a let to destructure that in the first and second value:

(let [[a b] my-vector] ...)
Similar things can be done with maps, say you have {:a 1 :b 2} , you can destructure it like this:
(let [{:keys [a b]} my-map] ...)

Lennart Buit12:10:14

Ah nevermind, I guess you are specifically asking about the function

Lennart Buit12:10:24

I’m pretty sure there is a better page about it

Jim Newton12:10:42

I didn't see a reference to restructure in that link. Did I miss it?


I misunderstood what you asked

Jim Newton13:10:58

I was hoping it might be a programmatic interface to destructuring. In the coming weeks, I think I'm going to have to write a lambda-list parser which mimics the semantics of function lambda-lists. I was hoping the destructure function might be of help.


can’t say much why there is no documentation for destructure in clojure.core this is more like internal helper that accidentally become public (this is my opinion, don’t have any proof that this is true)

Jim Newton13:10:01

yes I think my mac spell corrected destructure -> destructuring in my OP


it is programmatic interface but in the form of helper for macro writers

Flo13:10:20 at the very end it's talking about the destructure fn: > However, in rare cases you might want to instead resolve the destructuring yourself in a macro. In this case, use the (undocumented) clojure.core/destructure function[...]


clojure.core/destructure is a function, not macro

Jim Newton13:10:37

looks like destructure produces the first operand of let. But it also gives me a sequence whose even elements are the names of the variables which the destructing will bind. That could be very useful.


you can think about it as a function to return bindings (destructure '[[x y] z]) => [gen_0 z, x (nth gen_0 0 nil), y (nth gen_0 1 nil)] ^ this is simplified form


so in one sentence it transform destructure syntax into clojure syntax


Is there a idiomatic/preferred binary tree in Clojure?


@watchtheblur sorted-map is a persistent red-black tree


Thanks @ghadi. I was looking for something that I could traverse/search children with. I realize I can implement a binary tree using a list and its indices, but I wondered if there was a better way (maybe objects)?


(use a vector and indices, lists are not for looking up by position)

👍 3

@watchtheblur on a sorted collection (eg sorted-map) the subseqfunction function can be used for efficient splitting, sadly it doesn't return a pair of new sorted collections though


@noisesmith seems like the censuses is to use a map instead of a vector for a binary tree? So if I wanted to grab the 2 children of a node, I would need to give indicies to the map to get the child, or is the map already structured as a tree (i.e. nested maps)?


there's also sorted-set, but neither directly exposes its tree structure


it requires interop, but at least it's public

Alex Miller (Clojure team)18:10:51

I can't imagine that's what OP actually wants, but that raises the question - what do you want? :) are you looking for a "by the book here's how to implement a binary tree" or are you looking to store some data in a tree for a reason, and if so there are probably better default answers


@U064X3EF3 I'm looking to practice leetcode-type questions with clojure. Since leetcode doesn't support clojure, I wanted a simple binary tree data structure to run algorithms on. I wanted to check with the community on a idiomatic approach

Alex Miller (Clojure team)18:10:16

in general, the best way to work with Clojure is to lean on the built-in data structures, so I'd probably make each node a map with slots for :left and :right (or whatever). Or a vector of 2 items. not sure which way works out better.


Got it, thanks @U064X3EF3!

Mario C.18:10:04

We have an app running in wildfly at work and it started to get unresponsive with users reporting that they are getting "BadGateway" issues. Looking at the logs I noticed quite a few errors. Main ones that stick out were "OutOfMemoryError: Metaspace", "NoClassDefFound", "I/O Exception Broken Pipe (Write failed)" and "Postgres connection attempt failed". Looking at NewRelic, I noticed that the UnloadedClass count started to increase along with a huge spike in GC (ConcurrentMarkSweep), where at the highest point, is where the application went down and users started reporting the errors. Operations restarted the servers and the app was working again. But this had happened last week as well. Currently I am researching but this is honestly a little over my head/experience level. I am thinking there is perhaps a memory leak going on but I am not so sure. Does anyone have any idea what could be causing this or could point me in the right direction?

Mario C.18:10:26

Oh, another thing, which could be coincidence, but the servers both went down on Monday/afternoonish


I don't have experience doing this myself, but there are several attach-them-to-the-running-JVM-process memory profiling tools that can aid in looking for what classes of objects are using the most memory.

Mario C.18:10:54

Yea I was trying to hook up VisualVM but was getting issues with it freezing


hprof is much lighter weight, and visualvm can load the dump file it creates


sounds like you are calling eval a lot

Mario C.18:10:53

Hmm, what makes you say that?

Mario C.18:10:14

That could be it actually, since some new code that went out about two weeks, uses the eval function


eval compiles clojure code to jvm classes, classes are allocated in metaspace


the correct amount to be calling eval is usually 0

💯 3
Clark Urzo18:10:51

If you really need to have some form of eval in your code, there’s always sci

Mario C.18:10:56

Actually, the code does not use eval. I thought it did but it does not after second look


@mario.cordova.862 are you trying to limit the metaspace? That seems to be a common mistake for folks coming from earlier JDKs where limiting the non-heap part of memory sometimes made sense (it's been a while so I've forgotten exactly what setting I'm thinking of on earlier JDKs)


(it used to be a good idea to limit PermGen -- it is not a good idea to limit Metaspace, as I recall...)


yeah, in my experience setting limits on metaspace is cargo-cult and/or mindless copying of old config from ancient jvms


it's not a useful setting

Mario C.18:10:17

I am just trying to figure out why this issue occurred. If its a problem with new code changes or something else.


something is allocating memory off heap, that's done by some libraries, it's done when you create new classes


the biggest culprits I know of are using eval to make new functions at runtime, and badly written interop with platform level stuff (eg. a memory leak in initialization of some adaptor to C code)


I guess "at runtime" is when all functions are created, but I mean in a deployed app after startup

Mario C.18:10:16

I think it might be the eval but it's odd that it just started to happen. Going to check if there is a limit on metaspace


IIRC that's the only way to get that specific error

Mario C.18:10:51

The metaspace one?


right, without the metaspace limit you get a different flavor of "vm out of memory"


and you'd need to create a lot of functions before that came up (I guess that's one benefit of a metaspace limit, catching metaspace leaks early...)


and now I wonder if there's any such thing as garbage collection of Classes (probably not...)


there is, but it is relatively new (I forget how new, or maybe if it is just a jep that hasn't landed yet)


I guess it isn't that new, java 8 or so


so something must be not only creating new functions via eval but also holding onto them I guess

Mario C.18:10:36

I am seeing this in the standalone.conf -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=2G

Mario C.18:10:00

This is setting a maxsize on metaspace, which should not have a limit?


I've never seen a drawback to just leaving it unlimited, but you probably also want to find out what's filling your metaspace and not getting cleared up


visualvm / yourkit / hprof can show you what's filling metaspace

Mario C.19:10:03

Going to investigate it then, thanks for the help guys! :thumbsup:


luckily anonymous functions / partial / reify / proxy etc. create class names that will lead you to the code creating them


Partial's is just something like clojure.core$partial$fn

Mario C.19:10:11

So eval is okay to use when "defining" something but during run-time it is an issue?


It is important to understand, and an important design feature, that clojure only creates classes during compilation, running code doesn't create new classes unless you call the compiler (calling eval)


So using any of the features @noisesmith mentioned will only generate a single class (and partial doesn't even do that, because partial doesn't really belong in that set) at compile time, and at runtime you will just get multiple instances of that class


The only way to generate an unbounded number of classes is by calling eval over and over


right, I shouldn't have included partial, but was thinking that instead of creating fns via eval, they might be creating reify or proxy


Reify and proxy will only generate new classes when compiled


right, inside eval


it would be weird but it's possible

Mario C.19:10:57

So if I do (eval '(+ 1 1)) this creates a class?


I would not try to define cases when eval is ok to use and when it isn't. I would say because clojure provides it neatly wrapped in a single function you don't realize the complexity of it, and whatever you are doing almost certainly doesn't require it


To execute byte code on the jvm it has to live in a class file


Live in a class, the class may just exist in memory


Eval has some special cases for very trivial expressions where it just acts as an interpreter and doesn't generate byte code, but in general for anything but the most trivial expressions, you will generate at least one class when calling it

Mario C.19:10:31

Im not so much trying to define cases when is it okay to use but I am trying to reason about some code. I am trying to verify that a certain portion of the code is the culprit. Since this part of the code will be calling eval during runtime where as the other portion is using eval but not during runtime


Remove the calls to eval


When there is a foot gun in your code you remove it or debate where it is pointing, and for #beginners the response has to be removing it. Debating about where it is pointed belongs somewhere else

👏 6

Hah, I know that guy

Mario C.19:10:11

I dont feel comfortable just yet removing it, since it is legacy code and its essentially the heart of the application so a little risky


If the application is written such that it can be shut down in an orderly fashion, and/or has multiple instances running in parallel that can take over for each other dynamically, e.g. like many highly available web services, you could potentially consider the workaround of monitoring the PermGen space usage, and when it reaches a threshold, force that instance to restart. That would be a workaround, mind you, with its own introduced operational difficulties.


hello, I'm writing a little tool to automate some of my daily task on Clojure and Conch (, however I have a function that I can't make pass a test. Here the code for both:

(defn exec-cmd [expression]
  (let [bin (:bin expression)
        args (:args expression)
        config (:config expression)]
    (sh/with-programs [echo mvn docker-compose]
                      (case bin
                        "echo" (echo args config)
                        "mvn" (mvn args config)
                        "docker-compose" (docker-compose args config)))))
(deftest test-echo-cmd-exec
  (testing "Execution of echo command OK"
    (is (= (exec-cmd {:bin "echo", :args '("hello" "world!"), :config {:seq true}})
           '("hello world!")))))
Any idea how can I make the test pass?, I can make the function to use echo by using apply but I need to pass extra configuration for the programs binded by Conch (that's why in the test case I'm passing a {:seq true})


by the way, the test fails with:

FAIL in (test-echo-cmd-exec) (core_test.clj:7)
Execution of echo command OK
expected: (= (exec-cmd {:bin "echo", :args (quote ("hello" "world!")), :config {:seq true}}) (quote ("hello world!")))
  actual: (not (= ("") ("hello world!")))

Ran 1 tests containing 1 assertions.
1 failures, 0 errors.
Tests failed.


@sdmoralesma ymmv but I find conch to be over engineered and I prefer to use ProcessBuilder / Process directly:

user=> (let [b (ProcessBuilder. ["echo" "hello"])
             p (.start b)
             res (.getInputStream p)]
         (.waitFor p)
         (slurp res))
*reformatted to be more readalbe


thank you for the answer, I'll give it a try, also I noticed that is possible to pass a folder for execution which helps in my case. 👍


yeah, the API is pretty nice actually, especially if you already know the posix exec api


except for the part where it doesn't let you know the PID :/


using the inputStream and outputStream together you can even run an interactive program, which is much harder to do in most languages


with the PID at least is not an issue for me... yet. About the streams, that might improve the workflow... after all the idea is to simplify my day, thanks for the hint

Alex Miller (Clojure team)23:10:58

In Java 9 they added a .pid() method

❤️ 3
👍 3