Fork me on GitHub
#clojure
<
2021-01-21
>
pinkfrog00:01:39

Hi. Is checking against the arity a runtime or compile time thing? For example, for (defn f [a b c]), calling (apply f ’(1 2)) will incur Execution error (ArityException)

andy.fingerhut00:01:03

it is checked at run time. Some linters like Eastwood and probably also clj-kondo can warn about some that can be detected from the source code at compile time, but probably not for apply calls with variable number of args supplied in the last sequence.

andy.fingerhut00:01:36

Many such cases involving apply are in general not possible to detect at compile time (as in, as hard as the halting problem is to solve)

pinkfrog00:01:48

The checking is checking against with the function it self, not the arglist meta info on the related var, right?

andy.fingerhut01:01:13

I am pretty sure it checks against the function itself. It is pretty easy to alter the arglist meta info of a function to mismatch what the function takes, and find out, in a short REPL session. See sample REPL session that I added in a thread to this message, if you are curious.

andy.fingerhut01:01:54

user=> (defn foo
  ([x y] (+ x y))
  ([x y z w] (+ x y z w)))
#'user/foo
user=> (meta #'foo)
{:arglists ([x y] [x y z w]), :line 7, :column 1, :file "NO_SOURCE_PATH", :name foo, :ns #object[clojure.lang.Namespace 0x55424fb0 "user"]}
user=> (foo 1)
Execution error (ArityException) at user/eval145 (REPL:11).
Wrong number of args (1) passed to: user/foo
user=> (alter-meta! #'foo #(assoc % :arglists '([x])))
{:arglists ([x]), :line 7, :column 1, :file "NO_SOURCE_PATH", :name foo, :ns #object[clojure.lang.Namespace 0x55424fb0 "user"]}
user=> (meta #'foo)
{:arglists ([x]), :line 7, :column 1, :file "NO_SOURCE_PATH", :name foo, :ns #object[clojure.lang.Namespace 0x55424fb0 "user"]}
user=> (foo 1)
Execution error (ArityException) at user/eval154 (REPL:14).
Wrong number of args (1) passed to: user/foo
user=> (foo 1 2)
3
user=> (foo 1 2 3)
Execution error (ArityException) at user/eval158 (REPL:16).
Wrong number of args (3) passed to: user/foo
user=> (foo 1 2 3 4)
10

pinkfrog01:01:47

yes. that’s what I checked. Thanks for that.

noisesmith20:01:00

it might be helpful to describe it as a lookup failure, you aren't validating that a given arg count is accepted, you are looking up the invoke arity, and blowing up when it isn't there

Takis_02:01:44

Hello,when we do (future & body),we run the body on a separate thread as far as i understand. All the body will be executed by the same thread?For (prn (.getId (Thread/currentThread))) at any place on the body will give always the same ID?

Takis_02:01:06

i have a function that i want to behave differently depending on the future it executes

Alex Miller (Clojure team)02:01:22

futures are multiplexed over a thread pool

Alex Miller (Clojure team)02:01:40

so could be many threads (they will die off if not used)

Takis_02:01:29

thank you , i will read more of those and think of alternatives

alexyakushev10:01:30

Hello folks! I often see a problem when in a long-running REPL the (RT/baseLoader) becomes this enormous chain of DynamicClassLoaders. Basically, every form compile operation adds another loader onto the stack. As a result, certain class-resolving calls become woefully slow as they have to walk the whole chain before they can report an error, for example. This classloader chain growth – is this something expected, or is there something wrong with my setup?

p-himik10:01:21

There was recently a discussion about it, you can still find it. tl;dr: on possible reason is if you use an old version of nREPL: https://github.com/nrepl/nrepl/issues/8

alexyakushev11:01:17

Thank you! I think I've seen this ticket before, but couldn't find it again. It looks like that the fix that was implemented is not a fix at all, since it only resets the context classloader of the thread while (RT/baseLoader) looks at clojure.lang.Compiler/LOADER dynamic variable first.

vemv18:01:07

The fix seemed to be effective (I cherry-picked over clojure.tools.nrepl because I don't use the more modern nrepl) On top of it, changing SoftReference to WeakReference here https://github.com/clojure/clojure/blob/c1ede7899d8d7dd4dcd6072d5c0f072b9b2be391/src/jvm/clojure/lang/DynamicClassLoader.java#L47 had tremendous effect in my dev experience - now GC kicks in more frequently so overall code loading time is faster and more stable (https://github.com/clojure/clojure/commit/85e99ee9 went in the exact opposite direction so I doubt an upstream proposal would succeed. Nonetheless building clojure with my change passes and the result is stable)

vemv18:01:43

> This classloader chain growth – is this something expected, or is there something wrong with my setup? My understanding is that it's somewhat normal to have even thousands of DynamicClassLoaders around. A same given DCL can only load a class once, so often on eval, load etc a new DCL must be created They do get GCed, which is what the Soft/Weak references control (with Soft it happens but e.g. 1h later when it's unavoidable) (of course take my analysis with a grain of salt - I'm just a guy and his YourKit)

alexyakushev20:01:33

@U45T93RA6 Thank you for the response! I also observed the SoftReference used for cache becomes problematic in my usecase. Those references free only when OOM error is triggered, so I went as far as artificially producing an OutOfMemoryError by creating a large array during the reload cycle. But it doesn't work very reliably still.

vemv21:01:48

If you find something interesting don't hesitate to share over DM :) I only started using my (slight) clojure fork this week, am open to other solutions/insights

vemv21:01:43

I never experienced an actual OOM btw. I use a lot of JVM flags though b/c of unrelated OOMs in the past

delaguardo14:01:46

In the docstring for reify macro there is a string

Methods should be supplied for all methods of the desired
protocol(s) and interface(s).
However, right now it is possible to implement a subset of methods:
(defprotocol Foo
  (method-1 [this] [this a])
  (method-2 [this] [this a] [this a b]))

(method-2
 (reify
   Foo
   (method-2 [this a b] (format "(method-2 %s %s)" a b)))
 1 2)
Is it ok to rely on that behavior? Probably, it is unlikely to change in the future because it potentially can break compatibility with older versions of clojure. Maybe the docstring should be changed?

NPException14:01:16

We could be nitpicky and say that the docstring may be technically correct already because it says "should" and not "must". 😄

delaguardo14:01:19

right ) but at the same time it brings confusion using all word )

borkdude14:01:31

I think it's more a good practice to prevent run-time errors than a necessity.

Alex Miller (Clojure team)14:01:19

they should be supplied, but it's ok to be lazy if there are methods you know won't be called or whatever

Alex Miller (Clojure team)14:01:39

and it would be a breaking change at this point to require all methods to be impl'ed

delaguardo15:01:23

thanks for clarification @alexmiller

jaide22:01:39

Can’t remember exactly but if you use recur that is somewhat optimized but it’s not equivalent to tail-call-optimization right? The downside of that being the recursion is not as performant as a tail-call-optimized implementation, which Java doesn’t currently support?

dpsutton22:01:59

recur is a self tail call and is optimized. tail-call-optimization usually means both self and other function calls. Clojure only offers explicit self-tail-call optimizations.

👍 10
p-himik22:01:28

There's also trampoline for mutual recursion.

Alex Miller (Clojure team)23:01:56

I think I've only used trampoline one time in 10 years of Clojure work

didibus23:01:52

How do you even implement a mutually recursive algorithm with an iterative approach?

phronmophobic23:01:37

that's what trampoline is for.

didibus23:01:09

Ya, but I was thinking like in Java what would you do? You'd just implement a form of trampoline?

didibus23:01:34

Or there's something you can do with like named nested loops or using break/continue ?

phronmophobic23:01:58

function process(startData){
   var nextOp =startOp;
   var nextData =startData;
   while (true){
      nextOp, nextData = doOp(nextOp, nextData);
      if (nextOp == QUIT_OP){
         break;
      }
   }
}

phronmophobic23:01:09

it's kinda the same idea

didibus23:01:40

Ya I mean that's just a trampoline.

phronmophobic23:01:42

also, that's psuedo js rather than psuedo java

didibus23:01:47

But I think actually you can use named loops for it

phronmophobic23:01:48

I'm not familiar with named loops, but that sounds plausible

didibus23:01:18

bar: for (...) {
   foo: for (...) {
      do_whatever();
      continue bar;
      }
   do_some_other_thing();
   continue foo;
   }
}
Something like that

paulocuneo23:01:39

which algorithm in particular :thinking_face:?

didibus23:01:14

I don't know, can't think of anything that requires mutual recursion 😛

didibus23:01:48

I'm just wondering, if there was some algo that is simple to implement with mutual recursion, what would the iterative approach look like?

andy.fingerhut23:01:26

You can implement your own replacement for the call stack.

didibus23:01:41

Hum, I need to be more precise. I'm talking about an algorithm that is mutually recursive but does not need the stack either. So like a case where not having indirect recursion TCO in Clojure is a "problem"

andy.fingerhut23:01:01

Then do a loop with a variable saying what "function" to call next, which is implemented as a case statement with one branch per "function". Each branch pushes something onto the hand-implemented "call stack" if you want to "call a function", or pops something off of it if it wants to "return".

andy.fingerhut23:01:29

You mean: there is mutual recursion, but every call is a tail call?

👍 3
didibus23:01:44

So trampoline IS the iterative approach to indirect and mutual recursion ?

didibus23:01:05

Ya, like I'm thinking a case where someone goes: "Dang it, this would be so much easier to implement if only Clojure had TCO for indirect recursion"

didibus23:01:06

And then, well Java doesn't have that either, so what would Java do? Would it just also create a sort of Trampoline, or is there some other imperative feature like GOTO, or labeled loops or other that can be used for it as well

paulocuneo23:01:34

TCO is about perf, not about easy 😛

didibus23:01:00

I feel TCO is about making performance easy

didibus23:01:22

Some algos are easier to implement recursively, (also that's the only way to implement them immutably sometimes), but if you didn't have TCO, you'd be forced to take the harder route of implementing it with the iterative algo

paulocuneo23:01:53

I guess readability will depend on the algo too

public static void visitNode(File file) {
  if (file.isDirectory()) {
    visitChildren(file.listFiles());
  } else {
    process(file);
  }
}

public static void visitChildren(File[] files) {
  for (File file : files) {
    visitNode(file);
  }
}
;; can be written like so:
(loop [[file & files' :as files] files]
  (cond
    (empty?    files)  nil
    (directory? file) (recur (concat (children file)
                                     files'))
    :else             (do (process file)
                          (recur files')))

didibus23:01:53

That's a good example

didibus23:01:03

So what would be the iterative approach?

andy.fingerhut00:01:38

As in, using TCO? Or using no calls at all?

andy.fingerhut00:01:19

If you mean using no method/function calls at all, the only thing I can think of is the one I mentioned earlier -- implement your own stack and loop while pushing/popping elements of that stack at appropriate times.

didibus00:01:53

I mean without using recursion

didibus00:01:50

And also without using a stack, since the example doesn't need one

didibus00:01:42

I guess it would just be a for loop where you mutate the collection being looped over as you loop over it

andy.fingerhut01:01:45

That example definitely needs a stack that has a depth proportional to the depth of the directory structure of the file system, unless I completely misunderstand the code example.

andy.fingerhut01:01:43

Or, I guess in the proposed Clojure implementation, the use of an explicit stack was avoided by just keeping one big collection of all files and directories not "visited" yet.

andy.fingerhut01:01:04

The total memory storage required is the same as if it had been done with a stack of parameter values.

andy.fingerhut01:01:45

In my experience, most places where mutual recursion shows up naturally in code, without forcing it, the maximum call depth is "small", as in the case where the Java code above has a maximum call depth limited by the maximum depth of the file system directory structure.

andy.fingerhut01:01:19

So TCO is not really needed.

didibus23:01:04

Don't you need a GOTO?