Fork me on GitHub
Ruy Valle01:08:47

Suppose a library contains function public a which depends on private function b. Is there a way to implement my own version of b and have a use my b , without having to copy a portion of the library’s source?


nothing cleaner than copying the code (alter-var-root and intern exist, but they won't lead to a cleaner result, since your code now relies on modifying something that other code might want to use as is)


If it makes sense that b should be hot swappable or behind some polymorphic construct consider a PR to the library as well

Ruy Valle02:08:44

that makes sense, thanks


doubt about "clojure internals" Why does use a "inline value"¹ is faster then "a reference value"²? (let-benchmark in thread) ¹ (mapv #(+ % 1) coll) ² (mapv #(+ % one) coll) "faster" means that it will take less time in a HUGE coll


My benchmark:

(let [coll (repeat 1e6 {})
      m {:my :meta}]
  (doseq [[id f] {:inside-fun #(mapv
                                 (fn [e]
                                   (with-meta e {:my :meta}))
                  :inside-let #(mapv
                                 (fn [e]
                                   (with-meta e m))
    (prn id)
    (dotimes [_ 3]
      (time (dotimes [_ 1e2]
My results
"Elapsed time: 1901.891841 msecs"
"Elapsed time: 1904.658297 msecs"
"Elapsed time: 1947.197886 msecs"
"Elapsed time: 6607.768999 msecs"
"Elapsed time: 6632.838103 msecs"
"Elapsed time: 6749.213546 msecs"


Actually, use a "inline function" looks a bit better then "reference function"

(let [coll (doall (repeat 1e6 {}))
      add-meta (fn [e]
                 (with-meta e {:my :meta}))]
  (doseq [[id f] {:inside-fun #(mapv (fn [e]
                                       (with-meta e {:my :meta}))
                  :add-meta2  #(mapv add-meta
    (prn id)
    (dotimes [_ 3]
      (time (dotimes [_ 1e2]
"Elapsed time: 1999.221744 msecs"
"Elapsed time: 1996.666202 msecs"
"Elapsed time: 2033.77291 msecs"
"Elapsed time: 2036.254896 msecs"
"Elapsed time: 2072.717188 msecs"
"Elapsed time: 2062.882369 msecs"
=> nil


The var one is dereferenced only once when the function is evaluated, so there is no significant difference:

user=> (def one 1)
user=> (mapv #(+ % one) [1 1 (do (alter-var-root #'one (constantly 2)) 1) 1 1 1])
[3 3 3 3 3 3]


@U04V15CAJ I understand it, but did you see the first benchmark? I think that the answer is something like "when where is no external references, the JVM can do a better optimization"


Hmm, that's an interesting difference.


It also occurs in bb @U04V15CAJ (using coll of 1e4 because 1e6 was too slow)

"Elapsed time: 917.150869 msecs"
"Elapsed time: 922.404064 msecs"
"Elapsed time: 949.586256 msecs"
"Elapsed time: 1750.694556 msecs"
"Elapsed time: 1756.074617 msecs"
"Elapsed time: 1783.673856 msecs"


I would expect one to be dereferenced on each call. Is that not the case?


There are two different things at play here, I think. names referring to local bindings vs names referring to vars. In this example:;cid=C03S1KBA2 the var is only dereferenced once


I see now that this example is also not good, since the alter-var-root is executed before mapv is doing anything


it should be a lazy collection without chunking to fix the example


Anyway, there are a couple of different situations in this thread, they should be discussed separately


Yeah I was fixing to say, “something’s not right about that.”


vars are always derefd at runtime


it’s why re-evaling defs at the repl works


is my understanding



(def one 1)

(last (mapv #(+ % one)
            (repeat 1e8 1)))

(alter-var-root #'one (constantly 2))


user=> (def x 1)
user=> (defn foo [] (+ x 1))
user=> (foo)
user=> (def x 2)
user=> (foo)
user=> (def foo (partial + x 1))
user=> (foo)
user=> (def x 3)
user=> (foo)




So I guess when used in a function body it's compiled down to a deref, but when passed as an arg, it it's only dereferenced once


ah yeah, that makes perfect sense


otherwise you’re passing vars, not values


so scratch what I said. direct linking influences this for function calls, not with other usages. ^:const influences this for all usages.


Well, of course. I was assuming repl interaction.


(i.e. no direct linking, no const)


if it would work otherwise, the REPL would be a hard place to work in it without reloading all your code all the time


interesting that #() works more like partial than (fn [])


didn’t think about it before, but that does make sense


huh, how so?


#() expands into (fn [])


oh wait, misread


user=> '#()
(fn* [] ())


Is it a type hint thing? In one case it's a primitive?


I have a 7-level deep list comprehension (`for`) and compilation of the ns fails with "filename too long". I assume the generated code (300+ lines) has too many nested lambdas and the classnames hit the 255 byte limit. Does anyone know of another for implementation which is maybe not as optimized as the one in core, but avoids this problem?


that's doseq, but you could do something similar with for


@ghadi thanks, I'll look into it


you could probably separate your 7 nested fors into 7 functions that each used for once


yeah, but it'd be pretty verbose (especially since I use data from every nesting level in the for body)


the real annoying thing about this is that it works fine when developing with the repl, but blows up when compiling


jsn: thanks, but I just have a deeply nested structure (with fixed levels). so the code looks more like (for [[p1 ys] xs, [p2 zs] ys, ...] ...)


guess I could also just quote the whole for and eval it to avoid the compilation issue, but that just feels wrong 🙂


do you mean compiling to disk?


@dpsutton I'm using AOT, that's when the error comes up


can you move this function into a non-AOT'd namespace?


(not sure if AOT is transitive)


@dpsutton: everything is AOT'd (except for library deps), I can't ship source code with the application


how about splitting up your code?


seems like the biggest win for you


@ghadi: I think that's the way to go as well; I was just toying with different ideas

Ben Sless17:08:31

Question regarding transducers and concatenation: I have 2+ collections which are returned from different functions. Ideally, they wouldn't even be collections, but eductions or something similar. I want to take these two collections, concat them, and pass them through a transducer without generating an intermediary lazy sequence. The partial solution I came up with is a new type which implements CollReduce:

(require '[clojure.core.protocols :as p])

(deftype Concatenation [coll1 coll2]
    [this f]
     (p/coll-reduce coll1 f)))

    [this f init]
     (p/coll-reduce coll1 f init))))
Really not sure this is the right way, and my first arity is wrong. Ideas and suggestions welcome This is what already works:
(def c (Concatenation. [1 2 3] [4 5 6]))

(transduce (comp (map inc) (filter even?) (map inc)) conj c)

(def e (->Eduction (comp (map inc) (filter even?) (map inc)) c))

(reduce conj [] e)


you should be able to simply use cat:

> (let [a [1 2 3]
        b [4 5 6]]
    (into [] (comp cat  
                   (map inc)
                   (filter even?)
                   (map inc))
          [a b]))
;; [3 5 7]

👍 3
Ben Sless17:08:14

That works if I pack them in a vector first, wondering if there's something more lightweight

Ben Sless17:08:40

But I guess it's simpler than my hare brained solution 🙃


I don't think there's any way to get around grouping/packing them into something (even creating a Concatenation is packing them). not sure what the most lightweight option is.

Ben Sless17:08:09

Technically a Concatenation would be lighter, it's an object with just one method and two members. A vector is an array + lots of stuff and methods. But with the price being confusing my coworkers I'm not sure it's worth it


that is an assertion without evidence


agree that cat is what you want

Ben Sless17:08:23

How is it without evidence?


1) reduce impls need to handle early termination (this impl is incorrect -- try passing a (take 5) transducer) 2) cat doesn't care about the source being in a vector, cat just unpacks items no matter where they came from (could be from an eduction source) 3) Concatenation only works on pairs of collections, so you'd have to make cons cells to do more (you said you have 2+ collections) 4) coll-reduce is not as fast as implementing clojure.lang.IReduceInit


"vector is an array + lots of stuff and methods isn't" doesn't make it slow

Ben Sless17:08:50

Absolutely. I was talking about the allocation. And I started by saying my implementation was incomplete and asking for suggestions on how I can flesh it out =\


sorry to be negative 🙂

Ben Sless17:08:09

It's okay, I tend to take things unnecessarily to heart

Ben Sless17:08:46

Won't coll-reduce be approximately as fast for IReduceInit since it's extended to it?


the dispatch for IReduceInit is special cased


it will be faster to dispatch but still dominated by the reduction time


but IReduceInit is "better" in that I think Rich regrets having the arity of reduce that doesn't take the explicit "init" arg


I think eduction+cat is the way to go here, all the benefits you're looking for + in the stdlib

Ben Sless17:08:21

I think I'll go with that one, thanks 🙂

Ben Sless17:08:07

btw, I tested allocation overhead of a vector vs. a Concatenation. It's measurable, but all of it is in the ns range

Ben Sless17:08:43

So I'll let it rest as premature optimization

Alex Whitt17:08:31

Anyone know of a way to execute a shell command, but redirect its output immediately to stdout? I don't mean printing :out after it's done executing. I've got a long-running command and I'd like to watch the output as it goes, before the process exits.


something like:

(require '[ :as sh])
(require '[ :as io])
(let [{:keys [out]} (sh/sh "cat" "")]
   (io/copy out System/out))

Alex Whitt17:08:59

But sh doesn't return until the process exits, AFAICT


oh, right. you might have to call (.exec (Runtime/getRuntime) ...) directly

Alex Whitt18:08:08

Ah, that's worth looking into. Thank you!


I think the best option is ProcessBuilder



(ins)user=> (-> ["ls" "-l"] (ProcessBuilder.) (.inheritIO) (.start) (.waitFor))
total 72
drwxrwxr-x 2 justin justin 4096 May 29 23:07 2-literate-9-2019
drwxrwxr-x 2 justin justin 4096 Apr 17 17:14 3-simple-9-2019
drwxrwxr-x 2 justin justin 4096 Apr 17 17:14 4-macros-9-2019
drwxrwxr-x 2 justin justin 4096 Apr 17 17:14 5-fennel-4-2020
drwxrwxr-x 4 justin justin 4096 Jul  2 19:53 6-scheme-4-2020
drwxrwxr-x 2 justin justin 4096 Jul 18 07:09 7-raw
drwxrwxr-x 5 justin justin 4096 Aug  1 13:24 8-arm-assembly
drwxrwxr-x 2 justin justin 4096 Jul 27 15:06 9-learner
-rw-rw-r-- 1 justin justin 1594 Aug 23  2019 composition.csd
drwxrwxr-x 2 justin justin 4096 Dec 13  2019 fennel-dist
drwxr-xr-x 2 justin justin 4096 Jul 14  2019 fennel-libs
-rw-r--r-- 1 justin justin  382 Jul 14  2019 Makefile
-rw-r--r-- 1 justin justin  779 Jul 24  2019 osc-in.csd
drwxrwxr-x 2 justin justin 4096 Aug  1  2019 simple-a
drwxrwxr-x 2 justin justin 4096 Aug 15  2019 simple-b
-rw-rw-r-- 1 justin justin 4826 Sep  7  2019 synth-compiler.fnl
-rw-rw-r-- 1 justin justin 1188 Jul 26 08:16 todo.otl

Alex Whitt18:08:37

Also worth looking into, thank you as well


if you don't include .waitFor the shell command will run in the background, but that can mess with the repls usage of the same IO ports


instead of .inheritIO you can map each of stdin, stdout, stderr to an arbitrary stream (eg. if you wanted to write a state machine that ineracts with a python command line)


oh, and this even works for programs that use terminal drawing (eg. replace ["ls" "-l"] with ["htop"] or ["vim"] and the programs work like normal, and your repl resumes when they exit)


Did I just miss the Clojure talk where someone quotes Charles Kettering?


> A problem well stated is a problem half-solved. — Charles Kettering


Hello. I wrote a commandline tool (compiled with GraalVM) that prints to the console using println. When running this executable I see the output in my terminal. Now I'm trying to use my executable in a separate program (specifically Godot's OS.execute function). This program should then output anything from my executable. The problem is that it's not printing anything, as if my program is not writing to STDOUT. If I use another command (e.g. ls) it does work. Am I supposed to do something special with Clojure to "properly" write to STDOUT for other programs to catch that? e.g. an EOF or something?

Alex Miller (Clojure team)20:08:23

println should be writing and flushing to stdout


All right, then I'm just doing something wrong


My program does block the process though, so maybe Godot is waiting for it to finish

Alex Miller (Clojure team)20:08:12

any chance you're doing something lazy and it's not being realized?

Alex Miller (Clojure team)20:08:30

that's a common issue - top-level is a map or something


afaik I'm using mapv everywhere


Ok, apparently Godot is waiting for the process to finish 😕


Which is really annoying because then I can't implement a watcher executable

Alex Miller (Clojure team)20:08:21

ah, if you did anything with future or agent you might need to (shutdown-agents)


clojure is poorly suited for that kind of IPC - as a workaround you could leave a socket server running and then execute code by sending / receiving TCP via the socket


you might have to start the clojure daemon process from outside Godot - I don't know what facilities Godot has for running helpers


I also don't know if Godot has socket support, but seeing that's a pre-requisite for http etc. one would hope it is htere


and worst case you could shell out to nc


Yeah I'll have to investigate a bit more what my options are regarding Godot. It's unfortunate that godot doesn't seem to support async logging


Godot is waiting? (Sorry, couldn't resist)

😄 9

in one window:

$ clj -J-Dclojure.server.repl="{:port 5555 :accept clojure.core.server/repl}"
Clojure 1.10.1
in another
$ echo '*clojure-version*' | nc localhost 5555
user=> {:major 1, :minor 10, :incremental 1, :qualifier nil}


I wonder if there's an easy way to get that without the prompt

Alex Miller (Clojure team)20:08:41

clj -e '*clojure-version*'

Alex Miller (Clojure team)20:08:57

or I guess I don't understand what you're asking


@alexmiller sorry, I meant as a way for @kevin.van.rooijen to implement his goal (a long running watcher that Godot can consume individual results from)


the socket repl works, but you end up needing to manually remove a prompt

Alex Miller (Clojure team)20:08:30

if you start the socket server programmatically, I think there is a config attr for whether to prompt or not


oh right, there's a dynamic var for the prompt function, or a higher order fn for the prompt right?

Alex Miller (Clojure team)20:08:18

well, I guess there's configurable support in clojure.main/repl

Alex Miller (Clojure team)20:08:48

but that's not specially exposed out through the socket server config - you'd need to make your own accept function with whatever customization you need


yeah - clojure.core.server/repl just passes a config to clojure.main/repl, I see that now, doesn't look like it would be super hard to set up, but also not turnkey


@kevin.van.rooijen @alexmiller aha this works:

$ clj
Clojure 1.10.1
(cmd)user=> (ns server.quiet (:require [clojure.main :as m] [clojure.core.server :as s]))
(cmd)server.quiet=> (defn repl [] (m/repl :init s/repl-init :read s/repl-read :prompt (constantly "")))
(cmd)server.quiet=> (s/start-server {:port 6666 :name 'repl :accept 'server.quiet/repl})
#object[ 0x736d6a5c "ServerSocket[addr=localhost/,localport=6666]"]
$ echo '*clojure-version*' | nc -q0 localhost 6666
{:major 1, :minor 10, :incremental 1, :qualifier nil}


of course there are tools like nrepl with proper protocols that differentiate one off results from longer running repl sessions etc.


interesting note, I wasn't able to do this without making a new ns, if you try to define an :accept function inside ns user, the repl process blows up with

user=> Exception in thread "Clojure Connection repl 1" Could not locate use
r__init.class, user.clj or user.cljc on classpath.
        at clojure.lang.RT.load(
        at clojure.lang.RT.load(
        at clojure.core$load$fn__6839.invoke(core.clj:6126)
        at clojure.core$load.invokeStatic(core.clj:6125)
        at clojure.core$load.doInvoke(core.clj:6109)
        at clojure.lang.RestFn.invoke(
        at clojure.core$load_one.invokeStatic(core.clj:5908)
        at clojure.core$load_one.invoke(core.clj:5903)
        at clojure.core$load_lib$fn__6780.invoke(core.clj:5948)
        at clojure.core$load_lib.invokeStatic(core.clj:5947)
        at clojure.core$load_lib.doInvoke(core.clj:5928)
        at clojure.lang.RestFn.applyTo(
        at clojure.core$apply.invokeStatic(core.clj:667)
        at clojure.core$load_libs.invokeStatic(core.clj:5985)
        at clojure.core$load_libs.doInvoke(core.clj:5969)
        at clojure.lang.RestFn.applyTo(
        at clojure.core$apply.invokeStatic(core.clj:667)
        at clojure.core$require.invokeStatic(core.clj:6007)
        at clojure.core.server$accept_connection.invokeStatic(server.clj:73)
        at clojure.core.server$start_server$fn__8879$fn__8880$fn__8882.invoke(server.clj:117)


It's an interesting idea. Godot does have some TCP functionality. That would probably be a good option


at the very least, it's likely the lowest friction option for getting ad-hoc results from a long running clojure process


Hey, I’m trying to convince Robert Virding to put some Clojurisms inside LFE and he asked: > One thing I don’t get with how clojure uses `{ }` and `[ ]`. Do they return literal values like LFEs `#( )` and `#m( )` or are they evaluated like LFE `(tuple …)` and `#m( … )`? I tried to look in some tutorials online but they describe them as literal values in the reader which are then evaluated. So are the literals ar are they evaluated and if so when? What is the best answer?


literals are evaluated while reading


eg. :foo is turned into a keyword by the reader


[:foo] is turned into a vector containing a keyword by the reader


this happens before eval


at the reading time


they are not like #() because #() only expands to a form that gets further evaluating


right, this is done by the reader


ok, without expansion. I got it


I’ll tell him


server.quiet=> '#(read but not evaluated yet)
(fn* [] (read but not evaluated yet))


you can use ' to see what is provided by the reader before evaluation


@marciol btw what is LFE in this context? I think I glossed it enough to answer your question but I am realizing I have no idea what that acronym is


Lisp Flavoured Erlang @noisesmith


He is in doubt about how literals are processed in Clojure, if they are expansioned so [] turns out (vector …) or not.


I'm not sure what "literal" vs. "evaluated" is supposed to mean there, but it might be significant that the data types returned by [] and {} are values rather than objects for equality - they aren't a place that is changed and equal based on pointer location, but a value that is equal based on nested contents


the reader does the same thing vector does, but AFAIK doesn't actually call the vector function


the easy and literal test of what you are asking:

server.quiet=> '[]
reading but not evaluating still gives us [], and not (vector)


but I wonder when that difference would matter - if you wanted to locally override vector and leverage the existing literals?


yes, so the reader will handle accordingly


(cmd)server.quiet=> (with-redefs [clojure.core/vector +] (vector 1 2))
(cmd)server.quiet=> (with-redefs [clojure.core/vector +] [1 2])
[1 2]
(ins)server.quiet=> (with-redefs [clojure.core/vector +] (read-string "[1 2]"))
[1 2]


I think that he is trying to emulate the same behaviour in LFE to handle literals on the reader.


Or, better saying, I caused the discussion about use the same approach


@marciol you might be interested in the tagged literal feature, if you haven't looked into it yet


it allows custom tags (that use vector or map literals or even eg. strings, but create whatever result data you like)


but I think a tagged reader with side effects would be considered pathological


Yes, I’m a fan of Aero for this reason @noisesmith