This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-08-03
Channels
- # announcements (63)
- # asami (29)
- # beginners (23)
- # calva (23)
- # cider (18)
- # clj-kondo (12)
- # clojure (156)
- # clojure-europe (30)
- # clojure-italy (6)
- # clojure-nl (5)
- # clojure-uk (6)
- # clojurescript (14)
- # conjure (26)
- # cursive (8)
- # datalog (143)
- # datascript (1)
- # datomic (6)
- # duct (2)
- # emacs (50)
- # events (4)
- # figwheel-main (6)
- # fulcro (7)
- # graphql (12)
- # jobs (6)
- # malli (1)
- # mid-cities-meetup (2)
- # off-topic (4)
- # pathom (6)
- # portal (7)
- # re-frame (10)
- # reagent (8)
- # reitit (10)
- # releases (1)
- # reveal (18)
- # shadow-cljs (39)
- # sql (4)
- # tools-deps (36)
- # vim (25)
- # xtdb (6)
Suppose a library contains function public a
which depends on private function b
. Is there a way to implement my own version of b
and have a
use my b
, without having to copy a portion of the library’s source?
nothing cleaner than copying the code (alter-var-root and intern exist, but they won't lead to a cleaner result, since your code now relies on modifying something that other code might want to use as is)
If it makes sense that b should be hot swappable or behind some polymorphic construct consider a PR to the library as well
doubt about "clojure internals"
Why does use a "inline value"¹ is faster then "a reference value"²? (let-benchmark in thread)
¹ (mapv #(+ % 1) coll)
² (mapv #(+ % one) coll)
"faster" means that it will take less time in a HUGE coll
My benchmark:
(let [coll (repeat 1e6 {})
m {:my :meta}]
(doseq [[id f] {:inside-fun #(mapv
(fn [e]
(with-meta e {:my :meta}))
coll)
:inside-let #(mapv
(fn [e]
(with-meta e m))
coll)}]
(prn id)
(dotimes [_ 3]
(time (dotimes [_ 1e2]
(f))))))
My results
:inside-fun
"Elapsed time: 1901.891841 msecs"
"Elapsed time: 1904.658297 msecs"
"Elapsed time: 1947.197886 msecs"
:inside-let
"Elapsed time: 6607.768999 msecs"
"Elapsed time: 6632.838103 msecs"
"Elapsed time: 6749.213546 msecs"
Actually, use a "inline function" looks a bit better then "reference function"
(let [coll (doall (repeat 1e6 {}))
add-meta (fn [e]
(with-meta e {:my :meta}))]
(doseq [[id f] {:inside-fun #(mapv (fn [e]
(with-meta e {:my :meta}))
coll)
:add-meta2 #(mapv add-meta
coll)}]
(prn id)
(dotimes [_ 3]
(time (dotimes [_ 1e2]
(f))))))
:inside-fun
"Elapsed time: 1999.221744 msecs"
"Elapsed time: 1996.666202 msecs"
"Elapsed time: 2033.77291 msecs"
:add-meta2
"Elapsed time: 2036.254896 msecs"
"Elapsed time: 2072.717188 msecs"
"Elapsed time: 2062.882369 msecs"
=> nil
The var one
is dereferenced only once when the function is evaluated, so there is no significant difference:
user=> (def one 1)
#'user/one
user=> (mapv #(+ % one) [1 1 (do (alter-var-root #'one (constantly 2)) 1) 1 1 1])
[3 3 3 3 3 3]
@U04V15CAJ I understand it, but did you see the first benchmark? I think that the answer is something like "when where is no external references, the JVM can do a better optimization"
It also occurs in bb
@U04V15CAJ (using coll of 1e4
because 1e6
was too slow)
:inside-let
"Elapsed time: 917.150869 msecs"
"Elapsed time: 922.404064 msecs"
"Elapsed time: 949.586256 msecs"
:inside-fun
"Elapsed time: 1750.694556 msecs"
"Elapsed time: 1756.074617 msecs"
"Elapsed time: 1783.673856 msecs"
nil
There are two different things at play here, I think. names referring to local bindings vs names referring to vars. In this example: https://clojurians.slack.com/archives/C03S1KBA2/p1596460259320200?thread_ts=1596457910.319500&cid=C03S1KBA2 the var is only dereferenced once
I see now that this example is also not good, since the alter-var-root is executed before mapv is doing anything
Anyway, there are a couple of different situations in this thread, they should be discussed separately
yeah:
(def one 1)
(last (mapv #(+ % one)
(repeat 1e8 1)))
(alter-var-root #'one (constantly 2))
user=> (def x 1)
#'user/x
user=> (defn foo [] (+ x 1))
#'user/foo
user=> (foo)
2
user=> (def x 2)
#'user/x
user=> (foo)
3
user=> (def foo (partial + x 1))
#'user/foo
user=> (foo)
3
user=> (def x 3)
#'user/x
user=> (foo)
3
So I guess when used in a function body it's compiled down to a deref, but when passed as an arg, it it's only dereferenced once
so scratch what I said. direct linking influences this for function calls, not with other usages. ^:const
influences this for all usages.
if it would work otherwise, the REPL would be a hard place to work in it without reloading all your code all the time
I have a 7-level deep list comprehension (`for`) and compilation of the ns fails with "filename too long". I assume the generated code (300+ lines) has too many nested lambdas and the classnames hit the 255 byte limit. Does anyone know of another for
implementation which is maybe not as optimized as the one in core, but avoids this problem?
you could probably separate your 7 nested fors into 7 functions that each used for once
yeah, but it'd be pretty verbose (especially since I use data from every nesting level in the for
body)
the real annoying thing about this is that it works fine when developing with the repl, but blows up when compiling
@ak407 https://github.com/clojure/math.combinatorics 's cartesian-product
, perhaps?
jsn: thanks, but I just have a deeply nested structure (with fixed levels). so the code looks more like (for [[p1 ys] xs, [p2 zs] ys, ...] ...)
guess I could also just quote the whole for
and eval
it to avoid the compilation issue, but that just feels wrong 🙂
@dpsutton: everything is AOT'd (except for library deps), I can't ship source code with the application
@ghadi: I think that's the way to go as well; I was just toying with different ideas
Question regarding transducers and concatenation:
I have 2+ collections which are returned from different functions. Ideally, they wouldn't even be collections, but eductions or something similar. I want to take these two collections, concat them, and pass them through a transducer without generating an intermediary lazy sequence.
The partial solution I came up with is a new type which implements CollReduce
:
(require '[clojure.core.protocols :as p])
(deftype Concatenation [coll1 coll2]
p/CollReduce
(coll-reduce
[this f]
(p/coll-reduce
coll2
f
(p/coll-reduce coll1 f)))
(coll-reduce
[this f init]
(p/coll-reduce
coll2
f
(p/coll-reduce coll1 f init))))
Really not sure this is the right way, and my first arity is wrong.
Ideas and suggestions welcome
This is what already works:
(def c (Concatenation. [1 2 3] [4 5 6]))
(transduce (comp (map inc) (filter even?) (map inc)) conj c)
(def e (->Eduction (comp (map inc) (filter even?) (map inc)) c))
(reduce conj [] e)
you should be able to simply use cat
:
> (let [a [1 2 3]
b [4 5 6]]
(into [] (comp cat
(map inc)
(filter even?)
(map inc))
[a b]))
;; [3 5 7]
That works if I pack them in a vector first, wondering if there's something more lightweight
I don't think there's any way to get around grouping/packing them into something (even creating a Concatenation
is packing them). not sure what the most lightweight option is.
Technically a Concatenation would be lighter, it's an object with just one method and two members. A vector is an array + lots of stuff and methods. But with the price being confusing my coworkers I'm not sure it's worth it
1) reduce impls need to handle early termination (this impl is incorrect -- try passing a (take 5)
transducer)
2) cat
doesn't care about the source being in a vector, cat just unpacks items no matter where they came from (could be from an eduction source)
3) Concatenation only works on pairs of collections, so you'd have to make cons cells to do more (you said you have 2+ collections)
4) coll-reduce is not as fast as implementing clojure.lang.IReduceInit
Absolutely. I was talking about the allocation. And I started by saying my implementation was incomplete and asking for suggestions on how I can flesh it out =\
Won't coll-reduce be approximately as fast for IReduceInit since it's extended to it?
but IReduceInit is "better" in that I think Rich regrets having the arity of reduce that doesn't take the explicit "init" arg
I think eduction+cat is the way to go here, all the benefits you're looking for + in the stdlib
btw, I tested allocation overhead of a vector vs. a Concatenation. It's measurable, but all of it is in the ns range
Anyone know of a way to execute a shell command, but redirect its output immediately to stdout? I don't mean printing :out after it's done executing. I've got a long-running command and I'd like to watch the output as it goes, before the process exits.
check out https://clojure.github.io/clojure/clojure.java.shell-api.html and https://clojure.github.io/clojure/clojure.java.io-api.html#clojure.java.io/copy
something like:
(require '[clojure.java.shell :as sh])
(require '[ :as io])
(let [{:keys [out]} (sh/sh "cat" "README.md")]
(io/copy out System/out))
But sh
doesn't return until the process exits, AFAICT
oh, right. you might have to call (.exec (Runtime/getRuntime) ...)
directly
Ah, that's worth looking into. Thank you!
I think the best option is ProcessBuilder
snippet:
(ins)user=> (-> ["ls" "-l"] (ProcessBuilder.) (.inheritIO) (.start) (.waitFor))
total 72
drwxrwxr-x 2 justin justin 4096 May 29 23:07 2-literate-9-2019
drwxrwxr-x 2 justin justin 4096 Apr 17 17:14 3-simple-9-2019
drwxrwxr-x 2 justin justin 4096 Apr 17 17:14 4-macros-9-2019
drwxrwxr-x 2 justin justin 4096 Apr 17 17:14 5-fennel-4-2020
drwxrwxr-x 4 justin justin 4096 Jul 2 19:53 6-scheme-4-2020
drwxrwxr-x 2 justin justin 4096 Jul 18 07:09 7-raw
drwxrwxr-x 5 justin justin 4096 Aug 1 13:24 8-arm-assembly
drwxrwxr-x 2 justin justin 4096 Jul 27 15:06 9-learner
-rw-rw-r-- 1 justin justin 1594 Aug 23 2019 composition.csd
drwxrwxr-x 2 justin justin 4096 Dec 13 2019 fennel-dist
drwxr-xr-x 2 justin justin 4096 Jul 14 2019 fennel-libs
-rw-r--r-- 1 justin justin 382 Jul 14 2019 Makefile
-rw-r--r-- 1 justin justin 779 Jul 24 2019 osc-in.csd
drwxrwxr-x 2 justin justin 4096 Aug 1 2019 simple-a
drwxrwxr-x 2 justin justin 4096 Aug 15 2019 simple-b
-rw-rw-r-- 1 justin justin 4826 Sep 7 2019 synth-compiler.fnl
-rw-rw-r-- 1 justin justin 1188 Jul 26 08:16 todo.otl
0
Also worth looking into, thank you as well
if you don't include .waitFor
the shell command will run in the background, but that can mess with the repls usage of the same IO ports
instead of .inheritIO
you can map each of stdin, stdout, stderr to an arbitrary stream (eg. if you wanted to write a state machine that ineracts with a python command line)
oh, and this even works for programs that use terminal drawing (eg. replace ["ls" "-l"]
with ["htop"]
or ["vim"]
and the programs work like normal, and your repl resumes when they exit)
Hello. I wrote a commandline tool (compiled with GraalVM) that prints to the console using println
. When running this executable I see the output in my terminal. Now I'm trying to use my executable in a separate program (specifically Godot's OS.execute function). This program should then output anything from my executable.
The problem is that it's not printing anything, as if my program is not writing to STDOUT. If I use another command (e.g. ls
) it does work. Am I supposed to do something special with Clojure to "properly" write to STDOUT for other programs to catch that? e.g. an EOF or something?
println should be writing and flushing to stdout
any chance you're doing something lazy and it's not being realized?
that's a common issue - top-level is a map
or something
ah, if you did anything with future
or agent
you might need to (shutdown-agents)
clojure is poorly suited for that kind of IPC - as a workaround you could leave a socket server running and then execute code by sending / receiving TCP via the socket
you might have to start the clojure daemon process from outside Godot - I don't know what facilities Godot has for running helpers
I also don't know if Godot has socket support, but seeing that's a pre-requisite for http etc. one would hope it is htere
and worst case you could shell out to nc
Yeah I'll have to investigate a bit more what my options are regarding Godot. It's unfortunate that godot doesn't seem to support async logging
in one window:
$ clj -J-Dclojure.server.repl="{:port 5555 :accept clojure.core.server/repl}"
Clojure 1.10.1
user=>
in another
$ echo '*clojure-version*' | nc localhost 5555
user=> {:major 1, :minor 10, :incremental 1, :qualifier nil}
I wonder if there's an easy way to get that without the prompt
clj -e '*clojure-version*'
or I guess I don't understand what you're asking
@alexmiller sorry, I meant as a way for @kevin.van.rooijen to implement his goal (a long running watcher that Godot can consume individual results from)
the socket repl works, but you end up needing to manually remove a prompt
if you start the socket server programmatically, I think there is a config attr for whether to prompt or not
oh right, there's a dynamic var for the prompt function, or a higher order fn for the prompt right?
well, I guess there's configurable support in clojure.main/repl
but that's not specially exposed out through the socket server config - you'd need to make your own accept function with whatever customization you need
yeah - clojure.core.server/repl just passes a config to clojure.main/repl, I see that now, doesn't look like it would be super hard to set up, but also not turnkey
(like the provided https://github.com/clojure/clojure/blob/0035cd8d73517e7475cb8b96c7911eb0c43a1a9d/src/clj/clojure/core/server.clj#L180 )
@kevin.van.rooijen @alexmiller aha this works:
$ clj
Clojure 1.10.1
(cmd)user=> (ns server.quiet (:require [clojure.main :as m] [clojure.core.server :as s]))
nil
(cmd)server.quiet=> (defn repl [] (m/repl :init s/repl-init :read s/repl-read :prompt (constantly "")))
#'server.quiet/repl
(cmd)server.quiet=> (s/start-server {:port 6666 :name 'repl :accept 'server.quiet/repl})
#object[java.net.ServerSocket 0x736d6a5c "ServerSocket[addr=localhost/127.0.0.1,localport=6666]"]
server.quiet=>
$ echo '*clojure-version*' | nc -q0 localhost 6666
{:major 1, :minor 10, :incremental 1, :qualifier nil}
$
of course there are tools like nrepl with proper protocols that differentiate one off results from longer running repl sessions etc.
interesting note, I wasn't able to do this without making a new ns, if you try to define an :accept
function inside ns user, the repl process blows up with
user=> Exception in thread "Clojure Connection repl 1" java.io.FileNotFoundException: Could not locate use
r__init.class, user.clj or user.cljc on classpath.
at clojure.lang.RT.load(RT.java:462)
at clojure.lang.RT.load(RT.java:424)
at clojure.core$load$fn__6839.invoke(core.clj:6126)
at clojure.core$load.invokeStatic(core.clj:6125)
at clojure.core$load.doInvoke(core.clj:6109)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invokeStatic(core.clj:5908)
at clojure.core$load_one.invoke(core.clj:5903)
at clojure.core$load_lib$fn__6780.invoke(core.clj:5948)
at clojure.core$load_lib.invokeStatic(core.clj:5947)
at clojure.core$load_lib.doInvoke(core.clj:5928)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$load_libs.invokeStatic(core.clj:5985)
at clojure.core$load_libs.doInvoke(core.clj:5969)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$require.invokeStatic(core.clj:6007)
at clojure.core.server$accept_connection.invokeStatic(server.clj:73)
at clojure.core.server$start_server$fn__8879$fn__8880$fn__8882.invoke(server.clj:117)
at clojure.lang.AFn.run(AFn.java:22)
at java.lang.Thread.run(Thread.java:748)
It's an interesting idea. Godot does have some TCP functionality. That would probably be a good option
at the very least, it's likely the lowest friction option for getting ad-hoc results from a long running clojure process
Hey, I’m trying to convince Robert Virding to put some Clojurisms inside LFE and he asked: > One thing I don’t get with how clojure uses `{ }` and `[ ]`. Do they return literal values like LFEs `#( )` and `#m( )` or are they evaluated like LFE `(tuple …)` and `#m( … )`? I tried to look in some tutorials online but they describe them as literal values in the reader which are then evaluated. So are the literals ar are they evaluated and if so when? What is the best answer?
literals are evaluated while reading
eg. :foo
is turned into a keyword by the reader
[:foo]
is turned into a vector containing a keyword by the reader
this happens before eval
they are not like #()
because #()
only expands to a form that gets further evaluating
right, this is done by the reader
server.quiet=> '#(read but not evaluated yet)
(fn* [] (read but not evaluated yet))
you can use '
to see what is provided by the reader before evaluation
Thank you @noisesmith
@marciol btw what is LFE in this context? I think I glossed it enough to answer your question but I am realizing I have no idea what that acronym is
Lisp Flavoured Erlang @noisesmith
He is in doubt about how literals are processed in Clojure, if they are expansioned so []
turns out (vector …)
or not.
I'm not sure what "literal" vs. "evaluated" is supposed to mean there, but it might be significant that the data types returned by []
and {}
are values rather than objects for equality - they aren't a place that is changed and equal based on pointer location, but a value that is equal based on nested contents
the reader does the same thing vector
does, but AFAIK doesn't actually call the vector function
the easy and literal test of what you are asking:
server.quiet=> '[]
[]
reading but not evaluating still gives us [], and not (vector)but I wonder when that difference would matter - if you wanted to locally override vector and leverage the existing literals?
(cmd)server.quiet=> (with-redefs [clojure.core/vector +] (vector 1 2))
3
(cmd)server.quiet=> (with-redefs [clojure.core/vector +] [1 2])
[1 2]
(ins)server.quiet=> (with-redefs [clojure.core/vector +] (read-string "[1 2]"))
[1 2]
I think that he is trying to emulate the same behaviour in LFE to handle literals on the reader.
@marciol you might be interested in the tagged literal feature, if you haven't looked into it yet https://clojure.org/reference/reader#tagged_literals
it allows custom tags (that use vector or map literals or even eg. strings, but create whatever result data you like)
but I think a tagged reader with side effects would be considered pathological
Yes, I’m a fan of Aero for this reason @noisesmith