This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-09-04
Channels
- # announcements (1)
- # babashka (41)
- # beginners (124)
- # cider (8)
- # clj-kondo (25)
- # cljs-dev (40)
- # clojars (4)
- # clojure (92)
- # clojure-europe (35)
- # clojure-italy (3)
- # clojure-nl (4)
- # clojure-uk (91)
- # clojuredesign-podcast (8)
- # clojurescript (41)
- # code-reviews (1)
- # cursive (11)
- # data-science (1)
- # datascript (76)
- # datomic (45)
- # emacs (4)
- # etaoin (3)
- # events (1)
- # figwheel-main (2)
- # fulcro (9)
- # graalvm (1)
- # jackdaw (6)
- # jobs (1)
- # jobs-discuss (3)
- # kaocha (4)
- # malli (25)
- # off-topic (42)
- # pathom (4)
- # reitit (11)
- # releases (2)
- # reveal (1)
- # shadow-cljs (53)
- # sql (4)
- # tools-deps (190)
- # vrac (19)
- # xtdb (6)
@emccue I'm pretty sure you can just say :scope "provided"
inside a dependency for Leiningen since it generates the pom.xml
and preserves that as I recall. On the other hand, there's no equivalent in deps.edn
(but you can of course add such dependencies into a pom.xml
file, generated via clojure -Spom
).
Is that answering the question you asked, or did I misunderstand?
You can do the above, but it's discouraged. In lein you should create a :provided profile.
https://cljdoc.org/d/leiningen/leiningen/2.9.3/doc/profiles#default-profiles mentioned here, but I can't find the document with rationale. Maybe because profiles can be toggled.
I make heavy use of ->>
when operating over a collection of data. I’m looking at something I wrote that has 4 remove
steps, 1 filter
step, followed by a map
and finally sort-by
. Assuming I’m using this code to process a sufficiently large set of data, what benefits (if any) would I see by switching to transducers here? My understanding is that a new data structure is being created at each of those steps when thread last is used, but with transducers, it’d just be one structure?
the number of elements is important too
transducers win as you increase both steps and seq size
if you're talking a seq of 10 elements or something, it's unlikely to matter
if you have 1000 it probably will
So, in that case should we just default to transducers then?
more words.... https://clojure.org/guides/faq#transducers_vs_seqs
I use both
a lot of times, the seq version is closer at hand and I know enough about the expected size that I still use seqs for a lot of stuff
from having done a lot of perf comparison of the two approaches, chunked sequences that you get with filter/map/remove are surprisingly efficient
So for something like this:
(->> coll
(group-by :key)
(map mapping-fn)
(into {}))
The transducer equivalent would be:
(into {} (map mapping-fn) (group-by :key coll))
Assuming a sufficiently large coll
and a sufficiently complex mapping-fn
is there any real benefit of converting this to a transducer?I can certainly see the benefit of converting my initial example (the one with 4 filters, a remove, and a map step) into a transducer, but would I see benefits for relatively simple examples like this?
more accurately, it would not create g to c
depends entirely on size and type of coll (whether it's something self-reducible). group-by, by it's nature, has to see all the input to do the groupings, so it's not really helped that much by being a transducer
not knowing anything else, I'd probably just write that in the seq form myself
Okay, this has been very helpful, thank you @alexmiller!
Just wanted to draw attention to this https://github.com/codesandbox/codesandbox-client/issues/704 in case any one has the energy for it
What’s an easy way to parse a file of newline-separated EDN dicts, where each individual dict may in turn contain newlines?
wrap it in []
and parse as an array of EDN dicts?
Or if it is too big, call https://clojuredocs.org/clojure.edn/read , then read a line, then read again
Good stuff, thanks! 🙂
Does anyone know how to use ring.util.response/partial-content
to stream a video file starting partway through (to allow seeking to unloaded parts of the video)? I've been using ring.util.response/file-response
.
it has been a long time since I've messaged with streaming video, but I believe that partial seeking behavior is a property of the video file itself, I vaguely recall having to add hints to mp4 files
I need to convert a double to float, sounds easy, but I got surprised here
user> (type 3.0861591259684055E172)
java.lang.Double
user> (float 3.0861591259684055E172)
Execution error (IllegalArgumentException) at user/eval6250 (REPL:15).
Value out of range for float: 3.0861591259684055E172
is there some .toFloat
or any alternatives to truncate the double to float?It seems to me that you would never be able to convert such a large number to a float, as it's greater than the maximum value a float can have (~3.4 x 10^38)
Just to make it more obvious - @U0CAUAKCG, there's E
in that number, which makes it quite huge.
You could check if the value you're converting is greater than the float limit, and then return the float limit if true.
(defn to-float [x]
(if (> x Float/MAX_VALUE) Float/MAX_VALUE
(if (< x Float/MIN_VALUE) Float/MIN_VALUE
(float x))))
ahhh i see, good idea. I'm processing double float audio buffers from one jna to another, so a performance is also a factor. I'll check
thanks @U016UGGNXB3 this works! It's a good safety net, otoh, just realised that this is a positive exponent not negative, so this is a huge number, which does not fit in the context of normalized audio signal from -1 to 1. So I'm reading wrong memory 😛
Anyone know why Rich declined this patch? (Improvements to cond) https://clojure.atlassian.net/browse/CLJ-200 Rationale: https://github.com/Engelberg/better-cond#rationale
I looked through the Google Groups archive and didn't see any responses from any of the core team on any thread where this has come up -- and it's come up in various forms half a dozen times since CLJ-200 originated (11 years ago).
I wonder if @alexmiller has any insight into this? (He commented last on that ticket, after it had been closed)
I think the general comment is that we don't agree those things are necessarily better or needed
but if you want them, then use better-cond
for me personally, I'd rather have less things in cond
:)
I have commented a few times on similar proposals that I just don't find better-cond
to actually be better -- I find it harder to read, but then I find doseq
and for
harder to read when they're full of :let
/ :when
etc 🙂
less is more in this area for me
JNA: callback object has been garbage collected
I seem to keep hitting this problem, is there any clojure trick for strong-reference?
My callback object can be local scoped, local scoped and refered to by multiple sources, and ns scoped (top level) atom, always within few seconds, my callback pointer is gc'd.
this would be the function I use to make the callback
(defn set-process-callback
[^JackLibrary$_jack_client jack-client callback]
(let [cb ^JackLibrary$JackProcessCallback
(reify JackLibrary$JackProcessCallback
(^int invoke [_ ^int nframes] (callback nframes)))]
(.jack_set_process_callback ^JackLibraryDirect jack-lib jack-client cb nil)))
and this could be the reference top-level
(def audio-callback (atom nil))
You might want to look into how this sort of issue is handled in libpython-clj: https://github.com/clj-python/libpython-clj
you can create a strong reference by keeping a reference to it. The easiest way might be something like:
(def callback-refs (atom #{}))
(defn set-process-callback
[^JackLibrary$_jack_client jack-client callback]
(let [cb ^JackLibrary$JackProcessCallback
(reify JackLibrary$JackProcessCallback
(^int invoke [_ ^int nframes] (callback nframes)))]
(swap! callback-refs conj cb )
(.jack_set_process_callback ^JackLibraryDirect jack-lib jack-client cb nil)))
@U7RJTCH6J that I already tried, at least with the toplevel atom. I would consider that strong too. So I'm equally confused. But afaik from reading, java considers anything not "final" a soft reference, and will gc all of them if it needs memory.
I was reading this here, https://blog.shiftleft.io/understanding-jvm-soft-references-for-great-good-and-building-a-cache-244a4f7bb85d also looking at cpython's bindings, entirely written in java, it seems I need to get java on this one
I use jna and a top level atom has worked every time
> But afaik from reading, java considers anything not "final" a soft reference, and will gc all of them if it needs memory. I think that only applies to java code. the garbage collector will not collect objects referenced in top level atoms or else that would break a ton of clojure code
I'm making another test with top-level atom. I'm not an jvm expert, but could it be that the jvm pointer can be thrown around while the native ones are constant? That could explain wild gc's behavior, but this is way above my knowledge on jvm
what's JackLibrary?
I'm only using the lowlevel/* for now, some functions there which I need which haven't been wrapped, also don't like OOP style wrapping anyway 😛
yea, I might just use jna directly
i've been using https://github.com/Chouser/clojure-jna/
I think libpython-clj has their own associated jna wrapper
so it happened again
(def callbacks (atom #{}))
;;under let
audio-callback (csound-process
csnd
jack-ports-in
jack-ports-out
ksmps-rate)
on-registration (fn [_] (reset! status :running))
on-unregistration (fn [_]
(stop @csnd)
(cleanup @csnd))
client-reg-cb (jack/make-client-registration-callback
{:on-registration on-registration
:on-unregistration on-unregistration})
]
(apply swap! callbacks conj [audio-callback on-registration on-unregistration client-reg-cb])
just few seconds inthe other thing that's easy to mess up is passing the wrong type. jna will automatically convert types, but if you pass 42
, that will be long when it might be expecting a float or an int
that is not enough
I don't think, anyway
I've been using jna directly, which will definitely crash when passed the wrong number type. I'm not sure how it works if you're using jna through some java stuff
now this is local scoped
audio-callback (reify JackLibrary$JackProcessCallback
(^int invoke [_ ^int nframes]
(loop [status (perform-ksmps @csnd)
and I store audio-callback in top-level atom
beofre I had a wrapper function around the reify
(defn set-process-callback
[^JackLibrary$_jack_client jack-client callback]
(let [cb ^JackLibrary$JackProcessCallback
(reify JackLibrary$JackProcessCallback
(^int invoke [_ ^int nframes] (callback nframes)))]
(.jack_set_process_callback ^JackLibraryDirect jack-lib jack-client cb nil)))
somehow someway, it causes gc :woman-shrugging:@U0CAUAKCG real quick and dirty solution
If there is no way for callbacks to be removed you don't have to worry about the semantics of repeated calls
I have a bit of an annoying transform that I would appreciate some help with. I built a nested reduce, and it mostly works, but it feels like there’s got to be a cleaner way. Input and desired output is as follows:
I've started using clojure.zip
for similar treewalking examples
what the nested reduce look like?
(apply merge-with into
(for [a [{:x {:id "123"}
:ys [{:type "Bar"
:y "42"}
{:type "Foo"}
{:type "Bar"
:y "43"}]},
{:x {:id "456"}
:ys [{:type "Bar"
:y "42"}
{:type "Foo"}
{:type "Bar"
:y "43"}]}
{:x {:id "789"}
:ys [{:type "Bar"
:y "99"}]}]
b (:ys a)
:when (:y b)]
{(:y b) [(:id (:x a))]}))
thanks @U0NCTKEV8