Fork me on GitHub

Is anyone aware of plans to improve the performance of sets? I am thinking about lack of support for a specialised IReduce implementation and perhaps an array backed implementation when they are small like maps? I could not find anything on jira...


I feel a bit like I'm punished for using sets sometimes πŸ™‚


#clojure-dev is probably a better place to ask

πŸ‘ 3

I'll be interested to hear responses to this from the core team (because the background to these decisions is always fascinating).

Alex Miller (Clojure team)20:08:07

Do you have an example use case?

Alex Miller (Clojure team)20:08:01

Or a particular perf issue?

Alex Miller (Clojure team)20:08:39

Sets are really maps (of every element to itself) so they actually get the benefits of a lot of the map impls

πŸ‘€ 3

I understand that set performance is not going to compete with vectors in the general case for say conj, but it could come a bit closer (like maps) when they are small. My use case was maintaining a group-by style index as part of a library, with sets as the collection as opposed to vectors. It is much easier to maintain such an index generally if the collection is a set - but the performance is of course not as good. Such indexes must support many different set sizes, but I suspect often they will be very small. Library WIP here: Any benchmarking i've done is very much napkin but it seems there is a substantial difference in small map vs small set perf for conj at least. Just got me interested in whether there was appetite for a ticket.


In principle a completely custom PersistentArraySet implementation (as opposed to on top of maps) might see gains, do not need to store the values twice. Though I guess its another class for the JIT to optimise.


Basically I might actually use a map rather than a set just to get the arraymap πŸ™‚


@U0GE2S1NH Can you give a bit more context on where you feel the need to have high-performance reduce on sets, and what problems you're trying to solve with that? I'm not getting the big picture from your comments above...

βž• 3
Alex Miller (Clojure team)22:08:39

You could certainly experiment with your own impl - it’s easy enough to implement IPersistentSet and the other necessary interfaces


My main issue was the small set problem, where it seems better to use maps-as-sets in some cases. To be fair the IReduce was just a hunch that after a bit of REPL investigation does not seem to really matter, in fact reduce-kv on maps seems slower than iterable reduce with entry packing and unpacking for set, despite being backed by same structure. So call me surprised. SORRY FOR WASTING YA'LL time with uninformed speculation on IReduce πŸ˜…


I have some files in resources/markdown that I would like to package in an uberjar like you would for any static resource. I'm finding that the assets in resources/public e.g. CSS files are getting deployed but the ones in resources/markdown are not. I'm using deps.edn and to package stuff, no Leiningen. What am I missing?


hmm, or maybe I'm wrong that it's not getting packaged:

$ jar tf target/rtc.jar |grep markdown
So maybe what I'm not understanding is where the file lives at runtime. What I'm doing now is throwing a FileNotFoundException:
(slurp "resources/markdown/")


@UH85MNSKE, use (slurp ( "markdown/"))


^^ the above will work at dev and runtime


I knew it was something silly like that! Probably won't be the last time πŸ™‚

😁 3

Anyone run into an issue with destructuring not liking the namespace focus?

;; This throws defn did not conform to spec
(defn my-fn [{:focus/keys [a]}])

;; These all work
(defn my-fn [{:focus/keys []}])
(defn my-fn [{:focusa/keys [a]}])
(defn my-fn [{:afocus/keys [a]}])
(defn my-fn [{:ocus/keys [a]}])


(defn my-fn [{:focus/keys [a]}]) works fine for me. does it work in another ns?


thanks for checking. You're right. I have a spec

(s/def :focus/keys (s/coll-of keyword?))
which is interfering somehow. I can define the function fine in a fresh repl outside of the current project.


weird. i'm a total spec noob so I don't know what might be the issue with your example.


it seems like your example should work without issue

Alex Miller (Clojure team)21:08:56

This is actually a known issue specifically with specs that have the name keys being accidentally applied in the destructuring

πŸ‘€ 3
Alex Miller (Clojure team)21:08:14

There is a ticket for it