Fork me on GitHub
#clojure
<
2021-04-07
>
ghadi00:04:22

it's sad because it puts the chained exception within the ex-data in addition to chaining it as the ex-cause I end up doing everything Alex mentioned minus setStackTrace

ghadi00:04:03

preserve the chained cause & ex message, but dissoc stuff from the data

ghadi00:04:58

for that case it doesn't matter that the outermost exception's stacktrace differs, the real meat of the stacktrace is on the chained cause

frozenlock00:04:59

aaah, I was trying to set! the data field (which obviously didn't work), but .setStackTrace might do it. Especially if it's as simple the example @hiredman pasted.

frozenlock00:04:49

Thank you all very much!

Michael Lan01:04:34

Within a REPL session, is it possible to add a new dependency? Using Deps and CLI.

Alex Miller (Clojure team)01:04:16

there is an experimental add-libs3 branch in tools.deps.alpha that provides this functionality. we expect it to eventually be included but there are some key integration questions we still have. Sean Corfield's deps.edn has some setup info for this https://github.com/seancorfield/dot-clojure

seancorfield01:04:44

That lets me add all of next.jdbc’s test dependencies into a running REPL that was started from another project (that may well depend on next.jdbc, but without those deps). This lets me run next.jdbc’s tests from my editor, even when working on another project.

Michael Lan01:04:58

Got it, so it’s not a permanent addition, just temporary

seancorfield01:04:44

It just loads libs into the REPL. If you want them present the next time you start the REPL, you need to add them to deps.edn.

seancorfield01:04:09

In one of my RDD talks I show how you can edit deps.edn to add dependencies and then call add-libs on that same hash map from within deps.edn (by having some code in deps.edn that is normally commented out).

seancorfield01:04:43

(I am tempted to automate that so I can just edit deps.edn, put my cursor just inside the :deps map or :extra-deps map, and hit a hot key and have add-libs called on that…)

Michael Lan01:04:23

haha, that sounds really convenient

seancorfield02:04:26

Just added it to https://github.com/seancorfield/vscode-clover-setup — you need the :add-libs alias from my dot-clojure repo active when starting a REPL and then you can just edit the :deps or :extra-deps in your deps.edn file and with cursor inside the lib spec hash map, ctrl-; shift+a and it sends it to add-libs and loads those dependencies!

Yehonathan Sharvit06:04:02

When I have a namespace with a couple of defmethod declarations, what’s the best way to make sure the namespace is loaded (I mean required)? For instance, let’s say I have a library whose main ns is my-lib.core and the defmehtod declarations are in my-lib.foo Should I write something like?

(ns my-lib.core
   (require my-lib.foo))
The problem is that when someone looks at the ns form, my-lib.foo seems superfluous. And in fact cider-refactor removes it as it considers it as a unused libspec vector

flowthing06:04:52

I'd like to write code that behaves differently depending on the user's Java version. More specifically, I'd like to write a function that uses java.lang.module.ModuleFinder if it's available, and just returns nil if not. Any general pointers on how to approach this? If you can point me to any prior art on this topic, I'd appreciate it.

flowthing07:04:51

Thanks! I'll look into that.

flowthing07:04:17

Thanks! That looks nice and straightforward.

Alex Miller (Clojure team)12:04:47

Note that that code doesn’t exist in core anymore

Alex Miller (Clojure team)12:04:15

But feel free to borrow it

flowthing12:04:48

Sure. I borrowed the gist of it, that got me there.

flowthing12:04:19

https://github.com/eerohele/Tutkain/blob/30578b438d6a57be25c83d24f79de281b79ffbd7/clojure/src/tutkain/repl/runtime/completions.clj#L137-L150 The mapcat fn still reflects, and I'm not sure how to hint it when java.lang.module.ModuleReference is not guaranteed to exist, but it's not really a big deal.

Alex Miller (Clojure team)12:04:31

you could use a fn and type hint the argument

flowthing12:04:42

Yeah, that's what I'd like to do.

flowthing12:04:58

I figured I could maybe use the same kind of type hint as with string arrays etc., but it didn't seem to pan out, or then I just didn't get it right. The "syntax" is a bit fiddly.

Alex Miller (Clojure team)13:04:36

I think the compiler is going to lead you into trouble in that case instead

craftybones07:04:35

What is the cleanest/most idiomatic way of taking an existing map with plain keywords and turning it into a map with namespace qualified keywords?

{:a 1 :b 2} => {:foo/a 1, :foo/b 2}

craftybones07:04:18

I am using rename-keys which works, but I am positive there’s a better mechanism for this

craftybones07:04:15

There was a reduce-kv based solution on stackoverflow which is similar to my rename-keys approach…

simongray07:04:27

@srijayanth creating a map-vals or map-keys is like an initiation ritual for a Clojure developer since they don’t come with the standard library… 😛 I like to for to iterate through the key-value pairs and then into to create the new map

simongray07:04:41

(into {} (for [[k v] {:a 1 :b 2}]
           [(keyword 'foo k) v]))

simongray07:04:00

np. I wrote that piece of code so damn many times

craftybones07:04:08

fyi, as noob as my question sounds, I’ve been using Clojure since 1.2 😄

craftybones07:04:54

I still don’t get why rename-keys is in clojure.set. I remember somebody providing an explanation that seemed adequate at that time

Yehonathan Sharvit08:04:21

It’s because rename-keys is part of relational algebra and the natural way to represent a relational in Clojure is as a set of maps.

simongray07:04:55

don’t worry, I’ve never even used a transducer… 😛

craftybones07:04:21

I love transducers. I wrote this simple version of 2048 that uses transducers

simongray07:04:54

yeah I know they’re great, I just don’t like how it makes my code look and I never had any real performance issues, so I never bothered to use them 😛

simongray07:04:43

probably should get over it and use transducers everywhere applicable

craftybones07:04:27

Even without performance issues, they come in handy.

craftybones07:04:48

It is a lot easier to compose transducers and pass them around

simongray07:04:22

right… do you have a recommended article or something for me to get into the right mindset?

simongray07:04:12

I’ve seen them around in codebases I’ve been working on, so I’ve edited them… just never ran into a situation that made me think “that calls for a transducer!”

jumar07:04:33

Btw. for map-vals et al I really like to use medley library. It also has map-keys:

(map-keys #(keyword "foo" (name %)) {:a 1})
;;=> #:foo{:a 1}

simongray07:04:13

@jumar it’s such a tiny bit of code that pulling in a library just seems overkill 😉 but I guess if you use medley for other things, it makes sense

jumar08:04:54

yeah, I use map-vals , find-first, index-by and assoc-some most frequently. What I like about map-vals in particular is that it has a lot more meaningful name than just using the idiom (although when you get used to it it's also quite readable) - another aspect could be performance but I don't care about that one in most cases.

craftybones07:04:52

@simongray - when transducers came out, I was a little befuddled about how they work. There’s a Rich Hickey talk where he exposes the insides of a transducer(it isn’t the StrangeLoop one I think). After that, every time I see code that has a series of 2 or more map/filter/reduce/iterative constructs, I eagerly convert them to transducers

craftybones07:04:54

We once had a discussion here about any scenario where a transducer might not be preferred. While someone pointed out something around eagerness/laziness etc, I came away feeling that there’s very few downsides to using transducers at all

craftybones07:04:53

For instance, here’s the transducer chain for that 2048 bit

(def xform (comp (remove zero?)
                 (partition-by identity)
                 (mapcat #(partition-all 2 %))
                 (map (partial apply +))))

craftybones07:04:25

And I end up using it as follows

(defn move-left [row]
  (take 4 (concat (sequence xform row) (repeat 0))))

craftybones07:04:51

If you see the xform, it reads incredibly clearly from top to bottom

craftybones07:04:15

remove zeroes, partition equal numbers together, chunk them in 2s and sum them up

craftybones07:04:51

I eventually wrote my own stateful transducer that pads the ends of collection with zeroes. I stuck that transducer to the end of the chain

simongray08:04:24

thanks a lot

simongray08:04:57

I guess I just need to get into the habit of converting most functions that contain a threading macro into composed transducers

craftybones08:04:29

Yeah, my personal limit is more than 2. if there’s more than 2 maps/filters in sequence, then that’s an ideal candidate for transducers

craftybones08:04:40

It really is a shame that not enough people use it. I find it awesome

3
craftybones08:04:58

I transduced the hell out of those advent of code problems

jumar08:04:09

I like this discussion about when to use transducers: https://groups.google.com/forum/#!topic/clojure/JjiYPEMQK4s I think it's very helpful to understand the use cases even without knowing how exactly they work. Especially Alex Miller's points: > I would say transducers are preferable when: • 1) you have reducible collections • 2) you have a lot of pipelined transformations (transducers handle these in one pass with no intermediate data) • 3) the number of elements is "large" (this amplifies the memory and perf savings from #2) • 4) you put to produce a concrete output collection (seqs need an extra step to pour the seq into a collection; transducers can create it directly) • 5) you want a reusable transformation that can be used in multiple contexts (reduction, sequence, core.async, etc) 

👍 3
simongray08:04:59

I rarely need reusable transformations, but maybe I just haven’t looked hard enough.

simongray08:04:20

that’s what I mean about the mindset. Gonna scan that thread for some rules of thumb.

simongray08:04:39

but obviously memory savings is advantageous too

simongray08:04:02

and @srijayanth is right in that lazy collections are rarely called for in practice

craftybones08:04:25

They are useful as hell though 🙂

simongray08:04:28

I can think of maybe 4-5 cases where I actually use lazy (usually infinite) collections

simongray08:04:10

and I’m pretty sure I only remember them because they are noteworthy for being the rare infite colls

craftybones08:04:35

I end up using cycle a fair amount. It is sometimes surprising how often that pattern shows up

craftybones08:04:07

You can make pretty cool spinners with a simple cycle

simongray08:04:33

yeah, in one case I use cycle to take N items from a randomised colour selection

simongray08:04:47

to colour some tabs in a CLJS lib I make

simongray08:04:16

and then iterate is another common case of infinite colls

craftybones08:04:31

(cycle "|/-\\")
A nice classic spinner 🙂

simongray08:04:40

hah, nice one

craftybones08:04:09

the other way is to have different css styles/classes cycled

craftybones08:04:42

You can also make those annoying prehistoric marquees from the geocities days

nilern09:04:48

Transducer chains aren't much harder to read than ->> chains so I would also prefer transducers even for not so performance sensitive code Premature optimization is one thing but wasting cycles for no good reason is another Like after Java 8 people realized that it makes no sense to go converting most loops to Streams because you probably lose 20-30% on performance and simple loops read just fine

Ben Sless11:04:12

Transducers also usually get you to stop and think for a few seconds, which is always good when programming. How many times did you find in code review someone used flatten when mapcat would have been fine? I often see it pop up when there's some confusion. While worrying about optimization prematurely is silly, so is shooting oneself in the foot from the onset

nilern11:04:36

Yeah flatten is often a red flag and not because it is slow but because is a deep flatten even correct...

Ben Sless12:04:15

yup, like i said, it's usually a place where mapcat should be used, but somewhere the dev lost control of the return type - I saw it recently with a function which returned a sequence of maps, simplified some things, switched to mapcat and broke a test, because the test mocked the function to return a map instead of a sequence of maps. That function was then mapped on the input sequence, which made flatten work and mapcat not. It always feels good to fix wrong assumptions in tests on top of offensive functions

Ben Sless12:04:03

whenever I see flatten I read "I'm not sure what's going on here, yolo"

raspasov14:04:01

Personally, if there’s more than one transformation happening, I switch to transducers.

☝️ 3
simongray10:04:10

@nilern you’re saying premature optimisation is a good thing?? 😛

nilern13:04:07

Not being wasteful by default is quite far from needless micro-optimizing or complicated algorithms

simongray10:04:33

anyway, you’re probably right

simongray10:04:19

I actually thought that Java 8 streams were kinda transducer-like, i.e. they were more performant than regular loops

simongray10:04:08

but obviously it’s not something I ever researched very deeply 😛

simongray10:04:14

like I seem to remember something something parallelism, laziness, blablabla

restenb12:04:35

so how do you deal with dependency cycles? I have this stateful "core" of what i'm writing at the moment, and an external service living two namespaces away, each depending on the previous. the service then needs to update the state. enter dependency cycle.

delaguardo12:04:05

you can extract everything related to state, including functions to update, into separate namespace and then require it from every service and the core.

noisesmith18:04:48

another option is to write code in terms of a protocol, have a shared protocol namespace, and a separate implementation namespace that most consumers never need to care about

nilern12:04:41

If there is a lot of data you can utilize more cores with .parallel() But in the single-threaded case transducer/Stream pipelines only beat handwritten code if it is too gnarly to do everything in a single handwritten loop so intermediate collections are added

nilern13:04:07

Not being wasteful by default is quite far from needless micro-optimizing or complicated algorithms

raspasov14:04:50

What’s better:

(vec
 (map-indexed
  (fn [idx x] [idx x])
  [:a :b :c]))
;or
((comp vec map-indexed) 
 (fn [idx x] [idx x]) 
 [:a :b :c])

raspasov14:04:26

(assuming I’m not using transducers)

localshred14:04:51

beauty is in the eye of the beholder 🙂

👌 3
raspasov14:04:52

Goal is to have a vector at the end.

nilern14:04:20

In that case comp is just pointless obfuscation

🆗 3
jjttjj14:04:29

Imo probably the first one, but there's also

(into []
  (map-indexed (fn [idx x] [idx x]))
  [:a :b :c])
(I'd still probably default to just wrapping the map-indexed call with vec)

👌 6
p-himik14:04:51

user=> (reduce-kv (fn [acc i v] (conj acc [i v])) [] [:a :b :c])
[[0 :a] [1 :b] [2 :c]]

raspasov14:04:51

Do you prefer that over all other options, in terms of readability or even shortness? Or you’re just showing other options 🙂

raspasov14:04:14

Or is this faster? (I haven’t done benchmarks on reduce-kv)

p-himik14:04:58

In terms of readability, I would not think about it at all and just extract it into a well-named function. :)

👌 3
nilern14:04:37

If you really want to optimize (persistent! (reduce-kv (fn [acc i v] (conj! acc [i v])) (transient []) [:a :b :c]))

p-himik14:04:40

into does exactly that. :)

nilern14:04:49

It uses normal reduce but yeah

nilern14:04:30

If the goal was to produce a map then reduce-kv can avoid allocating those kv pairs

👍 3
p-himik14:04:01

Vectors are associative collections from an index to a value.

nilern14:04:12

(into [] (map-indexed vector) [:a :b :c])

p-himik14:04:38

This gives a wrong result though. Ah, the edit. :)

nilern14:04:02

Sorry about that

nilern14:04:25

A map result could also be handy (zipmap (range) [:a :b :c])

raspasov14:04:05

I’m looking for the neatest code for an indexed vector for React components; map is no-go there.

raspasov14:04:25

Needs order.

raspasov14:04:52

And idx accessible on each one.

nilern14:04:16

In that case, not handy

raspasov14:04:12

In any case, thanks for the input!

craftybones16:04:05

(mapv vector (range) [:a :b :c])

craftybones16:04:59

But beware that mapv is eager

Alex Miller (Clojure team)16:04:12

It was in #announcements yesterday

craftybones16:04:13

Fogus wrote this one. I am going to read tea leaves and say that Alex was busy getting spec 2 ready. Sometime this week? 😉

🙏 9
borkdude16:04:21

Thanks for publishing the survey and the write-up @fogus!

craftybones16:04:09

Urgh:

user=> (keyword "emp.id" (str 100))
:
user=> :
Syntax error reading source at (REPL:2:0).
Invalid token: :

craftybones16:04:08

I can obviously write a fn to normalise this, but is there any real way to deal with numeric keys?

Timur Latypoff16:04:30

I remember someone from the Clojure team was saying that keywords should not start with a digit (like symbols), but the ability to construct them has been retained for backward compatibility.

craftybones16:04:57

The problem I have is that I am receiving a json that I am then namespacing

craftybones16:04:09

As I said, I can always use a fn to prefix something

Timur Latypoff16:04:40

You can use non-keyword keys (string or integer keys) just fine, if that works for you @srijayanth

craftybones16:04:15

yes, I’ll just add a prefix

Michael Gardner18:04:28

I seem to recall Rich saying that he might've used transducers as the basic abstraction for Clojure's collection APIs instead of seqs, if he'd thought of them back then. Am I remembering correctly? I'm very curious what that would look like, if so.

nilern18:04:45

Sounds strange. Sometimes you really need first and rest -- https://www.metosin.fi/blog/malli-regex-schemas/ comes to mind.

nilern18:04:58

Parsing style usage is an extreme case but stopping processing early or processing multiple collections gets awkward. There are solutions like reduced or various flatmapping and zipping operations but even if the little extra allocations go away after inlining the pipeline abstraction stops being a friend at some point. I have used a lot of https://odis-labs.github.io/streaming/streaming/Streaming/Stream/index.html in OCaml but Java 8+ Streams seem similar.

raspasov18:04:47

@U01R1SXCAUX Yes I clearly recall this from a talk; I can’t remember which one but that sounds about right;

raspasov18:04:09

@nilern I am not so sure about rest ; perhaps it’s a coding style, but I don’t really use it often in day-to-day code; It does feel pretty low level to me (as in, not something I’d default to);

flowthing18:04:42

From the History of Clojure paper: > I think transducers are a fundamental primitive that decouples critical logic from list/sequence processing and construction, and if I had Clojure to do all over I would put them at the bottom.

nilern18:04:16

Already in my PHP days I noticed that web apps are mostly a bunch of map, filter and doseq; even reduce feels quite unusual. Libraries and compilers are not so straightforward.

nilern18:04:29

https://github.com/pixie-lang/pixie put transducers more at the bottom; but it still has seqs

raspasov18:04:44

I don’t recall him saying he wouldn’t have seqs; I remember it was a short comment/sentence rather than a fully articulated argument; Don’t want to speculate on what the exact idea of transducers vs. seqs would be;

raspasov18:04:04

@nilern My language before Clojure was also PHP!

raspasov18:04:32

The 2nd best! 😝

🏆 3
nilern19:04:00

> I was taught assembler in my second year of school. > It's kinda like construction work — with a toothpick for a tool. So when I made my senior year, I threw my code away, > And learned the way to program that I still prefer today.

😄 3
Michael Gardner19:04:10

yeah, that History of Clojure quote is likely what I was thinking of. I'll take a look at pixie

Ed19:04:21

I think he said something along the lines of "I wouldn't have made sequences lazy by default if I'd thought of transducers first"

3
Ben Sless19:04:00

dedupe is a good example of how things would have looked differently had transducers been around from the start (probably)

Ben Sless19:04:26

The arity which accepts a collection is just:

([coll] (sequence (dedupe) coll))

blak3mill3r23:04:05

I thought it might've been in this talk, not sure https://www.youtube.com/watch?v=4KqUvG8HPYo

blak3mill3r00:04:57

@U01R1SXCAUX regarding: "I'm very curious what that would look like, if so." IMHO: In this fantasy, not too much would be different about the Clojure core (some of it could've been elided, though). clojure.core/into with a transducer is a great way to build a collection today (perhaps even The Best Way), but seqs are still more 'within reach' in some way that involves momentum the community has thinking with seqs, and also plenty of code written that way, more than it involves anything about the shape of the core libraries today. My intuition is that, in this fantasy world, when building collections almost everybody would be reaching for into + transducers now, instead of the ->> full of seq transformations, that seqs would be much less common, that the core would be a little smaller without e.g. the non-transducer arities of map,filter plus the core.async versions of the same... So the core could've been a little simpler that way, but I suspect we're not missing out on much.

Jan K20:04:17

Sometimes the transducer solution forces you to hide mutable state in them, where with reduce you could just use the state argument, which feels like "cleaner" FP. I wonder if it would be possible to reinvent transducers so they would get a state argument passed in (like reduce) to somehow avoid mutation inside.

nilern20:04:41

Might as well slap an arbitrary monad in there while you are at it 😜

jjttjj20:04:49

You could replicate something like that with a transducer like the xform library's reductions https://github.com/cgrand/xforms/blob/62375212a8604daad631c9024e9dbe1db4ec276b/src/net/cgrand/xforms.cljc#L491

lilactown20:04:09

you would need the reducing machinery to be aware of the state attached to the pipeline of transducers

vlaaad06:04:06

If a local state is mutated in the forest and no one is around to observe it, does it make a sound?

😄 6
nilern07:04:36

The state is not as encapsulated (in the sense of the ST monad) as say, the transients inside into but I think transduce, into et al. do encapsulate it so it is only a problem when calling transducers directly which basically only happens in library code

didibus01:04:07

I do think that state in transducers does kind of make it harder to have a parallel transduce no? similar to the reducers fold?

nilern13:04:11

I guess so although things like drop are not parallelizable anyway

3