Fork me on GitHub
#clojure
<
2023-07-13
>
Jason Bullers02:07:02

So I was thinking about map and the variable number of collections you can pass to it as a result of some interesting discussion in an earlier thread: https://clojurians.slack.com/archives/C03S1KBA2/p1688751822392039 In that thread, OP asked if anything else in Clojure work that way, and also pointed out that map works differently than in other languages in this regard. I don't really have an answer to either of these points (am curious too, though), but it leads me to another question: From what I can tell, it seems like map is kind of doing double duty as both a "traditional" map function and also as a sort of generalized zip function. Does anyone have any insight into why this would be the case?

Bob B02:07:52

I guess, to me, map is a specialization (`zipWith1` as it were), so if it's possible to do zipWith* with one function, why not?

Bob B02:07:33

or, for example, the F# List module has zip and zip3 for tupling, but it calls zipWith map2 and zipWith3 map3

Jason Bullers02:07:26

I've heard Stuart Halloway talk before about key Clojure philosophies being simplicity, power, and focus. If I understand right, these kind of point toward really boiling things down to their essence. Arguably, zip is "specialized" in that, at least in languages I've seen it in, it accepts two collections and produces zipped tuples. There's two axes then for generalizing: number of collections, and zip function. If you did both those things, which I feel would be a pretty "Clojure" thing to do, then you'd have something like (zip zip-fn & colls) and I suppose at that point, is it really any different from (map map-fn coll) to warrant a whole other function? Conceptually, zipping is mapping, so I suppose that's the reason to combine them?

Bob B02:07:16

well, zip would be (partial map vector), right?

Jason Bullers02:07:24

@U013JFLRFS8 ah, interesting about F#. I only did a quick Google for Haskell and used my fuzzy recollection of Python's zip as my data points

Jason Bullers02:07:21

Right, in the tuple case, (partial map vector) would do it (and still be more general than impls that always expect two input collections)

Jason Bullers02:07:51

But that's only true because map allows varargs on the collections provided

Bob B02:07:39

and at least in haskell, f# and python, zip is exactly tupling, right? in haskell, zipWith is for an arbitrary function (which is map2 in F#) disclaimer: I think these are equivalent, I don't do a lot of F#, and I actively avoid haskell

Jason Bullers02:07:33

Actually, I misspoke about Python earlier only taking two lists: it takes any number of iterables, but produces tuples for each, so effectively what you'd get using vector as the zipping function as you said above

Bob B02:07:40

right... I was probably unclear before when I said map as a specialization - I meant that in haskell it's arguably an inconsistent name - if zipWith is map with 2 collections and zipWith3 is map with 3 collections, then maybe map should be called zipWith1 (not really, but that's what I meant by map being a specialization of zip for only one collection)

2
Jason Bullers02:07:44

and yeah, just searched and found the Haskell zipWith... interesting. No varargs option though, which maybe would be too ridiculous to model types-wise

Jason Bullers02:07:02

So you think that's it then? They just aren't different enough to warrant both existing in core?

seancorfield02:07:33

If your language is curried -- all calls take one arg -- then producing a vector from arguments and using it as the one arg for the "mapping" function makes sense.

Jason Bullers02:07:35

A more generalized map with varargs colls gets you best of both worlds with less surface area

Bob B02:07:48

well, that and mapping vector might be a bit too specific maybe, because maybe you want to map list or set or sorted-set or ... I'm no authority, that's just what comes to mind for me

seancorfield02:07:08

Right, Clojure's map is the more generic approach...

seancorfield02:07:25

(but it's only possible due to varargs, vs curried arguments I think)

Jason Bullers02:07:47

@U013JFLRFS8 Yeah, but then you could have: (map f coll) (zip f & colls)

seancorfield02:07:57

(hence map2/map3 or zipWith1, zipWith, zipWith3 elsewhere)

seancorfield02:07:25

(apply map f colls) 🙂

Jason Bullers02:07:29

Would that be "less surprising"? That was kind of a point raised in that other thread that reaching for the multiple collection version of map isn't super intuitive as an idiom for transposing

Jason Bullers02:07:57

@U04V70XH6 that's what started this whole rabbit hole for me 😆

Bob B02:07:17

but that sort of assumes that you're coming from a context that has zip/zipWith, right?

Jason Bullers02:07:47

It sounds like a common enough thing in other languages, though. It's just interesting that they draw that distinction while Clojure doesn't

seancorfield02:07:40

(just read that other thread)

seancorfield02:07:16

@U04RG9F8UJZ Both F# and Haskell are curried by default tho' right? Everything really has one argument (or can be invoked with one argument and returns a function of the others)...

seancorfield02:07:56

So you have to name the arities explicitly (to know when you've consumed all arguments). And those languages don't have full varargs (they can't).

seancorfield02:07:42

Clojure's map is possible because of both varargs and arity overloading -- which conflicts with currying.

Jason Bullers02:07:05

Right, so that explains things being limited to two or three collections in those languages.

Jason Bullers02:07:16

Hmm, I looked a bit further at Python, and Python has: map(function, iterables) and zip(iterables) so they draw a semantic difference between the two, even though presumably you could just implement zip in terms of map

seancorfield02:07:54

That first one takes a function and a tuple of collections yes?

Jason Bullers02:07:36

varargs I believe

seancorfield02:07:08

(I'm not familiar enough with Python but it does have varargs and "apply" (or * for spread now) I think, so Python could have map like Clojure...)

Jason Bullers02:07:10

Yeah, Python's map is like Clojure's signature wise (minus the transducer 1-arity)

Jason Bullers02:07:21

But I guess for Python, zipping n iterables to tuples is a common enough use case that it was worth a separate function. Both are built-ins

Jason Bullers02:07:56

vs Clojure where you can apply map vector colls if you really need it

Jason Bullers02:07:44

Which is what prompted this whole train of thought: why have to do that rather than having the multi-arity zip and a single coll map (since from the other thread, I got the feeling that single coll map is the more common use case and multi-coll -- effectively zip-- isn't particularly intuitive)

Jason Bullers02:07:57

And of course, there's no-one that lives in Rich's head but Rich, so I was just fishing for "philosophical" supporting (or dissenting) arguments

Jason Bullers02:07:24

I guess maybe the core team in general and Rich in particular have a certain threshold for how "different" two things need to be in order to warrant that both be in core? And beyond different, I guess how clunky to implement with what's already there

Jason Bullers02:07:57

Although: why mapcat then? 😆

seancorfield02:07:12

If you have apply and (multi-coll) map, zip is "trivial" to write 🙂

seancorfield02:07:19

mapcat requires two apply calls so it's not as "trivial" as zip?

Jason Bullers02:07:03

I suppose that could be crossing some ergonomic threshold. It is a bit awkward for sure

seancorfield02:07:08

I don't tend to question core decisions too much -- after all these years, I have heard enough really good justifications that I tend to start from the premise that "they thought about it really hard") 🙂

😄 2
Jason Bullers02:07:23

Do you personally find apply map f colls intuitive and have you used it to transpose collections before/often?

didibus02:07:24

I love that map takes more than one coll personally. What I hate is that it's kind of lonely doing it.

seancorfield03:07:01

We don't have apply map anywhere in our 138k lines of Clojure at work... but, yeah, it feels fairly intuitive to me, seeing it written down.

didibus03:07:23

Like why run! doesn't do the same 😞

didibus03:07:15

Though some of these might get weird when the colls don't have equal elements

Jason Bullers03:07:29

How might reduce work?

Jason Bullers03:07:42

The reducing function would take the acc

Jason Bullers03:07:49

and then one arg for each coll, I guess?

seancorfield03:07:51

The reducing function would need to take n + 1 args, for n colls.

Colin (fosskers)03:07:33

I scanned the responses here but didn't see anyone mention the obvious: mapcar in Common Lisp works this way.

Colin (fosskers)03:07:39

You can pass it as many collections as you want.

didibus03:07:39

Ya, either that, or it could implicitly give you a tuple of them

Jason Bullers03:07:19

I don't know any other lisps 😆 good to know

👍 2
Jason Bullers03:07:46

does CL have a separate zip?

Colin (fosskers)03:07:49

Related: one thing that surprises me about transduce is that it can only accept one input source.

Jason Bullers03:07:41

We've found a winner for precedent

Colin (fosskers)03:07:12

Common Lisp has basically all the usual FP operations we expect from other langs, but often the names are different

Jason Bullers03:07:05

@U0K064KQV yeah, thinking on some of the other core functions, it seems like they could have done well with the same treatment. Especially run!

Colin (fosskers)03:07:18

Clojure brought all the names "back home" so to speak

👍 2
seancorfield03:07:06

sequence is the multi-input transducing function, yes? (producing a lazy sequence of results)

Colin (fosskers)03:07:49

That's half-way to what I'd want, since it only produces lazy seqs and doesn't let you apply your own reducers.

👀 2
Jason Bullers03:07:53

So it seems we've traded one question for another: we seem to have settled on "why no zip ?" being because "it's not useful enough to exist because varargs map", and replaced it with "why not more vararg seq functions?" Progress?

Jason Bullers03:07:24

@U058DHAN3UP interesting... sequence feels a bit more like map in a sense, while transduce is like reduce. The multiple colls follows the same "rules" there

seancorfield03:07:31

Not sure whether it's a useful data point or not but, in our 138k line code base at work, we have precisely one mapv vector and one map vector which would represent cases where we could otherwise use "vararg seq functions"...

2
seancorfield03:07:34

(there aren't very many apply calls in our code either -- some apply hash-map (7), apply = (8), apply dissoc (7), apply merge(-with) (6), and just a few others)

2
Colin (fosskers)04:07:54

I have implemented Transducers in other Lisps, and did succeed at a multi-arg transduce implementation, so it's certainly possible.

Colin (fosskers)04:07:23

I'm not sure yet how it would interact with Clojure's optional init value that can be passed as the second-to-last arg.

Colin (fosskers)04:07:35

The other implementation don't do that

Colin (fosskers)04:07:59

Clojure's does (I think) primarily because conj is polymorphic

didibus04:07:29

Wait... no zip? Hum, didn't realize that. True, there's zipmap, but no zip to zip into sequences

didibus04:07:43

So... now I think that the multi-arity of map and mapv are probably meant to replace zip. They're more powerful cause you can choose the type to zip into. I wonder what type a default zip would use? Zip into list? vector? or set? This would also explain why run! and reduce don't support this, because if it was meant to provide "zip-like" behavior, something that just does side-effect or reduces isn't zip-like.

Jason Bullers12:07:10

Arguably run! would be zipping collections into function arg tuples, but maybe that's a stretch. @U064X3EF3 any nuggets of insight from the inner circle?

Bob B12:07:31

I think that might be part of it - if there were a default zip that usedhttp://e.gaa vector, then this conversation would be about 'why isn't there a ziplist, and a zipset, and eager versions?'

😂 4
Jason Bullers13:07:35

Right, but there still could have been a generalized zip where you provide the zipping function separate from a single coll map function. And then we wouldn't all be wondering why map got the special varargs treatment but not other functions like reduce or run!

didibus18:07:54

That zip would effectively be the same as the current map no?

Jason Bullers18:07:32

It would, but having two functions might offer some semantic clarity: map is about mapping one collection to another by applying a given fn, while zip is about putting together elements of each of the given n colls. Presumably then there isn't such a question like "if I can map over n colls, why not run! or reduce?" I suspect this may be just a case of "it is because it is", and a tension between "what's the point of map and zip being separate" vs "how pervasive should varargs colls be in seq fns". I was just wondering if there's something missing here that makes it more complicated or nuanced. One of those major lightbulb moments where you realize it's not that simple. I'm totally okay with "it was useful to combine map and zip, but there's not a strong enough use case for varargs on the other fns", if that's the answer. The asymmetry makes it a brain itch I want scratched. Fun discussion 🙂

👍 2
seancorfield18:07:39

Back when I was doing my PhD on FP (early/mid-80s), one of the experiments I did was to explore what "primitive" FP functions might look like. map can be built on chn -- cons, head, nil -- it has the structure of map but also passes in the cons fn, the (f (first l)) style function, and... I think n was the nil value to use (for the cons) at the end of the list or maybe the fn to test for nil at the end? I can't remember 40 years later. Then you can make that more generic by abstracting a few more pieces away, and I called that function r (as a general recursive function). Using r you can build chn and therefore map but you can also build reduce and a bunch of other useful functions.

seancorfield18:07:50

Oh, here we are... I talked about it back in 2016 here in #beginners 🧵 starts https://clojurians.slack.com/archives/C053AK3F9/p1459096805000610

👀 2
seancorfield18:07:51

It includes a stab at the r function from memory: > I think r was something like this (defn r [p c h t n l] (if (p l) n (c (h l) (r p c h t n (t l)))))

🤯 2
seancorfield18:07:55

(so map f is r empty? (comp f first) cons rest () roughly)

👍 2
Jason Bullers18:07:40

Those single letter variables make my eyes twitch, but a fascinating concept and exploration of how abstract you can get. You've gone looking for the first turtle in "turtles all the way down", I don't know if "majestic" is the right word for what you found, but it sure is something 😄

Jason Bullers19:07:23

Thanks for sharing

daveliepmann08:07:15

I'm reminded of a similar situation with a colleague who was surprised (to the point of indignance) at the fact that Clojure apply takes individual initial args x y z as well as the sequence args. Apparently this isn't universal in other languages' spread/splat/apply, and it frustrated my colleague that Clojure combined two tasks: "apply this list of args to the function" and "add these elements to the list we'll use for args". My perspective is that it's often handy and I see no reason for it to bother me. I have the same feeling with Clojure's map: yes it's technically doing two things, but from a design perspective I don't see a compelling need to pull them apart. The only downside seems to be surprise on first discovery.

Jason Bullers13:07:11

I actually really like that aspect of apply. I can't count the number of times I've done a fn in Java like foo(int x, int... xs) and been annoyed by having to prepend in the body before doing any work. And I was annoyed before starting Clojure... It'll be even worse if I go back, I'm sure.

mx200008:07:07

I am thinking about the naming of the game engine I am working on. So far it was very creatively called 'engine'. It is based on libgdx. I thought about 'gdx', but it is maybe causing confusion with libgdx itself. Here is the link (work in progress, no documentation, readme, tests yet) https://github.com/damn/gdx Any ideas for a short name ?

valerauko08:07:13

Not a particularly interesting idea but there's always the clj-gdx / gdx-clj option

mx200008:07:41

I have considered it but I find it somehow redundant

mx200008:07:02

I really like the name GDX a lot but I am not sure it could cause problems for being so short and possible colliding. Although I checked and in the clojure library world (checked on clojure toolbox website) there is nothing called that.

Emanuel Rylke09:07:38

CGLDJX (CLJ and GDX zipped together); clajer (short for CLoJure pLAyER)

mx200009:07:32

Someone suggested 'clogged' also 😉

vlaaad10:07:51

clodex 😛

valerauko13:07:30

the "dx" reminds me of the ancient pinball game "DX-Ball" so maybe "flipper"

Otso Björklund09:07:46

cleng (Clojure engine) for potential confusion.

emccue15:07:47

gdx will not collide with anything. Thats why you have a group-id

mx200015:07:10

Good point. But what is an idiomatic group id? I don't have a domain? Is jitpack enough with GitHub username ? Will it work with clojars ?

oyakushev16:07:34

I think somebody needs to say this to you, so let it be me: don't overthink so much of it upfront. Naming is important, yeah, but things like groupId are so minor. When your project gains the traction you want, you'll be able to grab a cool domain name and move to a different groupId and it won't be a problem. For now, just stick with whatever the default is. The value that your project can bring is much more important at this point than the embellishments.

2
emccue16:07:40

You'll have io.github.yourname/gdx that's good enough

damien21:07:38

Hi all, I am tinkering with JDK 19/20 virtual threads. When I run a test program that calls (:import (java.util.concurrent Executors)) I get this error when running that code using openjdk 20.0.1

λ clj -M src/threads.clj                 
Execution error (UnsupportedOperationException) at jdk.internal.misc.PreviewFeatures/ensureEnabled (PreviewFeatures.java:49).
Preview Features not enabled, need to run with --enable-preview
What's the idiomatic way to resolve this?

hiredman21:07:11

the import has nothing to do with this

hiredman21:07:48

you are using a preview api, but did not start the jvm with the --enable-preview flag

Joshua Suskalo21:07:40

Add a -J--enable-preview line to your clojure cli invocation to start the repl.

Joshua Suskalo21:07:20

If you're using emacs add a .dir-locals.el file to your project root with the following contents

((clojure-mode . ((cider-clojure-cli-global-opts . "-J--enable-preview"))))
If you use another editor I can't help much, but if you happen to run nrepl through the clojure cli directly, you can add
:jvm-opts ["--enable-preview"]
to the alias used to start it.

damien22:07:25

Yep thanks @U5NCUG8NR! I’m new to the clj tool after a few years away from Clojure. Adding -J—enable-preview to my invocation fixed it.

Joshua Suskalo22:07:45

Glad i could help!