clj-kondo can do this. I've used https://github.com/benedekfazekas/morpheus to visualize that data in the past, with some success. Worth a look.

There's a ton of stuff. • https://github.com/gfredericks/clj-usage-graph • https://github.com/greglook/clj-hiera • https://gitlab.com/200ok/codegraph • Relevant clj-kondo docs: https://github.com/clj-kondo/clj-kondo/blob/master/analysis/README.md Potentially useful when building your own: • https://github.com/clojure/tools.analyzer • https://github.com/jpmonettas/clindex

seems unnecessary? if you find it helpful, then go for it, but i don't ever struggle with reduce or into or any others. the fact that you have to manually write the conj makes me think it's not worth the effort

Yeah, if you didn't want control over the conj you'd probably just use transducers

Could you just use (empty acoll) in place of the condp?

(transduce (map f) conj (empty x) x)

Yeah, I'd probably never use this, but something just bugs me about reduce.

Compared to map

map just feels more elegant and I want a more elegant version of reduce

what don't you like about reduce?

but yeah I'd probably never use this in a team setting, unless everyone was using it

It's completely subjective

The ambiguity around the 2 arg vs 3 arg version

Something about the mental model is harder for me than map, more like a loop, on the scrutability scale

Oh, of course (into (empty x) (map f) x)

ah right

into will even use transients internally

but the user has access to the acc the whole time, so they can do stuff that reducers do there

map needs to return nils

The only thing reproduce does differently from reduce is fix the accumulator

but yeah it can probably be done in pure transducers

it gives the user access to update the acc in downstream items

Reduce also does this

right

so this is like a reduce with training wheels. Suppose to feel simpler, like map

map is reduce

yeayayea

but map returns nils, whereas sometimes you really do want a reduced collection, but without all the reduce ceremony

dunno what "map returns nils" means

You can't return a collection of a smaller size with map

keep

true

does keep let you change the kept value?

so more like filter

Does reduced work in map?

I guess that doesn't make sense

If I had a really big code base on a solo project I'd give it a try to see if it's useful but yeah doesn't seem useful enough to be its own thing at first blush

keep is super unless false is an important, significant return value from f

or nil, right?

keep discards all falsey results

right

reduce lets you build colls with falsey vals

or choose not to

few things besides reduce give you both options

other than loop

hmm maybe this is cleaner, and easier to work with transducers:

(defn reproduce [afn bfn acoll]
  (condp = (type acoll)
    cljs.core/PersistentVector
    (reduce #(afn %1 (bfn %2)) [] acoll)
    cljs.core/PersistentArrayMap
    (reduce #(afn %1 (bfn %2)) {} acoll)
    cljs.core/PersistentHashSet
    (reduce #(afn %1 (bfn %2)) #{} acoll)
    ; :else
    (reduce #(afn %1 (bfn %2)) () acoll)))

(->> [1 2 3 4] (reproduce conj inc)) ;=> [2 3 4 5]

(->> #{1 2 3 4} (reproduce conj inc)) ;=> #{4 3 2 5}

(->> {:a 2 :b 4} (reproduce merge (fn [[k v]] [v k]))) ;=> {2 :a, 4 :b}

as an api

Where the "reducing function" has access to the whole coll on each step, so the user can still choose to reduce or add to the coll at each step

mm'no, afn needs access to %2 too make that decision, so maybe it might as well be a single fn :thinking_face:

@U0HG4EHMH oh, right, with the (empty acoll)

(defn reproduce [afn acoll]
  (reduce afn (empty acoll) acoll))

I'm going to update the opening post with that since it makes it easier to understand

Hmm, with that api, you can't do stuff like (reproduce + [1 2 3]) ;=> 6 This is maybe better:

(defn produce [afn acoll]
  (reduce afn nil acoll))

(produce + [1 2 3]) ;=> 6
(->> [1 2 3 4] (produce #(conj (or %1 []) (inc %2)))) ;=> [2 3 4 5]
(->> [1 2 3 4] (produce #(conj %1 (inc %2)))) ;=> (5 4 3 2)
(->> #{1 2 3 4} (produce #(conj %1 (inc %2)))) ;=> (4 3 2 5)
(->> {:a 2 :b 4} (produce (fn [m [k v]] (assoc m v k)))) ;=> {2 :a, 4 :b} 

And you can just do a (into #{} ... after the produce if you need a set afterwards, like we do with mapping functions

In which case the name should probably just be produce, since the original collection type isn't being reproduced anyway (updated above example)

The idea is just basically a simpler reduce, that starts you off with simpler semantics, before you learn about reduce

(produce + [1 2 3]) doesn’t work here.

This seems like a strange hybrid of into, reduce and map. I am not sure if the semantics are necessarily simpler. It isn’t equivalent to reduce, which means, if you wish to use produce to generate anything other than a collection, then you will have to learn about reduce anyway.

Is reduce really that hard to learn?

I use this to teach reduce and it seems to work well enough

(-> init
    (f first-element)
    (f second-element)
    (f third-element)
    ...)

I mean it's learnable. Just not as smooth as map

produce has a uniformity to it that makes it easier to reason about longer term iterations IMO

Just not as useful as reduce

I also don't want to waste my time scanning my eyes all the way to the end of a reducing function just to see if/what first param it's taking (I already know what's flowing on the right hand side of the thread-last pipeline). With tools like produce you can signal to the reader that this only takes a single sequence argument after the mapping/reducing function, like most other things in your thread-last pipeline

Don't make the reader's eyes backtrack, if possible. These reductions are just easier scan with your eyes fast down a thread-last form:

(->> #{1 2 3 4 5 6 7 8 9}
  (produce #(conj %1 (inc %2)))
  (produce (fn [m n] (+ m (if (even? n) n 0))))) ;=> 30

Not sure why (produce + [1 2 3]) ;=> 6 wasn't working for you

Oh, maybe it works in cljs and not clj

Yeah, + is different

Yeah, I guess it wouldn't be a subset of reduce in that regard :thinking_face: hmm

Yeah it could just be a thing for sequences, but yeah I think that makes it less interesting

So it's back to this original thing that reproduces the source type:

(defn reproduce [afn acoll]
  (reduce afn (empty acoll) acoll))
(->> [1 2 3 4] (reproduce #(conj %1 (inc %2)))) ;=> [2 3 4 5]

Maybe it should be called induce, since it only reduces what's in the collection but not the outer collection type 😆

I wonder how often, in the wild, the source and target collection types of reduce calls are necessarily different :thinking_face:

Well, you can change the return type half way through the reduction, if you wanted to. It just starts with the type you had

Sounds more and more like the kind of feature they need in Python

> I wonder how often, in the wild, the source and target collection types of reduce calls are necessarily different :thinking_face: A significant minority for me/codebases I work in

> The ambiguity around the 2 arg vs 3 arg version [is inelegant & bugs me] Reminds me of Rich denouncing the 2-arg arity in https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/InsideTransducers.md. When given the opportunity to try again, he put the onus on f instead: > Who knows what the semantics of reduce are when you call it with a collection and no initial value? No one, right. No one knows. It's a ridiculous, complex rule. It's one of the worst things I ever copied from Common List was definitely the semantics of reduce. It's very complex. If there's nothing, it does one thing. If there's one thing, it does a different thing. If there's more than one thing, it does another thing. It's much more straightforward to have it be monoidal and just use f to create the initial value. That's what transduce does, so transduce says, "If you don't supply me any information, f with no arguments better give me an initial value."

Interesting. Yeah, it's unfortunate I guess. Rich prolly won't impose a new version of reduce on the community, so if the community doesn't adopt and experiment with new idioms then the core team will be disincentivised from causing the imposition of the new semantics. Like a better-reduce or something

It's a bit of a catch 22, trying to grow the core semantics, when what we have is "good enough"

Which has it's obvious benefits as well

I wonder how often reduce calls in the wild use the 2-arg version :thinking_face:

Yeah same

Is there a library out there that allows you to add functions to clojure.core globally for the whole project? Maybe poly-filling community ideas into core would make community experimentation easier and have less friction

Cause honestly nobody is going to proactively litter every namespace across their whole project/lib/app with :refer [induce]

The js community seems to be able to do polyfills without everything falling apart. We could probably do it better, given the circumstances

Sounds like a util ns, which I'd vastly, enormously prefer over magic requires

It's more like, "I'm the lead dev at acme widgets and I want our team to join two other companies in trying new thread-last semantics across all of our apps and projects" and then those companies could report on the results of the experiment

We could litter our namespaces with this experiment in what we might consider a core function of the language, with organizational rules, "remember to require this stuff into all your new namespaces, thx!!!" But that obviously sucks

And so no real experiments in core-ish functions like induce ever really happen because there's just too much friction for those kinds of experiments

They might happen at clojure.core, but then the community just becomes passive consumers of clojure.core's preferences, but, again, they're disincentivised from that kind of experimentation, so community market effects are such that some notional induce, even if it's a good idea and should replace reduce in many circumstances for clarity's sake, will just never be adopted. There's too much friction for all parties involved to get something like induce adopted

> that obviously sucks i wonder what other folks do because i don't find it that onerous

I mean, in a healthy system, I think the clojure core team should be seeing more inspiration from the community for new ideas. It should be easier for the community to incubate ideas and for the core team to be able to say, "yup, we want that," and be able to integrate an idea that they can already see work in example prod environments. That happens with libs, just not as much for one off core functions

It's not onerous normally. I'm talking about the kind of function you want to be globally available, as a kind of language wide dev UX experiment

> no real experiments in core-ish functions does run! not count as one of these?

Not familiar with it's backstory

> I'm talking about the kind of function you want to be globally available I am too, or at least I think I am

Just that it's a core fn, very similar to reduce and map, introduced recently

Did the community inspire that change? Was it battle tested in libs in the wild before the core team brought it in?

not afaik

Not that things have to be. And to be clear, I think we're better off that things don't work that way for the most part. Most people ought not be messing with polyfills

at least, not "in the wild" meaning a contrib-style system of community creating something which graduates to core.

So yeah, we have the incubator repos. Some of them are gems. And that serves it's purpose for those kinds of features.

But then you have these "kitchen sink" libs that have these awesome little functions, but you know who's not going to bother trying that kitchen sink lib, in every namespace, just so I can try out induce, whether it's in the incubator or not? Me. Too much friction

However, if it's just a matter of adding the lib to my deps (and maybe having to require it in one of the namespaces) then I think it wouldn't be too much bother for an organization to do organization wide

And most of the devs never have to worry about whether it's required or not

Like letting users create an origin trial, test it in org, without having to fork clojure just to report back to the community and the core team on the results of the experiment

And of course the core team is under no requirement to even look at your experiment. I'm just saying that conducting the experiment seems like a lot of friction right now

I guess enforcing an organizational wide user file that make some functions globally available would be another way

Reader tags work this way though, so there is some prior art

Require it in one namespace and it'll work across the whole project

In cljs you don't even need to require it, just add the dep

This is specifically about "core functionality" though, usually pertaining to developer UX, which is the only reason you'd want to go around the ns form to require it in

Like, imagine 100 years from now, some clojurists are talking and one says, "you know, back in the day, Rich did say that reduce kinda sucked." So someone responds, "Fine, let's all agree to start using something better starting now." So folks start experimenting with different things but between the years of 2120 and 2130, every namespace everywhere was littered with tons of different implementations of new-reduce and my-reduce and, in the end, nobody could agree on a new reduce because people were more fed up with the polluted namespaces than they were happy with the new thing, so it never got added and clojurists were doomed to an eternity of never improving on reduce. That would be post apocalyptic lol

Isn't that precisely what happened with Common Lisp, and Clojure was the solution?

In some ways, not others

Reader tags hasn't CLified things

the drawbacks of reduce are extremely minor and well-solved by just...not using the 2-arity version

JS is clearly getting the polyfill thing right, as a tool talk about the future

That may be true, but does it have to be true in a million years?

When can we finally have something better than what rich admitted was bad?

there are multiple solutions to the stated problem • ignore it • deal with it case-by-case • linter warns on 2-arity usage • magic global requires • clojure 2.0 • clojure plays the role of CL as new lang supplants it Of those, I might like magic global requires the least

it's also very weird to me to fix "reduce is complex because of its 2-arity" by introducing complexity at the ns/require level

Don't you think the best option would be: * the community slowly adopts TechA, it grows steadily until core team decides it belongs in core Right now, that path is hard because reaching critical mass on adopting core idioms involves too much friction

Isn't that the healthiest adoption pipeline? People bang on ideas and the core team gets to bring in the best parts

Right now, core team just has to have extreme insight to know a priori what ideas will be good and bad in the wild

that sounds like a social issue, not a technical one

Can they really know that induce is a good idea? What if it's mostly good for beginners? There's no system for trying things out on larger test groups with less friction

as opposed as I am to magic global requires, I am enthusiastic about tools/environments which provide racket-style language levels for clojure. the social/technical context there is completely different.

I guess I'm imagining a scenario where the core team can facilitate language level experiments with some polyfill system, while not imposing any requirement or expectations on the core team to pay attention to any particular polyfill/trial

How does JS avoid the CL fragmentation issue with their polyfill? They ship the polyfill in the lib, right? Do js devs ever yell about how they got bit by a polyfill under their stack that they didn't know about?

I guess the risk is lower since js doesn't have macros

And the cl fragmentation issue is really a social one, of unrecognizable syntax

Whatever the case may be, polyfills are working for js so we must be making mountains out of molehills here with this particular aversion

But a polyfill in js essentially is a macro

> And the cl fragmentation issue is really a social one, of unrecognizable syntax not sure I agree — plenty of the fragmentation was behavior in competing libraries which we currently enjoy in core, e.g. seqs and other abstract data structures, i think threads, that kind of thing

i mean i agree it's social 🙂

Agreed (though I'm foggy on the CL history)

Another example, I have this injest library that adds +>, x>>, some others, that mostly introduce new semantics to the existing thread macros. There's a few other libs out there doing similar threading things and we're running out of names! And aliased thread macros suck (`(clunky/-> ...`). And maybe we want to test some new semantics for the core ->, maybe just adding some validation, disallowing literal function definitions as top level forms in a thread, and we've run out of cool new thread operator names and we resigned ourselves to shadowing ->. If 25% of the devshops out there adopt safe-> transparently for their -> then core can have an idea of what people are finding useful in core without having to know about the usefulness of everything beforehand. Right now, there's too much friction for 25% of devshops to adopt safe-> transparently

Yeah, a lot of better-foos have come and gone. Again, we don't want better-foo if it means bringing in kitchen.sink.utils every time we want to use it. It's just not worth it. So clojure/script doesn't end up with SDKs like jQuery. Either we use core or we just build it locally bespoke. To bring in a lib, it has to do a lot of what I don't want to do myself. So kitchen sink libs are never really successful in clojure, containing a bunch of little things that most of us individually aren't interested in, so we never actually end up giving better-cond a shot. Like it never really had a chance, even if it really was better, because we just don't have enough desire to include it in every single namespace, in every project, forever into the future. It's too much friction

It'd be nice to accommodate optimizing compiler plugins through some polyfill thing too. Why not let the community incubate optimizing strategies that may never make it into core? Somebody is going to write the optimization layer one day anyway, right? But yeah, I'm not saying core should pay any attention to any given experiment. I just have very little incentive as a clojure developer to participate in any experimental libraries that try to do something better than core. It's too much of a penalty for my downstream users and team members for me to impose those experiments on downstream users. Stepping outside of clojure.core just isn't worth it, given the economics of things.

I'm pretty excited about injest, for instance. I tested it on an app servicing a primary part of the company I was working for and in the end I decided against bringing injest into the app. For that particular app, it only brought us nominal performance improvements. Unless it's absolutely necessary, it's technical debt, so I didn't think it was worth the educational overhead for all future developers of the app to impose the new semantic. So of course I'm never going to bring induce into a production app that I promise will be easily supported even after I'm gone. But I'm saying that's a problem. I should have more incentive to buy projects into less vanilla idioms. Evolution should be less costly, somehow

I probably wouldn't have opted that app into any experiments anyway, too important, so maybe not a good example. For internal facing apps though, sure

Idk, I don't have the right answers. I'm just surprised how well polyfills are working out in the js world though and I'm curious if we could benefit from it somehow

Well, it's not as if we need permission, I'm sure polyfill type things (monkey patching clojure.core, for instance) can be done in libs, and devs don't need permission to do so. It's just not something we do

> I mean, in a healthy system, I think the clojure core team should be seeing more inspiration from the community for new ideas. It should be easier for the community to incubate ideas and for the core team to be able to say, "yup, we want that," we do have this - you file a question at https://ask.clojure.org and the community can then vote on it. We look at the top-voted things in every release and use that as one important signal for what we work on. We may ultimately decide not to include it but we are actively considering things here by vote.

there is very little reason to monkey patch clojure.core (and that really seems even detrimental to you if at some later point core adds the function). We have a whole namespace system - when you need a function, include the namespace and use the function.

Yeah, I don't mean to imply otherwise. And I may be asking for the impossible. I kinda want the conservativeness and exciting new stuff at the same time

Yeah, that's true. But what about polyfills? Would you say JS polyfills are largely obviated by macros? I think polyfill experiments show pretty powerful language level experiments

you have it - make all the namespaces with all the functions and macros you want

there is very little you truly need language level change for in Clojure

Yeah I agree

and those few things probably have 10x-100x impact than you realize

I think my larger point too was that just culturally we're just super averse to letting some new better-cond or induce kind of new semantic into our codebases

I know I am

So I don't know how we'll ever end up with "better reduce" you know? But maybe it's for the best

there is no barrier to do so - they are just functions. we have added things from clojure utility libraries in the past (`update-keys` and update-vals are variants of things that existed in many libs, for example)

Yeah, those are good examples

Those were in tons of libs, right?

those came about because ask clojure questions were filed, they got voted up to the top, we evaluated, and added them

Right, which is an awesome pathway. And evolution does happen

of course

I'm not trying to complain that y'all are doing something wrong or something. I'm just wondering how we can accelerate innovation in general. But Clojure does evolve, so it's not like I'm saying there's some systemic problem

And I mean innovation in the community space, not necessarily in core

And when talking about this induce fn, it dawned on me that even if it was good, there's pretty much no way we'd adopt it as a community, just because of cultural reasons. I'm disensentivized to produce too many idioms that compete with core idioms, as that eats into my own lunch in many ways. So unless y'all forced something like induce on the community, the community will probably never adopt such a core idiom that directly competes with reduce. I doubt we'd even vote it in if we could, even if it might benefit beginners, for instance. And I'm not actually sure there should be an induce like thing in core. It's just an example

And it's interesting because reduce is one of those examples where even Rich agrees that there could be a better version out there, but we're still stuck with semantics from developer inertia going all the way back to CL