concatenate them, get the distinct ids, sort those, then iterate through the sorted ids to generate the rows of your table.

Thanks!

(let [coll1 [2 1 3 4 5 6 7]
      coll2 [2 7 4 9]
      coll3 [1 2 5 12 10 11]]
  (-> (concat coll1 coll2 coll3)
      (distinct)
      (sort)
      (->>
       (reduce
        (fn [acc e]
          (let [in1? (.contains coll1 e)
                in2? (.contains coll2 e)
                in3? (.contains coll3 e)]
            (conj acc [(if in1? e nil) (if in2? e nil) (if in3? e nil)])))
        []))))
;; => [[1 nil 1]
;;     [2 2 2]
;;     [3 nil nil]
;;     [4 4 nil]
;;     [5 nil 5]
;;     [6 nil nil]
;;     [7 7 nil]
;;     [nil 9 nil]
;;     [nil nil 10]
;;     [nil nil 11]
;;     [nil nil 12]]

((fn f [& inputs]
   (lazy-seq
    (when (some seq inputs)
      (let [elem (apply min (keep first inputs))]
        (cons
         (for [i inputs
               :let [e (first i)]]
           (when (= e elem) e))
         (apply f (map (fn [m] (if (= (first m) elem) (rest m) m)) inputs)))))))
 [1 2 3 4 5 6 7],
 [2 4 7 9],
 [1 2 5 10 11 12])

I think that works but only if they are sorted already?

sure, merge sorts usually do

I have to study your solution a little more haha.

But thanks as well!

simplification of the reduce version

(let [coll1 (set [2 1 3 4 5 6 7])
      coll2 (set [2 7 4 9])
      coll3 (set [1 2 5 12 10 11])]
  (->> (concat coll1 coll2 coll3)
       (sort)
       (distinct)
       (reduce
        (fn [acc e]
          (conj acc ((juxt coll1 coll2 coll3) e)))
        [])))

oh man, I have an idea

actually that reduce should just be map

(let [coll1 (set [2 1 3 4 5 6 7])
      coll2 (set [2 7 4 9])
      coll3 (set [1 2 5 12 10 11])]
  (->> (concat coll1 coll2 coll3)
       (sort)
       (distinct)
       (map (juxt coll1 coll2 coll3))))

I think as simple as you can make it with clojure.core without getting into obfuscation

Beautiful. One nit: I'd do distinct before sort, for (probably slightly) better performance.

probably changing distinct into dedupe is the best perf fix

good catch

That could be better for smaller inputs, but fundamentally distinct is O(n) (as is dedupe) whereas sort is O(n*lg(n)) so you'd want to minimize the number of elements you're sorting.

also, sorting the collections before making sets of them would spare cycles, but sacrifices some simplicity in terms of the number of locals being used

(iteration
 (fn [[elem inputs]]
   (let [i (map (fn [m] (if (= (first m) elem) (rest m) m)) inputs)]
     [(apply min Long/MAX_VALUE (keep first i)) i]))
 :vf (fn [[elem inputs]]
       (for [i inputs]
         (when (= (first i) elem)
           elem)))
 :some? (fn [[_ inputs]]
          (some seq inputs))
 :initk [Long/MIN_VALUE [[1 2 3 4 5 6 7], [2 4 7 9], [1 2 5 10 11 12]]])

(I just had the realization that the reason I can never get a merge sort to work right as a reduce is that it is an unfold not a fold)

@U0K064KQV works with arbitrary number of inputs:

(defn merge-sort [& cs]
  (let [v (map first cs)]
    (when (apply not= nil v)
      (let [low (reduce min (keep first cs))
            v (map #{low} v)
            rcs (map (fn [v? c] (if v? (rest c) c)) v cs)]
        (cons v (lazy-seq (apply merge-sort rcs)))))))

Here is @U051SS2EU solution running live https://viebel.github.io/klipse-embed/?src=KGxldCUyMCU1QmNvbGwxJTIwKHNldCUyMCU1QjIlMjAxJTIwMyUyMDQlMjA1JTIwNiUyMDclNUQpJTBBJTIwJTIwJTIwJTIwJTIwJTIwY29sbDIlMjAoc2V0JTIwJTVCMiUyMDclMjA0JTIwOSU1RCklMEElMjAlMjAlMjAlMjAlMjAlMjBjb2xsMyUyMChzZXQlMjAlNUIxJTIwMiUyMDUlMjAxMiUyMDEwJTIwMTElNUQpJTVEJTBBJTIwJTIwKC0lM0UlM0UlMjAoY29uY2F0JTIwY29sbDElMjBjb2xsMiUyMGNvbGwzKSUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMChzb3J0KSUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMChkaXN0aW5jdCklMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAobWFwJTIwKGp1eHQlMjBjb2xsMSUyMGNvbGwyJTIwY29sbDMpKSkp&lang=clojure

Possibly try looking at Immutable.js? Afaik they copied Rich's implementations

The original hamt paper is pretty good http://infoscience.epfl.ch/record/64398/files/idealhashtrees.pdf

I know agents might solve this but I'd like to just get what I have working without too many changes.

Something like

(with resources-left ;; locking
  (do x)
  (do y))

Do everything inside swap! - should be fine as long as you don't side-effect in that "everything".

As you see my code returns two things, only the latter of which should go in the atom.

(swap! resources-left-atom ->runnable-jobs ready-jobs)

> As you see my code returns two things I understand what you mean now, but to be precise - your code above returns the value of (reset! ...), which is resources-left. Two solutions that I see: • Store runnable-jobs in the atom • Use swap-vals! and run find-runnable-jobs twice - once within swap-vals! to find the new resources and reset the atom and the second time outside of swap-vals! but using its results to actually get the runnable-jobs value

Thanks for your suggestions. Are you able to write a two-three line snippet of pseudocode for the last suggestion?

(defn find-runnable-jobs-and-resources-left-afterwards [resources-left possible-jobs]
  "Must be a pure function."
  [(dec resources-left)
   [(first possible-jobs)]])

(def resources-left (atom 10))
(def possible-jobs (vec (range 10)))

(let [[old-resources-left new-resources-left1]
      (swap-vals! resources-left (fn [rl]
                                   (let [[rl _rj] (find-runnable-jobs-and-resources-left-afterwards rl possible-jobs)]
                                     rl)))

      [new-resources-left2 runnable-jobs]
      ;; This call is exactly the same as the one inside `swap-vals!`.
      (find-runnable-jobs-and-resources-left-afterwards old-resources-left possible-jobs)]
  (assert (= new-resources-left1 new-resources-left2))
  (run-jobs! runnable-jobs))

Thanks ever so much for your help 😄

I'll play around with it to see what it does 😄 Thanks for the suggestion

It's wrong because the function that you pass into swap! is now side-effecting. Which is explicitly prohibited for swap! in its docstring.

Good point 🙂

Since the side-effect is resetting the unwatched, local atom I do not see a great danger but still. I guess it is one of the reasons it felt ugh

There is danger - the inner reset! might be run multiple times, and between those calls, there might be another reset! from another thread that uses the same functionality. You can end up with resources left being set to 5, then to 4, then back to 5, whereas in reality the actual resources left will be 4.

slurping and spitting is fine, since rewrite-clj will maintain any whitespace and comments

Indeed it works perfectly

clj-kondo data analysis can sometimes be a handy way to quickly find what you are looking for, then rewrite-clj can be used to rewrite. Drop by #rewrite-clj if you wanna chat more.

In my particular case I literally had all the vars as vals in a map so I could just use that and meta to get the file name. Now I'm addicted to this kind of thing though, so I will check out clj-kondo analysis next time.

Awesome! Feedback on rewrite-clj and its docs are always welcome, so please share if you found anything good/bad/confusing in #rewrite-clj

here's the code on line 87 (parse-json-response (String. body "UTF-8"))

The second argument needs to be a charset instance, not a string

https://docs.oracle.com/javase/7/docs/api/java/nio/charset/Charset.html

What library are you using for parsing?

sweet! thanks

I'm using cheshire

although I have to say, that error message is super unclear; I don't think I'd have noticed the issue by myself

This error message is from clojure's reflector, not cheshire

But I'm asking because cheshire can deserialize bytes or stream directly, no need to turn it to a string at all

this is just adding extra work for nothing :man-shrugging:

there's a try catch that looks like this

(try+
   (json/parse-string body true)
   (catch ClassCastException _
     (parse-json-response (String. body (Charset/forName "UTF-8"))))

not sure why the person who wrote it originally added it

the whole mtd looks like this

(defn parse-json-response
  "Parse a body as JSON catching formatting exceptions."
  [^String body]
  (try+
   (json/parse-string body true)
   (catch ClassCastException _
     (parse-json-response (String. body (Charset/forName "UTF-8"))))
   (catch JsonParseException _
     (str "Improperly formatted API response: " body))))

it's recursieve

actually, it seems the error still isn't resolved after adding (Charset/forName "UTF-8")

The code above seems to be trying to handle different types of body while also specifying that it can be only String. So either ^String should be removed or the (catch ClassCastException ...).

Which one - depends on how the parse-json-response function is used in the rest of the code. If its argument can be nothing but a string, then ^String should be left in place.

cool, thanks

no plans to do so

Thanks. I was just curious 'cause I hadn't seen anything in http://ask.clojure.org about it, and it seems like it would be very useful in certain domains.

there have been informal conversations about it over the years

is there an example other than complex numbers?

In my personal work I'd like to be able to make vectors use the clojure core ops for their operations, but that's because I do gamedev and that's niche.

Automatic differentiation though is very useful in the ML space and is enabled by custom numeric types.

It's how Julia does AD, and as a result Julia has some really good support for custom ML models with low effort.

I'm not sure that there is really a big bar here that couldn't be solved with making your own functions

and if there are perf concerns, you're probably ultimately going to want to be working outside the clojure framework for numeric ops

The key usecase for AD would be to be able to use it to get derivative information on library code that you don't have source-level control over, so while yes "more functions" could do it, that increases maintenance burden and introduces more code.

But yeah, for most people it probably wouldn't affect much.

I was just curious if there had been thought put into it, and I'm happy to hear that there has been.

#sicmutils has all of this (AD, complex, quaternions, tons more) and does it with its own set of operators that shadow the core ones

it’s not that big of a deal to shadow +, -, etc; one nice trick is that you can overload all operations to handle symbolic arguments, and then “compile” numerical functions that need to be fast by • passing symbolic inputs • simplifying the resulting expression, extracting common subexpressions etc • wrap up in a fn and call eval on the result basically what Julia is doing, as a library here

Right, and that's pretty nice, but the issue is that a math-based library can't use this unless you do like a with-redefs or they already use it

Yup, and with-redefs won't actually work because the math operators get inlined

I had forgotten that :) THAT is the big win, being able to differentiate through other people’s code etc

Yes, exactly

Asking the obvious question, why a 5th solution?

Obviously not OP, but looking at it, it's the fact that there's no reliance on either global state or protocol extension. "it's just data"

Along with user-defined lifecycle steps.

That's an interesting perspective from what appeals to you as a user

Have you ever found you need any other life cycle step other than start and stop?

It enables architectures where the system isn't the "root" of the program, and when designing a program like that I think yes, other steps would be useful.

Could you elaborate on this perspective?

Sure. For example you could design a socket server that has one component that's the socket, and then send a clocked signal to accept connections, adding all the connections as components into the system. Then on clock pulses the connections can register themselves as complete and get removed, and when you want to halt the program you can send a halt signal.

Just off the top of my head.

Also the fact that you could use a system map as a component instance itself enables you to use data-driven signal sending and handling between components, rather than needing a protocol with a fixed small set of acceptable signals.

Nothing stopping you from defining your own protocol just like your own signals, no?

there is something stopping you from associng in a new signal handler if you use protocols though

And with topological sorting I don't see the utility in nesting systems

You mean dynamically registering a new handler?

Yeah, Which protocols can't really do

besides just making a "signal" method on the protocol and implementing what donut.system is already doing yourself

Basically as I look at this, this is a functional representation of Alan Kay's message-passing style OOP that predates what the term is now used for. Which I think there's some use for when composing systems.

I also wanted custom signals for e.g. :heartbeat signals or other signals that could be used for polling system health. in the past, with integrant, I also tried to create a custom step to do verbose validation before actually starting a system, and found that to be pretty awkward

This is helping me think of how I could come up with clearer motivating examples for donut.system's capabilities. All of them address real-world problems I've run into

it might not be obvious why component groups are nice, but I've found them very useful for a couple cases: • creating multiple instances of a component, where that component depends on a little sub-graph of components • creating a foundation for component libraries. I can create a set of related components that are able to use local references, publish that, and then you can just stick that in your own system map and it will work

really appreciate this feedback

Yeah, being able to use it in libraries is something I'm kinda two ways about. On the one hand it's nice to not be limited and it's flexible, but on the other hand these sorts of systems are usually useful in applications, less in libraries, and if libraries start providing things in terms of components that forces you to choose this dependency injection library.

it would be very useful in larger companies and with things like polylith though.

that's a really good point. I've tried to design it so that libraries can easily provide a donut.system component group, but they don't have to in order to work correctly. libraries wouldn't even need to require the donut.system library, they could just provide some component group map

how would they do a cross reference though?

within a component group you can use (ds/ref :component-name) and that will resolve to the component named :component-name within the same group, so if your library has some set of related components they can use those local refs to refer to each other -does that answer your question? Or, does this example help? https://github.com/donut-power/system#groups-and-local-refs

Right, my point was more that if you don't depend on donut.system, you can't do cross references.

oh right, shoot. I see what you mean. good point

maybe I can redesign that to use a map or vector for refs instead of a donut.system/Ref

Definitely plausible, that could allow dependency-free components.

It'd be important to document though.

yeah

this is great, thank you!

I’ve changed the ref implementation to use [:donut.system/ref ref-key] or [:donut.system/group-ref group-name] instead of a record 🙂

Is there a quick summary anywhere on the differences between donut and clip?

There isn't - iirc some of the differences include • donut has component groups • donut doesn't try to do any symbol resolution (I vaguely recall clip having some set of rules for this?) • donut has built in support for subsystems • donut doesn't yet support alternative execution, like async exec, like clip does. It should be fairly easy to add that though

bonus point: donuts are tasty

that’s how I get ya!

You might also like the approach here https://github.com/galdre/morphe

I see in your example in the readme, is there a reason you're doing (println (pr-str ...)) instead of just (prn ...)?

usually to prevent interleaving of multithreaded output

Does that mean that prn is racy?

the output stream is a shared resource and each thread can interleave while writing

pr-str makes a string, and println writes each "part" as a chunk, so it's a matter of whether you write N chunks or 1

What is a "part" here?

args to println

println is basically calling pr on each arg passed to println

Got it yes, I was confused because the discussed readme has only 1 argument Personally I moved to prn after a few years of pr-str+println because I never use println with more than 1 argument (which comes naturally for me because I'm a heavy -> er... normally I'll add (doto prn) in the middle of a chain) I still use pr-str in other places like logging though.

ah, that makes sense, if you're passing more than one argument.