same with India I suppose now you mention it

he's not objectionable so noone remembers him

Or Israel as well I think.

I’m a little late to the golf party here; but if you take @U09LZR36F’s improvements and port it to use a transducer it’s almost 2x quicker in my crude experiments:

(defn gen-data+acceleration-transduce [{:keys [init scale count noise-factor acceleration] :or {acceleration 0}}]
    (transduce
     (comp (take count)
           (map-indexed (fn [i scale]
                          (+ init (* scale i)
                             (* (next-rand) noise-factor)))))
     conj!
     (transient [])
     (iterate #(+ % acceleration) scale)))

I think you're missing a call to persistent

conj! should do that in the arity-1 function body

Ah! Clever, interesting. Didn't know that trick.

oh actually 👀 not sure it does

I think this is right:

(defn gen-data+acceleration-transduce [{:keys [init scale count noise-factor acceleration] :or {acceleration 0}}]
    (persistent! (transduce
                (comp (take count)
                      (map-indexed (fn [i scale]
                                     (+ init (* scale i)
                                        (* (next-rand) noise-factor)))))
                conj!
                (iterate #(+ % acceleration) scale))))

i.e. it does call (transient []) for you for an initial value; but won’t call persistent! — which makes sense, as I guess you may still want to do more on the transient value.

The downside of this version is no laziness. Although sequence can do that.

Conceptually I like that the function generates infinite data and the consumer can use take instead of there being a count

Yeah — I chose to keep the existing contract for comparison. And I agree with you about not limiting it artificially. Transducers are supposed to give you that flexibility by splitting the xform from the starting sequence; however here the xforms are pretty tightly coupled to the starting sequence. You could probably refactor to take extra xforms to comp in, e.g. you could supply (take 10) as an argument — but I think that would start getting a bit messy.

Though you could possibly supply the collection of scaled accelerations, and take on that; prior to transducing.

Without knowing the domain, I'm unsure whether it makes sense to do that really

agreed — and in this case I don’t think it make sense, mainly suggesting it as a potential solution in other cases when this sort of thing arises

Yeah. Transducers are also just fun 😛

yeah they are — though I do sometimes struggle with splitting their concerns, whilst retaining things a lazy solution might give… the above tight coupling between input seq and the xform being an example of it

Thanks for this @U06HHF230! I use transducers a lot, but pretty much excursively in the context of (into [] some-xf coll). I really need to get comfortable using it in other ways =)… One thing I still find difficult is working out good ways of thinking around composing them, I’m also still not super comfortable just pulling an xform from a place and intuiting, oh, it just fits here… I suppose I don’t have a good feel for treating them similarly to how I treat higher order functions…

TBH that way is even nicer too:

(defn gen-data+acceleration-transduce [{:keys [init scale count noise-factor acceleration] :or {acceleration 0}}]
                              (into [] (comp (take count)
                                            (map-indexed (fn [i scale]
                                                         (+ init (* scale i)
                                                            (* (next-rand) noise-factor)))))
                                    (iterate #(+ % acceleration) scale)))

And it will already use a transient and conj! so should be just as fast

Comparing:

(time
    (dotimes [_ 100]
      (doall
        (gen-data+acceleration-transduce {:init 10 :scale 0 :count 2000 :noise-factor 0 :acceleration 2}))))
  #_#_=> "Elapsed time: 83.439927 msecs"

  (time
    (dotimes [_ 100]
      (doall
        (gen-data+acceleration-into-transduce {:init 10 :scale 0 :count 2000 :noise-factor 0 :acceleration 2}))))
  #_#_=> "Elapsed time: 112.475216 msecs"

Really need to spend some more time with criterium and up my benchmarking / profiling >_<… But in my mind not worth doing that until I have an app that’s got some meat to it =)… Almost there!

I suspect that difference is due to unreliable benchmarking that criterium would help iron out; if you look at the definition of into, it’s essentially exactly what I wrote above.

i.e. into is just a thin layer on transduce in that call path.

That’s reasonable, let’s give that a go =)…

After spending some time restarting because of editor existential failure

(criterium/bench
    (doall
      (gen-data+acceleration-old {:init 10 :scale 0 :count 2000 :noise-factor 0 :acceleration 2})))
Evaluation count : 35280 in 60 samples of 588 calls.
             Execution time mean : 1.639113 ms
    Execution time std-deviation : 53.330771 µs
   Execution time lower quantile : 1.588486 ms ( 2.5%)
   Execution time upper quantile : 1.788930 ms (97.5%)
                   Overhead used : 4.082033 ns

Found 3 outliers in 60 samples (5.0000 %)
	low-severe	 1 (1.6667 %)
	low-mild	 2 (3.3333 %)
 Variance from outliers : 19.0160 % Variance is moderately inflated by outliers

(criterium/bench
    (doall
      (gen-data+acceleration {:init 10 :scale 0 :count 2000 :noise-factor 0 :acceleration 2})))
Evaluation count : 44100 in 60 samples of 735 calls.
             Execution time mean : 1.239084 ms
    Execution time std-deviation : 87.727749 µs
   Execution time lower quantile : 1.162137 ms ( 2.5%)
   Execution time upper quantile : 1.502526 ms (97.5%)
                   Overhead used : 4.082033 ns

Found 4 outliers in 60 samples (6.6667 %)
	low-severe	 2 (3.3333 %)
	low-mild	 2 (3.3333 %)
 Variance from outliers : 53.4285 % Variance is severely inflated by outliers

(criterium/bench
    (doall
      (gen-data+acceleration-transduce {:init 10 :scale 0 :count 2000 :noise-factor 0 :acceleration 2})))
Evaluation count : 100620 in 60 samples of 1677 calls.
             Execution time mean : 593.705999 µs
    Execution time std-deviation : 23.468176 µs
   Execution time lower quantile : 568.977017 µs ( 2.5%)
   Execution time upper quantile : 649.404382 µs (97.5%)
                   Overhead used : 4.082033 ns

Found 5 outliers in 60 samples (8.3333 %)
	low-severe	 3 (5.0000 %)
	low-mild	 2 (3.3333 %)
 Variance from outliers : 25.4815 % Variance is moderately inflated by outliers

(criterium/bench
    (doall
      (gen-data+acceleration-into-transduce {:init 10 :scale 0 :count 2000 :noise-factor 0 :acceleration 2})))
Evaluation count : 90540 in 60 samples of 1509 calls.
             Execution time mean : 652.417338 µs
    Execution time std-deviation : 20.273083 µs
   Execution time lower quantile : 630.189989 µs ( 2.5%)
   Execution time upper quantile : 697.334784 µs (97.5%)
                   Overhead used : 4.082033 ns

Found 6 outliers in 60 samples (10.0000 %)
	low-severe	 5 (8.3333 %)
	low-mild	 1 (1.6667 %)
 Variance from outliers : 17.4268 % Variance is moderately inflated by outliers

I suspect that small difference between the into variant and the transduce one is just the instance check and the attaching of metadata that into does, and you’ll find that if you increase your count to maybe 50,000 or something larger that the difference will become much less significant. Depends on how you plan on using this though.

Attaching metadata? I didn’t know that into adds metadata. They’re basic functions to generate sample data, so I won’t be generating large numbers =)…

Yeah just that: (meta (into (with-meta [1 2 3] {:a :b}) [4 5 6])) ;; => {:a :b} It’s the same semantics with most of the core collection functions; that metadata should be forwarded onto the new collections as you build them.

Oh that’s cool =)…