Fork me on GitHub
#clojure
<
2018-04-30
>
sophiago08:04:28

Can anyone point me to examples of using transduce where map is f and the xforms are the functions called by map? I'm thinking of something like (map #(xf3 (xf2 (xf1 %))) coll) => (transduce (comp xf3 xf3 xf1) map coll) where coll is presumably nested and transduce returns it flat.

cgrand08:04:54

(map #(xf3 (xf2 (xf1 %))) coll) becomes (sequence (map #(xf3 (xf2 (xf1 %)))) coll) or (transduce (map #(xf3 (xf2 (xf1 %)))) into [] coll)

sophiago08:04:46

So when I said like it turns out the like is not really an improvement?

cgrand08:04:48

You can substitute (map #(xf3 (xf2 (xf1 %)))) by (comp (map xf1) (map xf2) (map xf3))

👍 4
sophiago08:04:39

Oh, yeah that last one might really do it for me.

sophiago08:04:05

I'm experimenting with this route before just going custom with Specter.

cgrand08:04:06

(map (comp xf3 xf2 xf1)) is another option (beware of composition order being different this time)

cgrand08:04:04

@sophiago your whole transformation is just a bunch of map? (it doesn’t ring quite right with > where coll is presumably nested and transduce returns it flat.

sophiago08:04:15

It's a pattern I use a lot. Basically applying different functions to each level of nesting that end up flattening them.

cgrand08:04:46

but (trying to understand) if you do (map #(xf3 (xf2 (xf1 %))) coll), xf3 must returns a single element and 99% of the flattening is done by #(xf3 (xf2 (xf1 %))), no?

sophiago08:04:12

Yeah, I sort of deferred thinking about that 😛

sophiago08:04:34

I think you've given me something really good to play around with, though.

sophiago08:04:50

Do you think I could use xf3 as the init?

sophiago08:04:19

This would require refactoring lambdas I lifted out of the nested maps anyway.

cgrand08:04:05

I’m not following, if you could come up with a concrete (runnable) example it would help me a lot

sophiago08:04:43

That was just a passing thought. I think I can probably get there from your example

dominicm08:04:20

@sophiago it may be notable that cat is a transducer also

dominicm08:04:41

(comp cat (map xf1) …) enables xf1/2/3 to be run over an already flat sequence.

sophiago08:04:23

Hmm... this may fit into how I just realized @cgrand's second, prettier, example probably wouldn't do what I want like the first would

sophiago08:04:37

A practical example would neither be minimal nor reproducible so I'm not sure how much it would help in explaining the pattern

cgrand08:04:18

@dominicm but then if you chain (comp cat (map xf1) cat (map xf2) cat (map xf3)) you can merge maps and cats into a mapcat

👍 4
cgrand08:04:46

but then it starts to looks a lot like what would have originally been a for, no?

sophiago08:04:29

@cgrand your first sentence stated my problem with that better than I did

cgrand08:04:58

(for [x coll, x (xf1 x), x (xf2 x), x (xf3 x)] x) (sloppy reuse of x)

sophiago08:04:47

I'm actually not averse to shadowing in bindings. Elsewhere, very much so 🙂

sophiago08:04:40

The semantics of for are quite different from map though, yes?

sophiago08:04:23

I may just be totally misunderstanding your use of it

sophiago08:04:18

So you can at least see what I meant about needing to refactor the xforms

sophiago08:04:32

(i also do realize i could use map-indexed but purposefully changed it to keep them all unary)

cgrand08:04:12

why this unary tropism? Matter of style?

sophiago08:04:03

More a weird experiment tbh. But it only adds lines 10 and 11 plus the calls to first and second in xf1. Otherwise, this looks much more legible than the version on my master branch where none of these are lifted.

sophiago08:04:42

What I would love, but am not going to do right now, is to write a macro where I can inline the xforms and have each level of mapping be like %1, %2, %3, etc.

cgrand08:04:25

what’s mul signature?

cgrand08:04:55

homogeneous arguments?

sophiago08:04:27

Yes. It's a transducer that takes and returns data.avl/sorted-map-by with a custom comparator

sophiago08:04:00

Also, I'd have to give some thought to how to handle line 18 even if I could get this to work with transduce

sophiago08:04:24

That's pretty minor though

sophiago09:04:16

(of course, it's broken)

sophiago09:04:03

Ah, that's not how partial works...

cgrand09:04:05

@sophiago is mul commutative? associative?

sophiago09:04:18

The issue now is with xf1 though. I should try rewriting it without the literal syntax and inlining it...

cgrand09:04:46

ok, I have done only some slight refactoring so far (to get the understanding)

sophiago09:04:50

I almost think the literals are creating a variable capture issue there

cgrand09:04:53

(defn chain2
  [f g order]
  (let [f (nth (iterate add-dim f)
               (dec (long (count (ffirst g)))))
        f' (diff-unmixed1 f order 1)
        g' (diff g order)
        xf1 #(*' (long (Math/pow 10 %1)) %2)
        xf2 #(get g' (transduce (map-indexed xf1) +' %))
        ; here I assume mul can be changed to a binary form and is commutative
        xf3 (transduce (map xf2) mul (multi-compose (nth f' (dec (count %))) g) %)]
    (->> order
         partition-set
         (map xf3)
         (apply add))))

sophiago09:04:37

Yes, that looks equivalent

sophiago09:04:03

_is disproving paradoxes :stuck_out_tongue: _

sophiago09:04:40

I think I just need to play the manual step-through debugging game and my version above could be used with transduce if I wanted

sophiago09:04:57

I can start from (#(map (map (map xf1 %)))) and the types are sane...composing these this way becomes rough at just one level

cgrand09:04:24

@sophiago what is partitions-set?

sophiago09:04:58

(defn partition-set [n] (->> (range 1 (inc n)) clojure.math.combinatorics/partitions))

cgrand10:04:59

n can grow very large? (I see you use “primed” operators)

sophiago10:04:50

the "prime" vs. shadowing is simply because they're derivatives

sophiago10:04:04

frankly, i was surprised composing function literals like this works so i'm guessing either the order of composition is off or there's capture going on

sophiago10:04:26

logically, i don't see why else (sequence (partial xf3 (partial xf2 xf1))) wouldn't work

sophiago10:04:01

when i use sequence the error is on count...

sophiago10:04:38

everything i've tried implies an order of evaluation issue

cgrand10:04:50

the n argument to partition-set and the *' and +' operators

sophiago10:04:10

oh, i see what you meant now

sophiago10:04:41

i don't quite understand how that could be the issue though

cgrand10:04:36

I’m just being curious

sophiago10:04:55

Well, confirming my (very broad) hypothesis about composition order: if I start eliminating lines after what should be the last map I get different errors. Same with count throwing.

sophiago10:04:21

This feels like calling eval inside a macro

sophiago10:04:28

(not sure if that makes sense)

sophiago10:04:13

I think it would be explanatory to rewrite the partials as literals: (partial xf3 (partial xf2 xf1)) should be #(xf3 (partial xf2 xf1) %) but instead it seems like (partial xf3 (xf2 #(xf1 %)))

sophiago10:04:03

Currently each xform is binary, but the first argument should be substituted in a different order from the second

sophiago10:04:49

So with partial the functions compose correctly, but are called incorrectly (I think?)

cgrand10:04:41

ah! I think I’m beginning to see what you want to achive

cgrand10:04:25

you have partitions-set wihich generates a 3-level deep structure (all-partitions, partition, component)

cgrand10:04:08

and you want to perform a computation in 3 stages, at each stage folding the deepest level

sophiago10:04:06

And to be clear, it does work...I'm just rewriting it a gazillion ways because I'm a bit insane like that

👌 4
sophiago10:04:53

It feels very much like issues I've encountered calling one macro from another

sophiago10:04:01

idk, at this point i surely need to step away from the keyboard until the answer hits me

sophiago10:04:03

but i'm very impressed you've held on to the point of even understanding the function without 500+ lines of context 🙂

cgrand10:04:18

have you worked with some langs of the APL family? this deep map and way if thinking reminds me of operator ranks etc.

sophiago10:04:06

No, actually. If I had a whole lot more free time I'd definitely play around with J

sophiago10:04:51

Other than Clojure, I primarily use Haskell so before heading down this road I slapped my head and thought "lenses" ...which is what Specter is, really. But this is really an exercise in lambda syntax so I'd prefer to avoid that.

cgrand11:04:01

@sophiago so far I can rewrite it like this:

;; mul and add needs to be transduce-friendly: 0, 1 and 2 arities
(defn chain2
  [f g order]
  (let [f (nth (iterate add-dim f)
            (dec (long (count (ffirst g)))))
        f' (diff-unmixed1 f order 1)
        g' (diff g order)
        xf1 #(*' (long (Math/pow 10 %1)) %2)
        xf2 #(transduce (map-indexed xf1) +' %)
        xf3 #(transduce (comp (map xf2) (map g'))
               mul (multi-compose (nth f' (dec (count %))) g) %)]
    (transduce (map xf3)
      add (partition-set order))))
each stage being a call to transduce to map and fold in one go Note that I moved g' call from xf2 to xf3 to make this pattern more regular

sophiago11:04:07

Thanks so much. I'll probably end up using that just because, well, quality. But it doesn't solve the theoretical problem that's driving me now, i.e. you still have each xform calling one another in the let.

cgrand11:04:56

if you want to avoid that you are going to allocate a lot of cons cells. I don’t know if I stil have it around

Igor Garcia11:04:15

Hello, guys. We've had recent issues with http-kit client and SNI (Server Name Indication) so we decided to migrate to clj-http, now I'm just trying to figure out if it fully supports SNI, and also if we can rely on the underlying retry mechanism or if we need to wrap the request with our own retry - any tips?

cgrand11:04:56

@sophiago I would personally entertain rewriting partitions to perform the computation on the fly 😉

sophiago11:04:26

I totally was about to before I saw using the combinatorics library would almost be a drop-in. But I'm really convinced that has nothing to do with this issue. I would need to desugar the let to think it through, but I'm guessing it bypasses the compiler's disallowance of nested function literals (oddly enough forking it that way is what drove me down this road, although this I'm not using that fork for this exercise) and the results are not how I reasoned them to be. I suppose I could test that using fns with different bound variable names

sophiago11:04:35

Ok, no...same issue 😕

cgrand11:04:53

(totally untested)

(defn dive [depth f & args+x]
  (let [args (into [f] (butlast args+x))
        x (last args+x)
        d (fn d [depth]
            (fn [x]
              (if (pos? depth)
                (map (d (dec depth)) x)
                (apply f (conj args x)))))]
    ((d depth) x)))

(defn chain2 [f g order]
  (let [f (nth (iterate add-dim f)
               (dec (long (count (ffirst g)))))
        f' (diff-unmixed1 f order 1)
        g' (diff g order)]
    (->> order
         partition-set
         (dive 2 map-indexed #(*' (long (Math/pow 10 %1)) %2))
         (dive 2 reduce +')
         (dive 2 g')
         (dive 1 reduce mul (multi-compose (nth f' (dec (count %))) g))
         (apply add))))

cgrand11:04:40

> the compiler’s disallowance of nested function literals what do you mean?

sophiago00:05:05

Sorry, I didn't mean to spend so long on that and had to try and get some sleep. I meant that in a literal sense: Clojure doesn't allow nested function literals and there are two lines in the compiler that enforce this. That's what this whole exercise of rewriting a working function was essentially about for me. I realized I could just lift the lambdas, but then they started to compose oddly.

triss14:04:23

so what’s more idiomatic these days? multimethods or protocols and records?

dominicm14:04:48

multimethods I think

triss14:04:38

so giving a map a :type and then dispatching on that?

ghadi14:04:00

Hard to answer a broad question with specific advice...

ghadi14:04:30

multimethods and protocols are both useful in different contexts

dominicm14:04:32

@ghadi aren't protocols/records more of a performance enhancement?

pesterhazy14:04:46

@triss, so I think that using a :type key is great unless you run into perf problems (or you're implementing your own data type)

pesterhazy14:04:45

there was a great flowchart on when to use maps vs defrecords but I can't find it now

ghadi15:04:26

that chart is useful if you already know you want to generate a new type

ghadi15:04:44

you may not need one if you can hang with multimethods

ghadi16:04:03

multimethods are good for when you need polymorphism across one method protocols allow you to have polymorphism across a bundle of methods

ghadi16:04:54

(defmulti foo ....)
;;vs.
(defprotocol Bundle
  (foo [_])
  (bar [_])
  (baz [_]))

tbaldridge17:04:35

or just use multiple multimethods

tbaldridge17:04:04

And protocols only allow for dispatch by the type of the first argument.

qqq19:04:59

Kotlin can output JVM code and JS code. *.cljc can output JVM code and JS code. Is there any example of mixing Kotlin/Clojure where both codebases generates both JVM and JS code, and can interop on both platforms?

Russ Olsen20:04:59

@triss Keep in mind that multimethods let you dispatch on anything you want, while records only look at the type. Sometimes can help make the choice for you.

👍 4
hiredman20:04:21

what makes you think it is working java?

wei20:04:36

it prints out the expected output with no exceptions

hiredman20:04:08

what dependencies does the java code pull in?

wei20:04:58

afaik just the lib in question

hiredman20:04:15

are you using the same version of java for both?

hiredman20:04:50

javax.json is newish (I am not sure if it was introduced in java 7 or 8 ), and it is part of java EE, which if I recall, is bundled with jdks, but not all jres

wei20:04:42

oh interesting

wei20:04:27

I'm running both versions on my laptop, so I assume it's the same version of java?

$ java -version
java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)

the2bears20:04:50

What if you try to import the class into your Clojure name space?

wei21:04:16

oh you know what, I included javax.json (https://mvnrepository.com/artifact/org.glassfish/javax.json/1.1.2) in deps and it worked

wei21:04:07

still not sure what the difference is with the original java code, but at least it fixed my problem. thanks guys!