Fork me on GitHub
#beginners
<
2020-05-04
>
Cameron00:05:43

the first thing I mostly did with Clojure was write a defncurry which I actually do miss; even if so many functions have a variable number of arguments, more often than not what I have doesn't and it was just kinda nice always being able to pass around partially evaluated functions that way without using partial. I'm sure many will immediately think of theoretical problems you might come across with that, but I would just say in practice it seemed to just be nice (just as it is in Haskell), and just as it is with transducers, although I still don't want to speak too soon

Cameron00:05:33

granted, I think I made it work with varargs as well by just deciding to only curry a set amount of arguments in those cases (my phrasing for currying might be off there; I always got the language for currying mixed up and could never be assed to look it up more properly)

seancorfield00:05:31

I've never found a need for generic currying but I quite often run into a specific case where providing a subset of arguments and getting back a function of the remaining arguments is useful so I will sometimes write a specific function to return a function instead of having all its call sites be partial or #( ,,, ) expressions.

seancorfield00:05:42

Currying is inherently at odds with varargs because you have to go from (f a) returning a function to returning a value at some point, based only on the arguments you've seen so far. I gather you've given up using your defncurry macro now @zdot101?

Cameron00:05:17

Yea ahahah well there's also one more important part -- the project was specifically structured to be used that way. I likewise don't think the need really comes up for me that much either otherwise. I was coming to Clojure from Haskell, but at the time I was just intending to use Clojure as a lisp -- as in, I knew it was opinionated, but it was its lisp like flexibility I wanted to play with at the time, I figured I'd come back to the idioms later. I wanted to implement Haskell's features in Clojure, and was representing the programs in a much more Haskellish way, so I started with defncurry.

Cameron00:05:36

Granted, I think I also saw some things implemented in the Clojure source with some private, sparingly used defncurry that I was extremely surprised by (maybe it was some particular transducer), so maybe the sentiment of useful but rarely useful isn't all that uncommon

Cameron00:05:15

trying to figure out what it was or where it was though, cause at the very least I don't think that's in clojure.core

Cameron00:05:29

aye, here we go

👀 4
Cameron00:05:46

(so no, no more defncurry, that was never supposed to leave that unusual project)

byrongibby08:05:43

Could someone please help me analyse why there is such a huge time difference between realising each part of a seq and the time it takes to realise the whole seq? I have a function that fetches data from a SQL database and creates a seq of XML elements using data.xml - note the time form:

(defn compile-observations [db ns1 series-id {release :release}]
  (for [obs (if release  
                (get-obs-by-release db {:release (java-time/sql-timestamp release) :series_id series-id})
                (get-obs db {:series_id series-id}))
          :let [obs-attrs (get-obs-attrs db obs)]]
    (time
      (xml/element (xml/qname ns1 "Obs") 
                   (merge {:TIME_PERIOD (str (:time_period obs)) 
                           :OBS_VALUE (:obs_value obs)}  
                          (zipmap (map (comp keyword :attr) obs-attrs) 
                                  (map :val obs-attrs)))))))
I call the function, and then realise the output by emitting it to a string:
(def ob (compile-observations db ns11 11 {}))
(time (xml/emit-str ob))
The output I get is attached. It shows that each element of the seq takes about 0.02ms to realise and the whole seq takes about 700ms to realise. There are about 100 elements in the seq, so what accounts for the time difference? 700-(0.02*100)=688?

jsn09:05:01

xml/emit-str seems like the obvious candidate

byrongibby10:05:24

I suppose so. Although it doesn't make sense that emitting XML should be that slow. It takes 15 minutes to spit a 6MB XML file with the result that any attempt to transmit the file over HTTP just times out. Using JSON for the same amount of data would be in the seconds range not minutes.

jsn10:05:24

Why suppose though when you can measure it? You can doall it instead.

byrongibby11:05:40

So I created the following test:

(ns xml-test-app.core
  (:require [clojure.data.xml :as xml])
  (:gen-class))

(defn xml-test []
  (for [i (range 100)]
    (xml/element :MyInnerElement {:MyAttr "Attribute"} "Content")))

(defn xml-test2 [elem-seq]
  (for [i (range 100)]
    (apply xml/element (concat [:MyElement {:MyAttr "Attribute"}] elem-seq))))

(defn xml-test-doall []
  (doall
    (for [i (range 100)]
      (xml/element :MyInnerElement {:MyAttr "Attribute"} "Content"))))

(defn xml-test2-doall [elem-seq]
  (doall
    (for [i (range 100)]
      (apply xml/element (concat [:MyElement {:MyAttr "Attribute"}] elem-seq)))))

(defn -main
  "I don't do a whole lot ... yet."
  [& args]
  (do 
    (println "doall writer:")
    (time
      (with-open [out ( "test.xml")]
        (xml/emit (xml-test2-doall (xml-test-doall)) out)))
    (println "doall spit:")
    (time
      (spit "test.xml" (xml/emit-str (xml-test2-doall (xml-test-doall)))))
    

    (println "No doall writer:")
    (time
      (with-open [out ( "test.xml")]
        (xml/emit (xml-test2 (xml-test)) out)))
    (println "No doall spit:")
    (time
      (spit "test.xml" (xml/emit-str (xml-test2 (xml-test)))))))
And the results seems to be exactly opposite to what I was expecting?
doall writer:
"Elapsed time: 340.2541 msecs"
doall spit:
"Elapsed time: 183.4967 msecs"
No doall writer:
"Elapsed time: 138.026 msecs"
No doall spit:
"Elapsed time: 91.4172 msecs"

byrongibby11:05:15

The elapsed times are actually the same no matter which order you run the different versions in. The first is always 300+ms and the last around 100ms. I am so confused by this.

jsn13:05:03

well, that definitely looks like hotspot's JIT improving the generated code after it had some time to profile and observe it

byrongibby14:05:24

So it turns out that the problem is that there is one db call that is slowing things down dramatically. I still don't understand why my original timing of the code didn't show this when I used doall. Sorry for wasting your time. 😕 You live, you learn.

littleli08:05:09

What would be the most idiomatic way to make a persistent collection out of .of static constructor methods on java's Map, Set, List etc?

littleli08:05:33

oh... this works just fine :)

(merge {} (java.util.Map/of :a 2 :b 4))

byrongibby09:05:29

Could you link the javadoc for the of method?

byrongibby10:05:41

That would explain why your code didn't work for me, I am running Java 8. Does the following work?

(into {} (java.util.Map/of :a 2 :b 4))

littleli10:05:08

yes, it works as well.

littleli10:05:47

I actually need it just because it's interop thing.. so there is a java piece that gives me an instance of these.

byrongibby11:05:43

I am also a beginner, but I would say using into is more idiomatic than merge in this instance. I have had my fair share of interop hassles as well :)

littleli11:05:50

I agree. You're right. I put an example to clojuredocs here: https://clojuredocs.org/clojure.core/into#example-5eaffbcae4b087629b5a18fd

Alex Miller (Clojure team)12:05:21

Clojure maps are java.util.Maps already

Alex Miller (Clojure team)12:05:43

So you don’t actually need to call .of at all

littleli12:05:12

@U064X3EF3 sure, but that's not the issue here, right? The problem is exactly opposite, that instances created by Map.of do not implement all kind of Clojure goodness. For example you have to use (get) to access their members. The instances are specialized, or rather degenerate implementations of Map interface.

Alex Miller (Clojure team)12:05:08

So you’re starting with java colls?

Alex Miller (Clojure team)12:05:30

Sorry if I’m confused, it seemed like the question was about how to call of

littleli12:05:19

Yes. It's a bit clunky, because of course some functions are working just fine because they don't assume certain details... (map) and others actually works great with List.of or Set.of created instances.

littleli12:05:19

they cannot be used as functions, that's for sure 🙂

sroller15:05:53

@patrick.farwick I recently completed a course on Udemy which showed exactly that - a (def ..) within a (defn ..). The author was clearly not a very experienced in Clojure to say the least. But if the training helped people to get into the eco-system so be it.

😮 4
Patrick Farwick16:05:11

@sroller interesting. I am also pretty beginner so am not going to say there is never a use case, I just haven't seen one myself.

seancorfield21:05:52

(conversation moved to #clojure channel)

Joshua23:05:20

I'm working with IP addresses, prefixes, routes, etc. Is there a go to library or a standard way of representing them in Clojure?

Joshua23:05:33

Or should I use something like http://java.net.InetAddress from Java?

Alex Miller (Clojure team)23:05:39

That’s probably what I would use first

👍 4
gerred23:05:41

yep. I maintain a small conversion in for http://java.net.InetAddress that's very specific to what I'm doing. maybe I'll generalize it at some point since I'm doing some "get an IP address" and CIDR calculation work as well and that's all in Clojure.

mister_m23:05:01

what does CIDR mean

gerred23:05:41

tl;dr: "how do I take a block of IP addresses and assign a unique address from that"

mister_m23:05:47

sounds interesting

gerred23:05:09

the importance of CIDR calculations here, by the way, is if I'm working on a distributed system, I might be bound by the number of possible IP addresses I can use.

gerred23:05:24

so making sure I have a large enough range, but not too large (for other, hard but well established reasons) is important.

gerred23:05:07

but all of the core INetAddress methods I just bare wrap

Joshua23:05:48

Great, thanks

Joshua23:05:14

I'll keep it as an array of bytes until I actually need it and then I'll make some util functions

gerred23:05:03

same, I maintain it as a byte array and all of my funcs generally assume it

Aron23:05:18

I am a bit of trouble using go-loop recur in a react hook. Anywhere I try to recur, it says it can't recur there