Fork me on GitHub
#clojure
<
2016-06-21
>
josh.freckleton01:06:06

I'm trying to recreate the tree examples on https://en.wikipedia.org/wiki/Tree_traversal using clojure.walk and I must not understand prewalk/`postwalk`, because the output from clojure doesn't match what I'd expect, especially on the postorder traversal, am I doing something wrong?:

(def nums3 [:F [:B [:A [:D [:C :E]]]]
            [:G [nil [:I [:H nil]]]]])
(clojure.walk/prewalk-demo nums3)
(clojure.walk/postwalk-demo nums3)

seancorfield01:06:52

What part of the output doesn’t match what you expect?

seancorfield01:06:38

(I just ran your code in a REPL and the output was what I would have expected)

josh.freckleton01:06:02

From wiki, I would expect something from which I could see the following order: Pre-order: F, B, A, D, C, E, G, I, H. Post-order: A, C, E, D, B, H, I, G, F.

josh.freckleton01:06:27

Post order's not clear to me how clj is the same as the wiki version

seancorfield01:06:31

Remember that Clojure doesn’t modify nodes in-place — which is what I think that wikipedia page is assuming; instead Clojure’s walk builds up a new tree based on the results of calling the walking function on each node and leaf of the tree in order.

josh.freckleton01:06:09

hm, that makes sense, but I must still not quite get it, since I would expect the leaves to be visited (on the postorder) like wiki's: A, C, E, D..., should they not be visited in that same order?

josh.freckleton01:06:34

(preorder visits the leaves in the expected order...)

seancorfield01:06:22

Both are defined in terms of clojure.walk/walk — which is defined in terms of inner/outer functions — and walks any data structure (not just trees).

josh.freckleton01:06:08

cool, I have the source code pulled up and I'll peruse walk then to see where I'm going wrong!

seancorfield01:06:32

Ah, I think I see where the disconnect is: your "tree" is just a nested vector, so the result you get will depend on your representation...

josh.freckleton01:06:28

is there a better representation? I just started looking into tree-seqs, never used em before though.

seancorfield01:06:13

I guess I’d ask: what problem are you trying to solve?

josh.freckleton01:06:19

I'd like to construct a tree, and then process (reduce) it from the leaves-up. EG reduce that example tree into a string, such as ACEDBHIGF

codonnell01:06:58

@josh.freckleton: you might want to check out specter

seancorfield02:06:58

Your nums3 tree seems to have three nodes at the top level and two everywhere else…?

seancorfield02:06:34

(just an observation)

josh.freckleton02:06:16

@codonnell: interesting, checking it out now @seancorfield: hm, good point, I fixed it, but same issue 😞

josh.freckleton02:06:18

(def nums3 [:F [[:B [:A [:D [:C :E]]]]
                [:G [nil [:I [:H nil]]]]]])
(clojure.walk/prewalk-demo (z/seq-zip nums3))
(clojure.walk/postwalk-demo (z/seq-zip nums3))

josh.freckleton02:06:45

alas, i think the reduce function I need needs to be hand-coded, here's preorder, and I'll work on a postorder reduce:

josh.freckleton02:06:01

(defn reduce-preorder
  "reduce a zipper with preorder traversal
  reducing function: (fn [x coll] ...)"
  [f coll zipper]
  (loop [zipper zipper
         coll   coll]
    (if (z/end? #spy/p zipper)
      coll
      (if (z/branch? zipper)
        (recur (z/next zipper) coll)
        (recur (z/next zipper) (f (z/node zipper) coll))))))

(reduce-preorder (fn [x coll] (str coll x))
                 ""
                 (z/vector-zip nums3))

seancorfield02:06:47

I have to say I don’t see how you’d get ACEDBHIGF out of nums3 with either preorder or postorder traversal

nathanmarz02:06:35

@josh.freckleton: fyi, here's how to do the postorder traversal with Specter:

(def nums3
  [:F [[:B [:A [:D [:C :E]]]]
       [:G [nil [:I [:H nil]]]]]])

(declarepath TreePostOrder)

(providepath TreePostOrder
  (multi-path
    [ALL vector? TreePostOrder]
    [ALL (complement vector?)]
    ))

(select TreePostOrder nums3)
;; => [:C :E :D :A :B :H nil :I nil :G :F]

josh.freckleton02:06:43

@nathanmarz: oh awesome, that looks like what I need! @seancorfield: thanks for taking a look at it, I was just surprised since the wiki looks pretty straightforward, but I couldn't replicate it... I think I'm on the right path though thanks to you, @nathanmarz and @codonnell . Thanks all!

codonnell02:06:24

@josh.freckleton: if you have more questions (or don't), feel free to stop by #C0FVDQLQ5

seancorfield02:06:08

Yeah, the difference here is that Specter knows what a "tree" is whereas clojure.walk just knows about built-in Clojure data structures. Right @nathanmarz ?

nathanmarz02:06:23

Well that definition of TreePostOrder encapsulates what we mean by "tree" in this context

nathanmarz02:06:31

It's defining "post order tree walk" in terms of composition of basic navigators

nathanmarz02:06:33

Specter has no innate concept of"tree"

seancorfield02:06:09

What format is it assuming for the non-leaf nodes of the tree?

seancorfield02:06:17

That would be leaves, but I see what you mean. The providepath declaration defines how to interpret the data structure (as a tree)

nathanmarz02:06:33

oh yea, my bad

lvh02:06:36

If I have a macro returning an arbitrary expression (not just a var), can I attach :tag metadata on it to prevent reflection?

seancorfield02:06:36

The issue here is that you can define trees in several different ways. [data left-subtree right-subtree] for example or [left-subtree right-subtree] and have non-vectors be leaves (but the nodes then can’t have data).

seancorfield02:06:26

That’s what confused me about the original nums3 since it looked like [data (:F) left-subtree right-subtree] but lower levels only had two elements...

seancorfield02:06:24

The tree traversal description on wikipedia assumes that triple representation (i.e., nodes can have data as well as subtrees)

nathanmarz02:06:40

that representation would probably be clearer with maps, e.g. {:data :F :left ... :right ...}

atul03:06:39

Hi all, greetings of the day! I was playing with some code from "Clojure in action" second ed.

user> (defn square [x]
        (* x x))
user> (class (map square (range 11000)))
clojure.lang.LazySeq

user>  (map square (range 11000)) ; Shouldn't it need a take? We are just mapping over a lazy seq
(0 1 4 9 16 25 36 49 64 81 100...)

atul03:06:07

Or does the REPL makes it eager?

jimmy03:06:48

@atul: If you wrap your code, others can help you easier.

atul04:06:37

@nxqd: Sure - thanks! Will edit it right away... 🙂

danielcompton04:06:44

@atul the REPL eagerly evaluates something if it is returned

atul04:06:22

@danielcompton: Thanks! So the REPL thinks I want the results right away! Hence goes and converts it to eager evaluation?

danielcompton04:06:44

If you def it, then it won’t be evaluated

fenak04:06:55

I’m still trying to get my head around clojure. if I have N maps like: {:a 1 :z 2} {:a 8 :z 9} and i want to make just one map with the sum of :a and the average of :z, how should i approach this problem?

fenak04:06:23

sorry about the emojis..

kauko04:06:53

There's an open source library I use in my own project, and I'd like to add a feature to it (and do a PR). If possible, I'd like to clone the library repo, set it up as its own project, and instead of the "official" version, use that cloned and altered version as the dependency in my personal project. That way I could make changes, and test whether they work easily. So the question is, can I use local files as a dependency in Clojure? 🙂

kauko04:06:35

Or alternatively, does this kind of workflow even make sense? 😄 Is there a better way?

nathanmarz05:06:40

@kauko try lein install

cskksc05:06:43

@fenak: something like this would work. Although I would create helper functions for this if its being used in more than 1 places

(let [xs [{:a 1 :z 2} {:a 8 :z 9}]]
  {:sum-a (->> xs (map :a) (apply +))
   :average-z (->> xs (map :z) ((fn [nums] (float (/ (apply + nums) (count nums))))))})

kauko05:06:04

@nathanmarz: hmm. So the library I was talking about is devcards. I'd clone its repo, do some changes, then run lein install in there? How then do I use that local version as a dependency in my own project? Sorry, I tried googling, but you can guess "lein install" gives other results than what I'm looking for 😄

cskksc05:06:02

@kauko: When you lein install after making the changes, the updated library is installed in your local maven repo. You could add a -U flag, like so lein -U clean, run, so that leiningen picks up the updated library

kauko05:06:45

local as in "local to my machine", or local to the repo?

cskksc05:06:21

local to the machine. In most cases, its path is ~/.m2/repository

kauko05:06:58

woot, awesome 🙂 Thanks, I'll try it out later!

lewix05:06:11

What's a good implementation of the selection sort algorithm in clojure

seancorfield05:06:12

Are you trying to write one just as an exercise in learning Clojure?

lewix05:06:44

seancorfield: kind of, and I'm actually curious to see how it would look like

seancorfield05:06:06

@lewix: Well, you're going to need to be able to find the minimum value in a sequence and return that sequence with that element at the beginning, then recursively do that for the rest of the sequence.

lewix05:06:58

@seancorfield: let me try to put something together and you give me feedbacks

ebaxt05:06:34

We’re struggling with a bug where a nested map lookup fails (it returns an “old” value) when using a lookup function from a 3.party lib, but works when we in-line the lookup function in the calling ns. It only happens after some load. Any ideas what can cause this?

seancorfield05:06:17

@ebaxt: Not without seeing the code... Can you put it on http://refheap.com and share the link?

seancorfield05:06:17

@lewix: Sure. Tonight, or tomorrow 🙂

ebaxt06:06:03

@seancorfield: It’s one of the weirdest issues I’ve seen.. we managed to connect to the running system with a REPL and added a bunch of printlns. The request that goes into (friend/current-authentication) is definitely correct, but the returned value is from a previous call. We only use the 1-arity version of current-auth, so the dynamic identity var in friend isn’t the problem either. It’s starting to look like some kind of JIT-bug where the entire call is inlined incorrectly (I know that sounds speculative, but that’s how weird it is).

seancorfield06:06:55

@ebaxt: Yeah, I don't see anything obviously wrong there... sorry.

atul06:06:29

@cskksc: Thanks for the "thread-last" macro example! Cool one!

fenak06:06:29

thanks @cskksc , very helpful! o

fenak06:06:53

i’ve started to do some stuff with clojure and clara-rules, but still having some hard times figuring stuff out

cskksc06:06:53

@fenak: I highly recommend solving some http://www.4clojure.com/ problems 🙂

zzamboni07:06:32

@fenak: I can fully agree with @cskksc - I've mostly finished #C0M8PCF7U and started going through the 4clojure problems, it really helps to solidify the knowledge and bring it into a more practical perspective.

zzamboni07:06:21

Only problem is that 4clojure is running Clojure 1.4.0 AFAICT, so I have encountered some slight differences from latest documented behavior (e.g. "interleave" cannot be used with 0 or 1 arguments)

grav08:06:02

I’m on my slow computer, and I get a timeout in every project when lein repling. I can add :timeout to each project file, but that sucks. What are the alternatives? Can I supply a timeout from the command line?

tap08:06:26

What about adding to global ~/.lein/profiles.clj ?

grav08:06:23

@tap that’s a good suggestion. I’m just not a fan of global configuration. It always seems to bite you

grav08:06:54

But I guess if there isn’t a command line option, that’s the next-best thing

tap08:06:41

I'm not aware of the way to give command line option for leiningen. Anybody? Do you always have timeout problem even with newly created empty project? Is it possible that there’s code run during repl startup which doesn’t need to? Any form that is not wrapped with defn? def with computational expensive expression?

grav09:06:04

@tap good question. it’s just some services that I start up, haven’t implemented them myself. Normally, when I’m on a faster computer, it isn’t an issue.

grav09:06:24

But it is definitely smelly that they’re so slow to boot

grav09:06:11

There’s a home-made macro which is under my suspicion, but don’t tell my co-worker 😉

firesofmay09:06:03

Hey, Is this valid?

> (cheshire.core/parse-string "{\"\": 42}" keyword)
{: 42}
Why does this return empty keyword? Or simply:
> (keyword "")
:

borkdude09:06:56

@firesofmay: yes, empty keywords or keywords of whitespace characters are legal

borkdude09:06:16

maybe legal is not the right word. nothing prevents them from being created

firesofmay09:06:00

@borkdude: How do you access that though?

borkdude09:06:46

(get m (keyword ""))

lmergen10:06:30

anyone is using emacs + cider + midje ? what do you use to execute a specific test from within emacs, midje-mode ?

lmergen10:06:31

ah, i thought midje-mode was broken, but apparently there's a pretty easy fix

bg11:06:44

@borkdude: re. the empty keyword thing, is it documented anywhere?

borkdude11:06:31

@bg: I looked at the documentation of edn, and it says keywords follow the rules of symbols and that they should begin with a non-numeric character

borkdude11:06:57

@bg: however, this doesn't work, so I don't think it's expected: (read-string (pr-str (keyword " ")))

borkdude11:06:29

@bg: I have asked about this before and the answer from the core team then was: Clojure doesn't protect you from doing things wrong

Petrus Theron12:06:06

Runtime exception after adding dependency [com.taoensso/carmine "2.13.0”] to compojure project, which also required newer encore [com.taoensso/encore "2.58.0”], throws:

java.lang.RuntimeException: Method code too large!, compiling:(ring/adapter/jetty.clj:20:1)
Some dependency conflict?

bg12:06:57

@borkdude: yeah. But it does become an issue if you're reading arbit JSON data.

borkdude12:06:05

@bg: doesn't that library have some sort of json validation?

borkdude12:06:17

@bg: it's weird json to begin with

bg12:06:07

It is weird. Adding a validator will add latency, @borkdude

Petrus Theron12:06:19

If I take out [com.taoensso/encore "2.58.0”], the error goes away.

bg12:06:04

@borkdude: the bigger question is what should keyword do in such cases. Not an easy call to take.

borkdude12:06:30

@bg: you'd probably want to validate it against some spec anyway as it is external data?

bg12:06:37

@borkdude: it's user generated data with an extremely loose spec. Anyway, thanks for the food for thought. Hammock time!

meowy12:06:12

Bit of a weird question, but...

meowy12:06:21

user> (slurp "")
SunCertPathBuilderException unable to find valid certification path to requested target  sun.security.provider.certpath.SunCertPathBuilder.build (SunCertPathBuilder.java:141)

meowy12:06:19

How come java version "1.8.0_92" doesn't like Let's Encrypt's certs?

meowy12:06:06

This is on Windows, by the way... Chrome and even Internet Explorer handle the cert perfectly fine.

meowy12:06:21

Ah, seems to be a known issue with Oracle's JDK, as I expected...

meowy12:06:44

Planned for 8u101 which comes out next month, pweh.

niwinz12:06:39

@meowy: it depends, I currently running the 1.8.0_92 on my linux laptop and (slurp "") works properly

meowy12:06:49

Oracle SDK?

meowy12:06:56

Or OpenJDK?

niwinz12:06:58

Not, openjdk

meowy12:06:09

That explains it, because it seems to be an issue with Oracle JDK.

niwinz12:06:34

yes, because oracle jdk comes with own trust strore and openjdk uses the system one

niwinz12:06:54

that is more updated than the oracle builtin truststore

meowy12:06:28

Yup... but on Windows, you only get the Oracle JDK, and that one uses that outdated keystore.

meowy12:06:27

I should be less lazy and actually reboot to Linux for Clojure dev. :P

lewix15:06:20

feedbacks please:

lewix15:06:06

(defn remove-one [item list]
  (let [[n m] (split-with (partial not= item) list)] (vec (concat n (rest m))))) 
       
(defn sorting-sort [unsorted]
   (loop [inner-unsorted unsorted sorted []]
     (let [min-val (apply min inner-unsorted)]
       (if 
         (or 
           (= 1 (count inner-unsorted)) 
           (empty? inner-unsorted))
         sorted
         (recur (remove-one min-val inner-unsorted) (conj sorted (apply min inner-unsorted)))))))

Alex Miller (Clojure team)15:06:28

@bg (keyword “ “) is not “doing things wrong” imo. Keywords are a language feature with two levels of support. At one level, keywords with certain constraints have read and print support. At another level, keywords can be programatically created and used without those constraints. So it’s intentional and useful for (keyword “ “) to succeed and there are many applications that rely on this capability. If you want to create a keyword subject to the reader constraints, why not use the reader? Something like (defn str->kw [s] (read-string (str ":" s))) - will either return a printable/readable keyword or throw. You could also explicitly check for those reader constraints or embellish that further of course.

Petrus Theron15:06:40

Has anyone used quartzite to schedule Quartz jobs? I am in a world of pain with triggers firing at unexpected times. I wrote down my woes on SO: http://stackoverflow.com/questions/37948850/quartz-ignores-time-of-day-for-daily-schedule-when-today-matches-day-of-week

trptcolin16:06:35

@petrus: i think this may be a situation you could handle with what quartz calls "misfire handling": http://www.nurkiewicz.com/2012/04/quartz-scheduler-misfire-instructions.html

trptcolin16:06:36

imagine if the process went down across your scheduled time - you might want it to go ahead and fire

trptcolin16:06:01

but i'm not sure you can distinguish between that case ^^^ and the one you describe

luxbock17:06:44

I have about 3gb of text files, each file is on average 1-4mb and for each file I'd like to read it, split it by regexp and then process each part and collect the results into one big collection

luxbock17:06:14

should I be reaching for core.async or core.reducers?

luxbock18:06:37

I'm guessing that the files are not big enough where using reducers on a single file would give me any gains

mpenet18:06:27

@luxbock: aphyr/tesser could be an option too

nonrecursive19:06:03

@luxbock http://clojure.org/reference/reducers mentions that reducers are good when “Work to be performed is computation (not I/O or blocking)” however I’ve found that you still get a great speed improvement using reducers to process many largeish files

nonrecursive19:06:24

@luxbock so one option is to parallelize the operation “read file, split by regex, process parts” with reducers

luxbock19:06:32

@mpenet: thanks, will check it out

luxbock19:06:45

@nonrecursive: yeah that's what I think I'll end up doing

luxbock19:06:34

I was playing around with the idea of having a core.async process reading and splitting files, and then adding those parts into a channel, which has a transducer that processes each part

luxbock19:06:43

and then using pipeline to try to parallilize it

nonrecursive20:06:52

@luxbock yeah that sounds cool 🙂 one thing I like about the reducers lib is that it’s so easy and quick to just try it and see if you get good performance

flyboarder20:06:33

Hello 🙂 Is clojure.main for clr documented anywhere, im trying to get a repl going by following Clojure.main.cs but that exits after running.

seancorfield20:06:18

@flyboarder: there’s a #C060SFCPR channel that may be more help? Not many people in it tho’...

flyboarder20:06:37

@seancorfield: thanks, ill drop a line in there

berrysoup22:06:35

how to load pom.properties from jar file? I have io error: java.io.FileNotFoundException: /home/jacek/ClojureProjects/sesame-loader/target/triple-loader-standalone.jar!META-INF/maven/adalab/triple-loader/pom.properties

berrysoup22:06:18

for java I'd user ResourceLoader from spring framework... 😛

bcbradley23:06:41

does anyone know of any decent graphics libraries for clojure? I am looking for something that exposes an api centered around immutability, like the rest of clojure.

bfabry23:06:03

@berrysoup: err except that documentation is confusingly wrong at first glance. just (slurp (io/resource

noisesmith23:06:28

@borkdude @bg: my take on things like (keyword "") is that you shouldn't be auto-keywordizing runtime input. It easily gives you nonsense values (as you've seen). Strings as keys in hashmaps is fully supported, and is much more likely to actually give reasonable results.