Fork me on GitHub
#clojure
<
2019-04-20
>
yuhan11:04:55

Is there such a thing as a "lazy set" collection in Clojure?

yuhan11:04:55

I'm imagining something with constant-time contains? lookup for items that have already been realized, but which lazily defers additional elements which are not yet needed

yuhan11:04:37

(def primes
  (lazy-set (filter prime? (range 10000))))

;; Only computes as many elements as needed until a match is found
(contains? all-primes 7) ;; => true

;; constant-time, because `3` has already been found in the set
(contains? all-primes 3) ;; => true

;; Forces evaluation of the entire collection to ensure an item does *not* exist in the set
(contains? all-primes 6) ;; => false

Ivan Koz12:04:25

@qythium

(defn make-lazy-predicate
  [coll]
  (let [state (atom {:mem #{} :unknown coll})
        update-state (fn [{:keys [mem unknown] :as state} item]
                       (let [[just-checked remainder]
                             (split-with #(<= % item) unknown)]
                         (if (seq just-checked)
                           (-> state
                               (assoc :mem (apply conj mem just-checked))
                               (assoc :unknown remainder))
                           state)))]
    (fn [item]
      (let [next-value (first (:unknown @state))]
        (get-in (if (or (nil? next-value)
                        (< item next-value))
                  @state
                  (swap! state update-state item))
                [:mem item])))))

(def lazy-even? (make-lazy-predicate 
                  (filter (fn [n] 
                            (Thread/sleep 100) 
                            (even? n)) 
                          (iterate inc 0))))
However, if coll\\seq is not infinite, predicate returns false when value is out of range. Can be modified to throw exception on (nil? next-value) if you need that.

👏 4
yuhan14:04:23

Wow, thanks! I'll study that code and see if it can be modified for my needs 🙂 Creating a lexically bound atom in a closure like that is definitely something new to me

Ivan Koz14:04:05

good luck :i_love_you_hand_sign:

andrewboltachev12:04:56

Hello! Using clojure.xml, how do I output XML with double quotes, not single ones? (e.g. <foo bar="123"></foo> not <foo bar='123'></foo>)?

ivana14:04:25

Hello. Can you give me some examples of clean functions, that can be tail or non-tail recursive and made stackoverflow and/or calculated long time (better both of these) without memoization? I wanna test something.

ivana14:04:58

with arguments on which it occurs

andrewboltachev15:04:28

@ivana it's e.g.

user=> (defn boom1 [] (loop [] (recur)))
#'user/boom1
user=> (boom1)

andrewboltachev15:04:04

but what do you mean "clean" functions?

ivana15:04:06

what about without jokes?

ivana15:04:17

something like

(defn coin-change [s coins]
  (cond (= s 0) 1
        (or (< s 0) (empty? coins)) 0
        :else (+
                  (coin-change s (rest coins))
                  (coin-change (- s (first coins)) coins))))
(coin-change 10000  '(1 5 10 25 50))

andrewboltachev15:04:29

@ivana w/o jokes, I clearly remember one first videos on core.typed has given example of functions which give StackOverflow. Only I can't find that video now 😞

ivana15:04:34

thanks, I'l see

👍 4
ivana15:04:38

it seems that video explains an example of SO cause of error in code, but not correct function with limited Java stack, like my example above

martinklepsch16:04:32

I have a Clojure process that's not exiting despite me calling shutdown-agents — what else might I be missing here?

ivana16:04:57

so maybe there is another Cllojure messenger where I can ask my question? can anyone suggest?

ghadi16:04:12

@martinklepsch check what threads are running on it by calling jstack on the pid. You might have another naughty thread that isn't the agent executors

ghadi16:04:47

There are programmatic ways of enumerating the running threads, too

didibus17:04:38

@ivana Fibonacci is a good example of that. It can be written recursively with or without tail recursion.

didibus18:04:31

Without tail recursion:

(defn fib [n]
 (if (<= n 1)
  n
  (+
   (fib (- n 1))
   (fib (- n 2)))))
With tail recursion:
(defn fib [n]
 (loop [n n a 0 b 1]
  (cond
   (= n 0)
   a
   (= n 1)
   b
   :else
   (recur
    (- n 1)
    b
    (+ a b)))))   
@ivana

theeternalpulse19:04:29

is there a built in function/macro that takes a set of symbols and returns a map with keys that mirror the symbol and value being the value behind the symbol. for example (let [a 1 b 2] (toMap a b)) -> {:a 1 :b 2}

dpsutton19:04:45

there was a clojureverse discussion on this which included at least one implementation

dpsutton19:04:50

i think its a great idea by the way

theeternalpulse19:04:56

it's one of the few things I like in JS lol

Ivan Koz19:04:43

is it save to use persistent vector reference from which transient one was created?

Graham Seyffert19:04:35

Yes, (transient my-var) creates a transient version of the original var without changing the original one

ivana19:04:12

@didibus thanks. But even exponential recursive Fibonacci can not fall in SO, cause universe is died before 🙂 And iterative one is a simply tail-recursive. I wanna test my SO-problem solver I write in Clojure. I just want more test cases, for make me sure. Cause all tests I can create are passed 🙂

didibus21:04:11

Hum, I'm not sure what you mean then. Just remove the loop/recur and instead call the function back and the tail recursive one will SO.

didibus21:04:30

(defn fib [n a b]
 (cond
  (= n 0)
  a
  (= n 1)
  b
  :else
  (fib
   (- n 1
   b
   (+ a b)))))

didibus21:04:57

(fib 100 0 1)
That will SO

ivana21:04:18

Ok. I write a function, that evaluates any function on any arguments without SO and I want to test it. You write a trivial example, that evaluates also. But it can be evaluated also with recur or trampoline, in opposite with my example above. I hoped in more complex examples & cases to test

didibus21:04:10

Oh I see. So you're trying to implement real tail call optimization?

didibus21:04:41

As far as I know, there's nothing that can't be done with loop/recur and trampoline.

ivana21:04:46

No 🙂 Real TCO is a trivial task on trampolines. I implement ANY CO, not only tail. See again my example - it is not tail call

ivana21:04:49

And if you think that any can be evaluated with trampolines, can you show it on my example coin-change?

didibus21:04:32

Hum... You mean for algorithms that actually need a stack? You convert them so they don't SO?

didibus21:04:16

Ah I see, cool

didibus21:04:45

So you re-write them so their stack is allocated on the heap?

ivana21:04:49

Yes. With an optional memoization and by adding of only some chars in function code&

didibus21:04:55

You know that you can use the -Xss jvm option to increase the stack size limit in Java?

didibus21:04:20

Just mentioning it, I still think what you're doing sound cool.

didibus21:04:28

I think reversing a list is a good example

ivana21:04:30

Yes. I played with it, but max Xss more less than heap

didibus21:04:17

True, a function stack contains a lot of additional data that arn't necessarily relevant to the algorithm, so using the fn stack is always more memory hungry

ivana21:04:30

And increase default thread stack in Java is not a good idea - a lot of threads will break the heap

ivana21:04:06

reversing list is also a tco )

ivana21:04:30

in realization with accumulator I mean

ivana21:04:08

but you are right - it can be non tail call implementation

didibus21:04:45

Ya, but I often see it done without tco

didibus21:04:10

And the non tco is much more readable and logical in my opinion.

ivana21:04:16

(defn bar [n] (if (= 0 n) 0 (+ n (bar (- n 1)))))
.......
(time (eval-cs bar 100000))
"Elapsed time: 675.22869 msecs"
5000050000

ivana21:04:44

such trivial examples works fine, as I mentioned

didibus21:04:31

Ya, sorry, I very rarely implemented an algorithm that absolutely needed a stack. So I can't think of anything right now.

ivana21:04:00

Thanks anyway! I just slightly glad about my rezult and want to test and tell about it for anybody )

didibus21:04:38

Ya, if you make it open source, I'll definitely keep it in mind, if one day I have this problem. Its not a problem I have often, but I'm sure if I did, I would love to find a library like the one you're making.

yuhan21:04:03

What about non primitive-recursive functions like the Ackermann function?

ivana21:04:05

Thanks. I have to make it slightly pretty-organized, and then I'l open it. But to be honest I'm disappointed a little, that it did not interested anyone exept you )

ivana21:04:34

Gooooood point - I'l try Akkermann now

yuhan21:04:16

I believe it's theoretically proven that it cannot be reduced to a form which doesn't involve the stack

ivana21:04:26

Yes, it is not a primitive recursion. Give me some minutes to test )

yuhan21:04:34

there you go 🙂

(defn ack [m n]
  (cond
    (zero? m) (inc n)
    (zero? n) (ack (dec m) 1)
    :else     (ack (dec m) (ack m (dec n)))))

ivana21:04:01

Ok, thanks for sharing )

ivana21:04:49

Looks like it not working by directly using my method. But I'l try to modify it! Thanks again 🙂

didibus21:04:08

I think QuickSort might be another one where you can't get rid of the stack.

didibus21:04:50

Looks like generally its optimized by putting it on the heap and doing it iteratively.

ivana21:04:21

Can you show the exact code example for source?

ivana21:04:16

Naive QuickSort I mean. Or it like not real QS as Haskell examples everywhere? )

didibus21:04:31

😝 I'm afraid I'm not smart enough at this moment to put it together.

ivana21:04:26

So anyway I have to made a research with Akkermann

didibus21:04:34

Good luck 1

ivana21:04:27

Thanks again. I'l say the results about Ack and QS in a time.

ivana22:04:50

It works 🙂

(time (eval-cs ack 3 14))
"Elapsed time: 1991.828241 msecs"
131069
(time (eval-cs ack 4 1))
"Elapsed time: 853.480078 msecs"
65533

ivana22:04:26

But I was needed to slightly tweak an algorythm

ivana22:04:25

Now I'l try to find QS code your mean

didibus22:04:17

:thumbsup:

ivana23:04:33

"QS" also works w/o SO, but very (!) unefficient 🙂

(defn qs [l]
  (if (empty? l)
    []
    (let [[p & xs] l]
      (concat
       (qs (filter #(< % p) xs))
       (list p)
       (qs (filter #(>= % p) xs))))))
.............
(time (eval-cs qs (->> 10000 range reverse)))
"Elapsed time: 51633.130523 msecs"
[0 1 2 3 4 5 6 7 8 9 10 11 ..........

ivana19:04:27

If you want to see a sources, I just published a link on github in announcements

dimovich20:04:03

Hello All! I'm using clj-jade to render a Jade template, passing an array as parameter: (jade/render "index.jade" {"param1" [1 2 3]}) My friend is using Pug.js, and can do param1.length to get the size of the array, but it doesn't work when using clj-jade. How can I extent clojure.lang.PersistentVector to add a static field length that will be initialized when the vector is created?

Alex Miller (Clojure team)20:04:11

You should not want to do this

Alex Miller (Clojure team)20:04:01

Vectors implement Java Collection which has a size method and Clojure collections which have a count function.

dimovich20:04:32

@alexmiller Yes, size method works fine in clj-jade.

Ivan Koz21:04:20

@gseyffert but transient is not about vars 😃

Graham Seyffert21:04:10

Yeah it’s about seqs, but a seq will always be bound to a var?..

Ivan Koz21:04:22

its about collections

Graham Seyffert21:04:00

How is a collection different from a seq?

Ivan Koz21:04:18

sequence is abstraction

Ivan Koz21:04:21

well collection too, nvm

Ivan Koz21:04:37

however they are different entities in terms of implementation

oceanfish8122:04:47

People, I have two questions:

oceanfish8122:04:41

- who is using the Clojure client, for Kubernetes?

oceanfish8122:04:02

- which ZeroMQ bindings are used?

oceanfish8122:04:53

so two questions, the scopes of those are unrelated to each other

didibus22:04:47

@gseyffert seq are collections which support the first, next, and more fns. While collections only support count and empty and =.

didibus22:04:09

Also more relevant to Clojure, functions that work over seq takes the seq as the last argument, while those that work over coll (that aren't seq) take it as the first argument.

Alex Miller (Clojure team)22:04:00

^^ those might be helpful

Graham Seyffert23:04:56

Sure, but in the context of transients they’re used identically? Just (transient ...) to create a transient version

Graham Seyffert23:04:36

And that’s the same for seqs and collections

Graham Seyffert23:04:29

*treated vs. used

Graham Seyffert23:04:50

I’ve also just gotten into the colloquial habit of referring to a lot of things as seqs even if they aren’t technically...

Graham Seyffert23:04:20

Which is maybe worth breaking