Fork me on GitHub
#clojure
<
2019-06-22
>
markx05:06:47

Hi, I’ve been suffering from this stack overflow for a long time but couldn’t figure out why. What’s wrong with my code?

(defonce history-dict (atom '()))
(defonce message-dict (atom '()))

(defn add-completion-candidates!
 ([coll] (add-completion-candidates! coll message-dict))
 ([coll d]
  (log/debug (prn-str coll))
  (swap! d (comp seq #(take 5000 %) distinct #(apply conj %1 %2)) coll)))

(defn dict []
  (seq (distinct (into @history-dict @message-dict))))

(defn completer []
  (proxy [Completer] []
    (complete [reader cli candidates]
              (when-let [cs (not-empty (dict))]
                (.addAll candidates
                         (seq (map #(Candidate. %)
                                  cs)))))))
Sometimes, when I hit TAB, and this complete is called, it gives stackoverflow.

markx05:06:47

The error is not helping though.

#error {
 :cause nil
 :via
 [{:type java.lang.StackOverflowError
   :message nil
   :at [clojure.lang.PersistentHashMap$BitmapIndexedNode index "PersistentHashMap.java" 681]}]
 :trace
 [[clojure.lang.PersistentHashMap$BitmapIndexedNode index "PersistentHashMap.java" 681]
  [clojure.lang.PersistentHashMap$BitmapIndexedNode assoc "PersistentHashMap.java" 692]
  [clojure.lang.PersistentHashMap assoc "PersistentHashMap.java" 143]
  [clojure.lang.PersistentHashSet cons "PersistentHashSet.java" 99]
  [clojure.lang.PersistentHashSet cons "PersistentHashSet.java" 17]
  [clojure.lang.RT conj "RT.java" 677]
  [clojure.core$conj__5390 invokeStatic "core.clj" 85]
  [clojure.core$distinct$step__6413$fn__6414$fn__6416 invoke "core.clj" 5052]
  [clojure.core$distinct$step__6413$fn__6414 invoke "core.clj" 5048]
  [clojure.lang.LazySeq sval "LazySeq.java" 42]
  [clojure.lang.LazySeq seq "LazySeq.java" 51]
  [clojure.lang.RT seq "RT.java" 535]
  [clojure.core$seq__5402 invokeStatic "core.clj" 137]
  [clojure.core$take$fn__5909 invoke "core.clj" 2884]
  [clojure.lang.LazySeq sval "LazySeq.java" 42]
  [clojure.lang.LazySeq seq "LazySeq.java" 51]
  [clojure.lang.RT seq "RT.java" 535]
  [clojure.lang.RT nthFrom "RT.java" 981]
  [clojure.lang.RT nth "RT.java" 940]
...

markx05:06:31

I thought it was because of thouse lazy-seq, so I use seq everywhere to realize them, but am still getting the error.

seancorfield05:06:03

seq doesn’t realize the whole thing so you’re still getting lazy sequences built up.

seancorfield05:06:29

Replace seq with vec in both functions to make it eager.

seancorfield05:06:53

(although (vec (map ...)) can be just (mapv ...))

markx05:06:57

really! I didn’t know that. Why would seq return a lazy seq?

seancorfield05:06:26

seq only does enough work to distinguish an empty sequence from a non-empty sequence.

markx05:06:49

Also if I call those functions in repl, it works fine. It only throws error when in the app. Why is that?

markx05:06:09

And also, for future reference. Is it true that, whenever I realize a lazy-seq that’s very long, it will stack overflow?

markx05:06:02

I still don’t know what’s causing the stack overflow

seancorfield05:06:27

The REPL forces evaluation of everything so it can print it.

seancorfield06:06:23

Not all lazy seqs cause stack overflow on realization, regardless of size. It depends on how they are constructed.

seancorfield06:06:08

In your case, you were essentially creating lazy sequences of nested lazy sequences so you hit a stack overflow fairly quickly there because of the nesting, not the length.

markx06:06:22

Yeah that’s helpful! But could you point me to where the lazy seqs are being nested? As in which function?

seancorfield06:06:48

@markx Every time add-completion-candidates! runs it nests one lazy sequence inside another.

seancorfield06:06:48

The value swapped into the atom is a lazy sequence that contains the previous lazy sequence.

markx06:06:05

OK I see. So every time I add more candidates, I need to realize the whole seq. Thanks man!

Noah Bogart19:06:05

quick question, cuz i'm struggling a bit: if I use defrecord and then implement a protocol function within it, can I wrap the record in an atom and then pass that atom into protocol's function? or do I have to deref before passing it in?

andy.fingerhut19:06:44

Pretty sure you would have to deref it. There are very few places in Clojure where things are deref'd for you.

didibus19:06:37

You need to deref for sure

didibus19:06:09

The protocol uses the type to dispatch to the correct implementing function

didibus19:06:20

The type if wrapped in an Atom would be Atom.

didibus19:06:38

But you could implement the protocol for Atom, and have it deref.

Noah Bogart19:06:48

ah, that makes sense, thanks!

didibus19:06:44

Like this:

(defprotocol Foo
  (foo [this]))

(defrecord Bar [name]
  Foo
  (foo [this] (:name this)))

(extend-type clojure.lang.Atom
  Foo
  (foo [this] (foo @this)))

csd22:06:28

why do you extend-type both Atom and and clojure.lang.Atom ?

didibus23:06:36

Oh, that's a copy/paste mistake 😛

didibus23:06:22

I fixed it

andy.fingerhut19:06:14

@deleted-user I am pretty sure I am remembering correctly that I have seen multiple times that some spec users prefer to create namespaces that have nothing but specs in them, and those namespace names are used as the namespace part of the spec name.

didibus19:06:26

What you're doing is pretty weird

didibus19:06:34

The specs are global

didibus19:06:45

You don't need to re def them in each namespace you use them

didibus19:06:14

You can just do ::g/width

didibus20:06:04

Normally what I do then, is I have a namespace of common specs ya. Like I'd create a: (s/def :labyrinth/pos-int (s/and int? pos?))

didibus20:06:17

BTW, you do know there is already pos-int? in Clojure?

aaron5120:06:38

How do I specify a maven/clojars dependency on the command line with clojure CLI? Is that possible w/o a deps.edn file?

Alex Miller (Clojure team)20:06:28

yes, use -Sdeps '{:deps {DEP {:mvn/version "VERSION"}}}'

aaron5120:06:02

Of course! Thank you

danielcompton23:06:12

As a rough estimate, how many working Clojure programmers worldwide do people think there are? I've always guessed around 10,000, but I'm not really sure if that's too high or low