#'http://clojure.java.io/copy

user=> (require '[ :as jio])
nil
user=> (doc jio/copy)
-------------------------

([input output & opts])
  Copies input to output.  Returns nil or throws IOException.
  Input may be an InputStream, Reader, File, byte[], char[], or String.
  Output may be an OutputStream, Writer, or File.

  Options are key/value pairs and may be one of

    :buffer-size  buffer size to use, default is 1024.
    :encoding     encoding to use if converting between
                  byte and char streams.

  Does not close any streams except those it opens itself
  (on a File).

no need to pull a dep

awesome, thank you!

I was running into an issue where writing to a writer would result in a corrupted file, but using http://clojure.java.io/file "path-to-file" as the destination in copy fixed it for me

One of the super things about Clojure is that the API docs (for io/copy at https://clojure.github.io/clojure/clojure.java.io-api.html#clojure.java.io/copy ) have a Source link that you can follow to see how it works. Looking over the source for io/copy, it seems to me that if you give it a Reader and a File, all it does is open a Writer, recur do the same thing with a Reader and the new Writer, and then close the Writer. So, if you got different (worse) results when you gave it a Reader and Writer, probably you either opened your Writer differently, or didn't close it?

Well I opened it the way shown in the docs' example for the writer function, like:

(defn write-file [source-stream]
  (with-open [w ( "filename.whatever")]
    (copy source-stream w)))

Eclipse Memory Analysis Tool might be a better place to start off with. How big is the heap dump?

1.4G

okay, MAT can handle it no problem

tried using MAT, but still kinda got into the same problem. I was able to see that the mem hog is a seq, but can't find which seq is doing it.

If this isn't production code, you can send me the dump I can try poking at it

That's very kind of you. It's not, it's adventofcode stuff. 😊

I think the metadata (like ns/symbol name of the PersistentQueue) should be somewhere under this tree, but I can't find it.

you're probably holding a reference to the head of a long lazy seq, want to share your code?

Or maybe memoization caused it

I think I understand now the problem. It seems to be that the space complexity is pretty bad.

(defn shortest [p]
  (loop [queue (conj PersistentQueue/EMPTY [p])
         visited #{}
         size 0]
    (let [current-path (peek queue)
          current-node (last current-path)]
      (if (destination? current-node)
        current-path
        (recur (->> (next-steps visited current-node)
                    (map (partial conj current-path))
                    (apply conj (pop queue)))
               (conj visited current-node)
               (inc size))))))

How should I approach this kind of debugging in a more realistic/complex scenario? I was hoping that the heap dump would narrow down the namespace/symbol of the culprit.

What makes you think there are empty seqs to filter out?

Partition doesn't create empty partitions

Try using prn instead of println

@U0NCTKEV8 I mean, this list, for example has nested lists inside, and some one of them are empty (I believe, the second sub-list for example)

'((5474 4920 5381 8650 11617 7193 8161) () (10747 5855 13827 6294 13437 8125) () (6913 6443 3431 5357 1579 3590 4471 4971 4055 4937 2514 1679 2917) ())

Oh, I see now, I think.

The “empty” lists are not empty they have one empty string inside.

@U0NCTKEV8 What’s a good way to filter those sub-lists with empty strings? 😅

contains? maybe

filter #(not= (first %) "") seems to work.

Or (remove #(= (first %) "")

There's an #C0GLTDB2T channel that might be better for asking about these problems...

@U04V70XH6 Yeah, I was about to do that now, actually! Thanks

I didn't know Calva could have anything to do with a process started with "lein run". Have you followed the guide at https://calva.io/connect/ ?

Have you tried https://calva.io/debugger/ ? If its as good as the Emacs Cider debug then it should do what is asked for (I've only used Cider debug myself). There are also tools such as https://github.com/djblue/portal that will help you navigate the results of functions, although this is not a step debug tool. Or break down the code to specific expressions and call them with data that should give known results and evaluate those expressions rather than the larger piece of code.

@U05254DQM @U0HG4EHMH Thank you for your responses. @U05254DQM I have tried https://calva.io/debugger/ and it works when I test it for simple functions like the ones in the tutorial, but it does not work for my main program. I reckon maybe the problem is due to running with lein. May I ask you please test if with Cider debug can you set a breakpoint somewhere in this code ( https://github.com/lspector/clojush) and see values? If it works, I will move to Cider debug. I also tried to get values in a simple run with spit, and assign values to variables manually, but as there are lots of dependencies in the main code it is hard to break it down and evaluate separately.

You first need to use Start a Project REPL and Connect (aka Jack-In) to start a repl; or start nrepl with cider middleware as described at https://calva.io/jack-in-guide/ and connect to it with Connect to a Running REPL Server in the Project. If you plan to start the repl yourself, you can run Copy Jack-In Command to Clipboard to get the command Calva would use, and then paste that into a terminal. That command also demonstrates the deps and command options you would want to store in an alias for easier setup next time.