beginners 2024-05-27 | Slack Archive

Edward Hughes05:05:55

Would anyone happen to have an idea why line-seq has trouble reading from a server-socket wrapped by io/reader ? I'm doing the codecrafters project to re-implement redis, and while I can loop-recur over the socket with .readLine to receive a message string "*1\r\n$4\r\nPING\r\n", trying to use line-seq in any fashion completely fails to return a result. I can try into'ing it into a vector to realise it which gives me an empty string, re-implement the whole thing eagerly, but the best I can get is the function calling line-seq or equivalent trying to read the socket endlessly looping, which is even more confusing to me because line-seq should bottom out when it hits a nil while reading the stream.

hiredman05:05:12

Line-seq is lazy and you are not realizing it until after the reader has been closed

Edward Hughes06:05:47

Can you elaborate? I've tried (doall (line-seq (io/reader socket))) and (into [] (line-seq (io/reader socket))) I haven't actually put the reader in a with-open binding, so I'm not sure how it can be closed when I'm not closing it manually with the java method.

didibus07:05:48

Not sure, but how would line-seq know it's "over", so if you just call doall wouldn't it be infinite? At least till maybe the socket closes or returns a kind of EOF ? Try just calling first maybe ?

Edward Hughes07:05:43

line-seq recursively calls itself, and pops the next line off the reader to check it using a when-let [line (.readLine rdr)], so when you hit EOF, it should come back false and exit out the loop, no?

Edward Hughes07:05:38

trying to do any kind of access/slicing on the lazy-seq returns empty when run against the test battery, I think cause for some reason it never exits from the line-seq recursion

didibus07:05:25

Ya, but since it's a socket, what would cause it to EOF? Seems it would just block until it receives another message no?

Edward Hughes07:05:24

I mean, I figured I could get around that by checking if the socket is ready using the java method when I was fiddling around, but that didn't work either

didibus07:05:31

If you read from the socket, and no new lines are present, it'll likely block. Lazy seq will likely try to fetch 32 lines because of chunking.

didibus07:05:22

Try to send more messages to the socket, at least 32. That should return from first called on line-seq I think.

Edward Hughes07:05:13

I'll give it a shot, thanks

hiredman16:05:54

it for sure isn't a chunked-seq thing, seqs don't magically become chunked and the seq returned by line-seq is not chunked

hiredman16:05:54

it is always the case that for a given reader (first (line-seq rdr)) and (.readLine rdr) would return the same string

hiredman16:05:19

the difference between (.readLine rdr) and (doall (line-seq rdr)) is the .readLine will return the first read line and the realized line-seq will be all the lines read from rdr and block until rdr is closed

didibus16:05:58

Can't you accidentally chunk though? If you end up doing an operation over top that chunks it? Or are all the seq function going to respect that the source seq is unchunked and keep it so?

exitsandman16:05:18

iirc the operations will preserve seq "chunkiness"

exitsandman17:05:13

Honestly though I think most of the time as a beginner you'll want to look for eager alternatives, mostly because laziness and its implementation in Clojure is a weird beast in a language that already is a bit of a departure from the usual landscape, and as the language evolved the lazy part of sequences turned out to be much less of a salient point than expected. Which is to say, when in doubt you're probably better off with sticking with a loop collecting into a vector and/or looking into IReduce and friends.

didibus18:05:36

So it'll respect when it's not chunked as well and keep it so? OR just when it is chunked it will keep it chunked?

hiredman18:05:54

it depends what you mean

hiredman18:05:39

something to keep in mind that is easy to get tripped up about when using seqs with things like map and filter, etc is each processing step is building a new seq

hiredman18:05:54

the way chunked seq support works is they are regular seqs, you can process them via first and rest, but they have additional operations to get the first chunk and the rest of the chunks (chunk at a time instead of item at a time)

didibus18:05:14

That's what I mean, like if the initial seq is unchunked and then you map, filter, take, drop, parition, etc. Will they all respect that the initial was unchunked and keep it that way?

hiredman18:05:23

yes

hiredman18:05:45

"keep it that way" meaning the seq that they build is also not chunked

👍 1

Nim Sadeh17:05:13

hiredman17:05:59

println doesn't print to stdout, it prints to *out* which can be anything, so just because you see println output somewhere that doesn't mean all stdout output is going there

hiredman17:05:16

in particular some tooling like nrepl binds *out* to capture what is printed when evaluating forms to send it back to the client

Pablo19:05:51

Hi, I just installed [copilot](https://github-com.translate.goog/copilot-emacs/copilot.el) on emacs, but I’m constantly spammed with this error:

⛔ Warning (copilot): copilot--infer-indentation-offset found no mode-specific indentation offset.

Thats because I need to define identation for Clojure. There are some defaults:

(defcustom copilot-indentation-alist
  (append '((latex-mode tex-indent-basic)
            (nxml-mode nxml-child-indent)
            (python-mode python-indent py-indent-offset python-indent-offset)
            (python-ts-mode python-indent py-indent-offset python-indent-offset)
            (web-mode web-mode-markup-indent-offset web-mode-html-offset))
          editorconfig-indentation-alist)
  "Alist of `major-mode' to indentation map with optional fallbacks."
  :type '(alist :key-type symbol :value-type (choice integer symbol))
  :group 'copilot)

Is clojure-align equivalent to what is defined on other languages?

andy.fingerhut04:05:37

If you do not get an answer here, you could consider posting your question in the #C099W16KZ channel as well. If you do, it would be good to preface it with a statement like "Originally asked in #C053AK3F9, but there was not response in a day, so trying here in #C099W16KZ"

Nim Sadeh20:05:59

hey I am wondering if I can get help with some incredibly janky core.async code. There's probably a million ways to make it better, but I am specifically stuck on something weird to do with the order things are executing in. Here's the main entrypoint function. This does the following: 1. Setup scrapers - these just loop through values of a channel and scrape a an object (process/process) for each ID. 2. Setup a channel with increasing IDs, from which the scrapers get their next job to do 3. Await until all the scrapers are done. Currently, not checking if a scraper was done because it finished all the work (

(defn scrape
  []
  (let [next (chan)
        statuses (chan)
        scraper (fn []
                  (go-loop
                   [next-id ( (Thread/currentThread)
                                       (.getName))
                           :next-id next-id)
                    (if next-id
                      (if (try
                            (process/process (str next-id))
                            (catch Exception e
                              (µ/log ::error
                                     :message (.getMessage e)
                                     :data (ex-data e))))
                        (recur (! statuses :error))
                      (>! statuses :done))))]
    (go-loop
     [starting-id 180000]
      (if (<= starting-id 5000000)
        (do (>! next starting-id)
            (recur (inc starting-id)))
        (do (close! next))))
    (dorun (map (fn [_]
                  (scraper))
                (range 20)))
    (loop [counter 1]
      (


The code for process:

(defn process
  [document-id]
  (µ/with-context {:document-id document-id}
    (go (if (

When I run the code, I only see the log lines for the ::executing-thread event, and nothing for e.g., the ::already-stored-document event. I thought it had to do with the with-context call, but if I put a dummy log statement just under it and before the go loop, it logs just fine. I can see based on my S3 bucket's state that data is coming in, but I don't see any of my logs, event while the code is running for minutes. I understand that async logging means that things might not happen in any particular order, but I would like to see the progress of my application. Is there a way to do so?

Also would love any feedback about how I set up this job, it's clear to me that it's weird but not sure how to do better.

hiredman20:05:20


it is complicated, but in general it is best to assume everything inside a (go ...) is running on another thread, and the thread that starts it doesn't wait around for it

hiredman20:05:11


so depending on what with-context does, assuming it does some setup and then some clean up, the cleanup will have run by the time the code inside the (go ...) has run

hiredman20:05:30


you have a <!! blocking call in a go block btw

Nim Sadeh20:05:50


Thanks, it used to be in a thread rather than a go block. I’ll fix it

2024-05-27

Channels