Fork me on GitHub
#beginners
<
2024-02-25
>
Jim Newton15:02:09

when trying to debug some function, I'd often like to delcare that the return value must satisfy some predicate, so I can find where I'm accidentally returning the wrong kind of thing. Is there a idiomatic way of doing this? I.e., some sort of annotation on defn or on fn ?

Jim Newton15:02:30

ah yes, that will work for defn.

Jim Newton15:02:07

how about for fn?

Jim Newton15:02:59

or functions defined in letfn ?

teodorlu15:02:05

Not sure. I've never tried putting metadata on functions made with fn or letfn.

Jim Newton15:02:57

the documentation of fn and letfn don't mention it. šŸ˜ž

Jim Newton15:02:00

looks like letfn allows :pre and :post despite the documentation.

šŸ‘ 1
delaguardo19:02:25

it is mentioned in the guide.

delaguardo19:02:38

you can use macroexpand to see if the form expands to fn or not

Jim Newton15:02:27

Is the documentation for letfn wrong? https://clojuredocs.org/clojure.core/letfn ā€¢ The docstring says that that functions defined in letfn DO NOT take prepost-map? as in defn. However, my experimentation shows that they do. Am I reading the docs wrong? or are they wrong?

Jim Newton16:02:07

likewise the docstring for fn says that it does not take prepost-map?, yet my experimentation shows otherwise.

Bob B16:02:33

I'm guessing the docstring is a historical holdover: https://clojure.org/reference/special_forms#fn explains that fn (and macros that expand to fn) allow for pre/post maps

1
Jim Newton16:02:34

this what one would guess, that fn is the thing that implements the feature, not defn

delaguardo19:02:05

the doc doesn't say it "do not take prepost-map?"

Jim Newton09:03:39

hmmm. OK, I was interpreting the message to mean fnspec must be one of the following, neither of which includes prepost-map. How do you interpret the phrase? Is or intended to be non-exclusive or?

Jim Newton09:03:16

contrast this with defn in which case it is unambiguous

delaguardo09:03:37

exprs expands to all tokens that fn can accept according to the docstring. exprs => prepost-map? body

Jim Newton09:03:34

sorry, not sure what you are saying. are you telling me how letfn works, or are you telling me what the documentation for leftn states? Not trying to be a smart-ass, just trying to make sure I understand what you're saying.

delaguardo10:03:20

in the docstring for letfn: fnspec could be either (fname [params*] exprs) or (fname ([params*] exprs)+) I was referring to exprs mentioned there

Jim Newton10:03:45

ahhh, are you sure that the prepost-map is an expression? That would be surprising, I don't think it is evaluated? Is it?

delaguardo10:03:42

now I don't understand what are you talking about

Jim Newton10:03:43

to me, expressions are forms in an evaluation position.

Jim Newton10:03:06

they express a value.

Jim Newton10:03:30

Anyway, I plan to put a comment in the docs showing an example of prepost-map with letfn and fn, just to make it less ambiguous.

delaguardo10:03:42

(defn foo [x]
  {:pre [(pos? x)]}
  (inc x))

;; expands to 
(def foo
  (fn*
   ([x]
    (if (pos? x)
      nil
      (do
        (throw
         (new
          java.lang.AssertionError
          (str "Assert failed: " (pr-str '(pos? x)))))))
    (inc x))))
what is not evaluated here?

delaguardo10:03:41

btw, :pre and :post are just a convenient way to avoid adding branches in your code. The same code you could write yourself with more precise logging

ChillPillzKillzBillz16:02:18

Hello all, I am trying to write to file using data that comes over a channel. The channel has been defined as following:

(def loglivedata_chan (chan 20))
My code looks like so:
(with-open [wrtr (clojure.java.io/writer "livedatalog.json")] ;; open the file handle
  (go-loop [x (
when I put data into this channel on my repl, the following happens:
:>   (>!! loglivedata_chan (json/json-str {:a 1 :b 2}))
true
:>   
Log Live data listener is now dying...
Channel read as : {"a":1,"b":2}
nil
why is the go-loop exiting? I want it to wait for another item on the channel or quit only when the channel has been closed. Any suggestions? I also get the following error dump:
; Exception in thread "async-dispatch-6" java.io.IOException: Stream closed
; 	at java.base/java.io.BufferedWriter.ensureOpen(BufferedWriter.java:107)
; 	at java.base/java.io.BufferedWriter.write(BufferedWriter.java:224)
; 	at java.base/java.io.Writer.write(Writer.java:249)
; 	at livedataproc.MultThread$eval17615$fn__17642$state_machine__12344__auto____17645$fn__17647.invoke(NO_SOURCE_FILE:32)
; 	at livedataproc.MultThread$eval17615$fn__17642$state_machine__12344__auto____17645.invoke(NO_SOURCE_FILE:32)
; 	at clojure.core.async.impl.ioc_macros$run_state_machine.invokeStatic(ioc_macros.clj:972)
; 	at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:971)
; 	at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invokeStatic(ioc_macros.clj:976)
; 	at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:974)
; 	at livedataproc.MultThread$eval17615$fn__17642.invoke(NO_SOURCE_FILE:32)
; 	at clojure.lang.AFn.run(AFn.java:22)
; 	at java.base/java.util.concurrent.Thre
Alternatively should I open the file handle inside the go-loop? What is the most idiomatic solution to this?

hiredman18:02:43

There is a lot of not good stuff for such a short bit of code

hiredman18:02:38

You are not supposed to do io in go blocks

Bob B18:02:15

I think it's not that the go loop is exiting, but rather that it's async, and thus parking and returning. That causes the println outside the go loop to run, and the with-open block to close, thus leading to the stream closed message. Note that the stream closed exception comes from an async thread. The file handle could be re-opened each time through the go loop, but I'd guess that it'd be more efficient to sort of "buffer" some writes. You can decide/test for how much buffering makes sense in your use case, but I'd maybe call a function that'll append 'stuff' to a file, buffer stuff in the go loop, and when the buffer is full, call the function with the buffer and clear it (or something like that).

Bob B18:02:12

hiredman will probably have better suggestions than me

hiredman19:02:42

You can assume when you create a go block (go or go-loop) it runs on another thread, so you are opening a file, launching another thread to do stuff with the file, and then without waiting for that launched thread to finish, closing the file

hiredman19:02:21

go slices your code up into callbacks, and those callbacks are run on a fixed size shared threadpool, so generally it is a bad idea to gum up the pool by doing blocking/potentially blocking operations like io or whatever (even just cpu heavy code on that pool can stop other callbacks from running in a timely manner)

ChillPillzKillzBillz16:02:37

ok. So I've refactored my code. I now have 4 threads 1. main thread/data generator 2. thread for reading data from thread 1 and putting into a buffer 3. thread that keeps and eye on the buffer size; if the buffer len is above a certain size, put it into a file

ChillPillzKillzBillz17:02:48

and this doesn't work for me

ChillPillzKillzBillz17:02:07

the error message is as follows:

; Exception in thread "async-dispatch-1" java.lang.IllegalArgumentException: No matching method write found taking 1 args for class java.io.BufferedWriter
; 	at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:154)
; 	at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:102)
; 	at livedataproc.MultThread$logfilewriter.invokeStatic(NO_SOURCE_FILE:62)
; 	at livedataproc.MultThread$logfilewriter.invoke(NO_SOURCE_FILE:60)
; 	at livedataproc.MultThread$fileiothreadhandler$fn__15599$state_machine__12344__auto____15604$fn__15606.invoke(NO_SOURCE_FILE:121)
; 	at livedataproc.MultThread$fileiothreadhandler$fn__15599$state_machine__12344__auto____15604.invoke(NO_SOURCE_FILE:108)
; 	at clojure.core.async.impl.ioc_macros$run_state_machine.invokeStatic(ioc_macros.clj:972)
; 	at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:971)
; 	at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invokeStatic(ioc_macros.clj:976)
; 	at clojure.core.async.impl.ioc_macros$run_state_machine_wr
In my newbie eyes it appears the file pointer doesn't work anymore after the first attempt. I get 146 data elements written to the file... then it crashes.

ChillPillzKillzBillz17:02:18

any ideas what can be done?

hiredman17:02:06

check the arguments you are passing to .write

hiredman17:02:19

> java.lang.IllegalArgumentException: No matching method write found taking 1 args for class java.io.BufferedWriter means that looking at the argument types clojure cannot find a method that matches on the target

hiredman17:02:41

a wild guess would be you are writing stuff you pull from a channel, and when that channel closes you get nil and try to write that and the reflector cannot find a .write method it can pass nil to

ChillPillzKillzBillz17:02:58

my code looks like following

(:require [ :as io])

(defn logfilewriter [data]
  (with-open [wrtr (io/writer "livedatalog.json" :append true)] ;; open the file handle
    (.write wrtr data)) ;; write to file
    )

hiredman17:02:30

so what is data and where does it come from and is it always the same thing

hiredman17:02:52

(we know the answer is no)

ChillPillzKillzBillz17:02:59

data is json entries

ChillPillzKillzBillz17:02:19

it comes from a webservice

hiredman17:02:32

have you had logfilewriter log the type of data every time it is called?

hiredman17:02:42

(if not, how do you know?)

ChillPillzKillzBillz17:02:03

I've run the code for logfilewriter with dummy json and it works

ChillPillzKillzBillz17:02:12

should I do something else?

hiredman17:02:24

that is not the function that is throwing the error

ChillPillzKillzBillz17:02:40

besides if the data is changing then the error should be when I am agregating the buffer... no?

hiredman17:02:48

the stacktrace indicates .write is being called in a go block

hiredman17:02:11

like directly in a go block

ChillPillzKillzBillz17:02:12

yes... the logfilewriter function is getting called from within a go-loop

ChillPillzKillzBillz17:02:21

I am assuming this is wrong

ChillPillzKillzBillz17:02:35

I've never worked with async... so please excuse my ignorance

hiredman17:02:00

it is wrong(you should not do io in a block as I have said), but it is not the cause of the stacktrace

ChillPillzKillzBillz17:02:19

ok.. so please can you clarify ... where should I do io instead?

hiredman17:02:24

the stracktrace doesn't show the call to logfilewriter at all, just go block -> .write method call

hiredman17:02:48

on some other thread

ChillPillzKillzBillz17:02:03

this other thread is a go block as well?

hiredman17:02:33

either use clojure.core.async/thread or something similar on a threadpool you control or something (make sure to wait for the result)

ChillPillzKillzBillz17:02:47

so not go block but a thread will do

ChillPillzKillzBillz17:02:59

let me try that instead of wasting your time

ChillPillzKillzBillz17:02:10

I'll also try to find .write in the rest of my code

hiredman17:02:11

but the stacktrace is not the result of that

ChillPillzKillzBillz17:02:39

would you be kind enough to explain to me how you are walking thru the stack trace?

ChillPillzKillzBillz17:02:01

BTW is future a thread too? or is it too close to go and not tobe used for io?

hiredman17:02:05

frames like clojure.core.async.impl.ioc_macros$run_state_machine.invokeStatic(ioc_macros.clj:972) are running a go block

hiredman17:02:13

future is a threadpool

ChillPillzKillzBillz17:02:37

so I can do io in a future block.. right?

hiredman17:02:43

but not suitable for use with go blocks because derefing a future actually blocks

hiredman17:02:57

yes, you can do io in futures

šŸ‘ 1
ChillPillzKillzBillz17:02:12

let me try that and see what happens

ChillPillzKillzBillz17:02:36

thanks for your help!

ChillPillzKillzBillz17:02:48

I hope I won't have to bother you again

hiredman17:02:56

oh, my mistake, I missed this frame at livedataproc.MultThread$logfilewriter.invoke which is your logfilewriter

hiredman17:02:34

but when you do io you need to wait for it complete in some way, otherwise you are just spinning up an unbounded number of threads to do io

hiredman17:02:46

so for future the way to wait for it to complete is to deref it, which blocks until the future completes, but when I say "don't do io in go blocks" what I really mean is "don't block in go blocks"

hiredman17:02:51

so futures are not suitable, which is why clojure.core.async/thread exists, it returns a channel which a go block can "park" on

ChillPillzKillzBillz17:02:57

so within teh go block if I call a future with logfilewriter... it should be ok...

hiredman17:02:08

if you don't wait for the future to complete that is fine, but calling future without at some point waiting for it to complete is an anti-pattern

ChillPillzKillzBillz17:02:03

so @<future(call to logfilewriter)>

ChillPillzKillzBillz17:02:12

is what you mean by deref... yes?

ChillPillzKillzBillz17:02:39

a bit confused there... you concur... yes?

hiredman17:02:04

that is what I meant by deref, no don't do that

hiredman17:02:31

as I mentioned clojure.core.async/thread

hiredman17:02:00

(async/<! (async/thread (logfilewriter ...)))

hiredman17:02:39

something to watch out for is exceptions won't flow through there

ChillPillzKillzBillz17:02:26

ok... and this (async/<! (async/thread (logfilewriter ...))) can be inside a go... yes?

hiredman17:02:48

it has to be, <! can only be called in a go block

ChillPillzKillzBillz17:02:02

ok thanks, let me try that

ChillPillzKillzBillz18:02:59

thanks @U0NCTKEV8 after the changes I don't get any errors