Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2021-08-14

Channels

#clojure
<
2021-08-14
>
seancorfield01:08:58

In case anyone here is interested in #tools-build but not in the minutiae of #tools-deps and/or the CLI itself, Alex created a channel to talk about build.clj etc 🙂

ray10:08:04

Am gonna make a lib for PREPL to enable cancellation of long running form evaluation. Anyone know of previous work on this?

thheller10:08:39

nrepl interruptible-eval I guess

3
ray10:08:23

I’ll check it out. Was thinking specifically about work on prepls but maybe that will inspire too :) thanks

viesti10:08:18

here are the tricks that nrepl does https://github.com/nrepl/nrepl/blob/master/src/clojure/nrepl/middleware/session.clj#L158-L180, probably there's more to read then those highlighted lines :)

ray10:08:37

Thanks @viesti. What’s not clear to me is how state is retained after cancellation. So probably I don’t understand how state is shared across the thread pool.

thheller10:08:32

can't do interrupt with one prepl alone anyways, need a second connection since the first one is eval'ing and not reading

ray10:08:24

That’s what I aim to build - multiprepl

thheller10:08:54

also interrupts on the JVM are kinda tricky, almost nothing is actually checking the interrupt flag

ray10:08:48

With io-prepl the prepl is attached to a socket. One can close the socket and clone state

thheller10:08:42

define "state". thread bindings? or just clojure vars?

ray10:08:30

Clojure vars

thheller10:08:31

what do you mean by cloning then? the JVM doesn't exit when you intterupt the thread so all state is preserved? no need to clone anything?

thheller10:08:14

in theory the initial prepl just needs to sound out a message "hey I'm starting to eval, this is my thread id"

thheller10:08:36

and over the other connection you can then later send an eval to kill that thread if you want

ray10:08:21

interesting .. not sure if I can get the thread ID from io-prepl directly

thheller10:08:23

depending on how the kill actually happened the first connection might just loop again and let you eval again

flowthing10:08:51

FWIW, that's pretty much what I do in an editor plugin I'm making, and it works well. I just call .interrupt on the eval thread, so it won't help you if you do things like eval (range), but for me, at least, it's enough to be able to cancel a deref etc.

thheller10:08:52

more likely though the kill makes the thread unusable so you might as well kill that socket

ray10:08:46

yeah but then how to restore state?

thheller10:08:13

what state? 😛 clojure vars are global, they don't care about threads. they'll continue to exist as long as the JVM lives

thheller10:08:19

just thread bindings are lost

ray10:08:44

huh, fuck that's too obvious 🙂

thheller10:08:32

don't really need to modify io-prepl at all I think

ray10:08:38

I'll try it out later - nice

thheller10:08:47

just eval something when you connect to get the thread and store it somewhere

3
ray10:08:09

indeed, makes total sense

ray10:08:53

in my case I don't care about thread bindings

ray10:08:25

not yet at least ... famous last words

thheller10:08:53

io-prepl gets its own blocking thread so most of the stuff nrepl does you don't need

thheller10:08:19

the thread evaling is always the same for each connection

ray10:08:08

yes, somehow I didn't think to do the obvious thing of just pushing the evalled state on an atom

thheller10:08:49

what eval'd state? 😛 (def my-state "thing") you can access directly over the other connection, no work needed

ray10:08:25

lol even better, no work is the best 🙂

thheller10:08:53

the thread-bindings are also pretty much nrepl specific since it keeps all its own state in there. prepl is prettty much stateless so you probably won't need it

thheller10:08:25

unless you care about *1, *2, *3, *e which are the only bindings it maintains

thheller10:08:17

+ *out*, *err*, *in* of course but those you definitely don't care about 😉

3
😝 3
ray12:08:56

*1 etc might be interesting.

ray12:08:18

Can you let me use an atom now ;-)

🙃 3
ray22:08:34

*1 etc are still available so not a problem

ray22:08:32

(range) will not die though … thread keeps running even after closing the socket

thheller06:08:33

it should die once it fails to write to the socket? otherwise yes, (range) itself never checks the interrupt flag and happily keeps going

thheller06:08:06

the write to the socket however does so it should fail eventually

thheller06:08:42

oh .. nevermind just checked. it uses pr-str so it stops once it runs out of memory 😉

😬 3
ray08:08:35

pr-str is the default valf but could be replaced :thinking_face:

ray10:08:09

I think you can bind a limited *out*

viesti11:08:46

didn't read the implementation though

ray12:08:52

I only remember the famous blob 🙂

ray14:08:12

If I wrap pr-str in a Thread it could be killable

ray14:08:51

Or maybe a core.async channel

viesti14:08:39

could the printer be made to check for interrupt

viesti14:08:27

thinking *print-length*, but for interrupt (might not work nice work arbitrary data)

3
ray15:08:56

Yeah print-length of 1mb is probably fine for a REPL

ray15:08:42

could be tuneable by the user too

ray15:08:40

Main thing is that you avoid the dumb fu

ray15:08:59

But you can always insist :)

ray15:08:52

even a max tuneable length of 512Mb would still be way less than main memory 🙂

ray15:08:52

so yeah @viesti I'll give binding to a tuneable *print-length* a go. I'm already using a mix of core.async and futures so that the read part of the repl is not blocking ie the cancellation can occur immediately

👍 3
ray18:08:24

It works. The size of elements in range is small but the concept seems decent

ray18:08:03

Just means that cancelled threads will waste some resources

ray21:08:22

correction the *print-length* thing makes no difference… it just let me wait for the task to get killed, which - good news - it does get killed ,,,, just takes a while

ray21:08:39

so I’m gonna play with the core.async option as a replacement for directly using pr-str

👍 3
ray20:08:05

urgh, no luck … maybe I just don’t have enough core.async chops. It works even if sub-optimally, so I’m going to move on until my frustration abates 🙂

viesti09:08:03

just one battle in the war 🙂

😍 3
Jim Newton10:08:55

I've been trying to read the https://clojure.org/reference/transducers, and finally something dawned on me that I've been reading wrong. It first defines a reducing function and thereafter defines a transducer in terms of reducing function. It says a reducing function is a function like you pass to reduce and it gives a signature whatever, input -> whatever. But that's wrong I definitely do not give reduce a pair whose first component is a whatever and and whose second component is an input -> whatever. Finally I realized that what it means to say is (whatever,input)->whatever not whatever,(input->whatever). Am I the only person who was confused because of the lack of parentheses?

alpox11:08:36

I guess I just chose to read it like (whatever, input)->whatever because it says that they talk about a reducing function - and thats how I know reducing functions 😄

Jim Newton08:08:32

is omission of the parens common notation in modern literature?

alpox08:08:58

Im not very familiar with this kind of notation but I think ive seen it before without parens. I too think its confusing though.

pinkfrog11:08:55

Is there any practical difference between using an implicit namespaced keyword such as ::bar and explicitly stating out the namespace such as :foo/bar? I am mostly interested in the refactoring perspective.

Alex Miller (Clojure team)12:08:27

After read they are the identical keyword in the runtime, so it’s really more about managing what you write in your code. :: is of course shorter but can lead to surprises if you copy paste code or change the name of the namespace containing it

pinkfrog14:08:12

Thanks. I just learned the trick of ::ns/keyword where ns is some imported ns name. Given this, I will go with the ::keyword pattern, and access it in other ns with the ::ns/keyword way.