Fork me on GitHub
#clojure-dev
<
2020-02-24
>
sogaiu03:02:41

does anyone know what the 0[0-9]+ portion of the intPat regex in LispReader.java is supposed to handle? this is right before the (N)? at the end. https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/LispReader.java#L71 i am working on tree-sitter grammars for clojure and trying to understand these types of things in detail.

Alex Miller (Clojure team)03:02:49

I guess it's just bigints with N suffix

seancorfield03:02:11

The octal bit is 0([0-7]+)

seancorfield03:02:32

That's weird because it has a leading 0 and the N is optional. but 09N is a syntax error, and 010N is read as octal anyway. Maybe it's a way to parse all possible integer tokens and then the syntax error comes later when the parsed tokens are processed?

seancorfield03:02:58

Yeah, looking at the stacktrace, I think the regex is used to determine "this is a number" and then the token is actually read and that's when illegal numbers are flagged...? LispReader line 352 throws the Invalid Number for 09 etc.

sogaiu04:02:57

i read the alternation as among: 0, base 10 things with leading non-zero, hex, octal, num with radix, and mystery item the mystery item doesn't have its own capture group so may be it is for some kind of parsing as suggested (or some other purpose). iiuc, apart from the whole capture, there are only 8 capture groups and i only see the numbers 1 through 8 mentioned in matchNumber, so whatever the mystery item is for, it doesn't look like its value is directly reflected in any resulting number. thanks for the examination.

gfredericks11:02:21

you could feed that regex to the test.chuck generator and see if it produces anything that doesn't match the same regex with the mystery portion removed

sogaiu01:02:19

our mystery seems to be resolved, but i tried using test.chuck anyway -- imagination expander :) thanks for pointing it out -- i think i may have other uses for it!

👍 4
sogaiu12:02:19

thanks for the idea!

andy.fingerhut17:02:31

With the mystery portion removed, it appears that the regex will no longer match a string like "09"

andy.fingerhut17:02:38

Maybe the mystery portion is there so "09" does match the integer pattern, because otherwise such a string would fail to match it, and then the reader would try to match it against the float pattern?

Alex Miller (Clojure team)17:02:17

possibly something like that

andy.fingerhut17:02:34

And I think the float pattern would match "09", and such a string would be read as a floating point decimal 9.

andy.fingerhut17:02:39

Yeah, quick test by modifying Clojure source code to get rid of the mystery pattern shows that (read-string "09") return a Double type 9.0, but Clojure 1.10.1 throws NumberFormatException

sogaiu23:02:39

thanks for the investigation (and the previous tips for compiling clojure from source). i also built a modified clojure and was able to see for myself :thumbsup:

andy.fingerhut23:02:45

no problem. You made me curious what was going on, and thankfully it didn't take long to find out.

🙂 4
andy.fingerhut17:02:15

The current exception behavior for reading strings of digits with a leading 0, and at least one digit 8 or 9 in them, of throwing an exception, seems far preferable to returning a floating point number of the number read as decimal, given that all other strings of digits with a leading 0 are read as an octal integer.

seancorfield18:02:53

@andy.fingerhut Thank you for taking the trouble/time/effort to actually verify what I suspected!

Janne Sauvala21:02:08

Do you know if prepl is going to see more development in the near future? I noticed a few people got really exited when it landed in the Clojure code but I haven’t heard any official announcements about it

seancorfield22:02:39

@janne What development is needed? It was designed to solve a specific problem, and it does that well.

Janne Sauvala22:02:24

Oh, could you elaborate what is the specific problem?

Janne Sauvala22:02:26

I have only used nREPL and I know some (like you) like to use the socket REPL. I was thinking that prepl was missing some features because it’s not used by that many people

seancorfield22:02:08

prepl is designed to be used by code, not people. It's designed for tooling.

seancorfield22:02:54

It has no prompt. It returns a hash map of the result value, the output (if any), the error (if any) for each evaluation. Which is what a program wants when interacting with Clojure.

Janne Sauvala08:02:55

Thanks @seancorfield. That was my hunch it’s similar to the nREPL protocol (afaik). There was good discussion at the channel some of the current problems (that other repls also have)

seancorfield16:02:01

It's not like nREPL in some really fundamental ways. It's still a pure streaming REPL like the console REPL and the socket REPL. But, yeah, it's designed for program to program communication such as tooling in the same space that nREPL exists.

4
seancorfield22:02:25

Various tooling developers are starting to use it -- it's kind of a niche feature that will really only appeal to tooling developers.

dominicm22:02:54

Some tooling authors have had issues with it and would welcome work on things like interruptible evals, nesting, etc.

seancorfield22:02:15

Yeah, I think the lack of interruptible evals is probably the thing that would concern me most as a tooling author.

ghadi22:02:45

nREPL’s “interruptible eval” is totally busted

ghadi22:02:51

Calls Thread/stop, which is deprecated for good reason

favila23:02:27

What’s the alternative?

ghadi23:02:58

abandon the computation

👍 4
ghadi23:02:05

or check Thread/interrupt status

favila23:02:31

how could it do that with arbitrary code?

ghadi23:02:39

you don't

andy.fingerhut23:02:51

Out of curiosity, what does "abandon a computation" mean that is different than calling some Thread stop/cancel/whatever operation?

ghadi23:02:52

Thread/interrupt requires cooperation from the evaluated code

favila23:02:19

I agree that it’s busted, but the only safe option is cooperative multithreading which requires the running code to buy-in (at least checking interrupt status or yielding)

ghadi23:02:30

abandon means just don't wait on the result of the code

favila23:02:49

thread/stop at least stops the cpu from spinning forever

ghadi23:02:55

it's busted in conception, not implementation

ghadi23:02:03

but it's also busted in implementation

ghadi23:02:20

please read the doc of Thread.stop

favila23:02:36

I understand why thread.stop is bad

favila23:02:32

I would also be sad if nrepl didn’t use it 🙂

👍 8
andy.fingerhut23:02:34

It seems to me maybe roughly the same amount busted as "kill -9" on a Unix process that is one of potentially many cooperating processes? I understand those aren't identical, but seems they could have similar coupling issues that affect things in a similar way, e.g. JVM object locks <-> Unix file system locks.

favila23:02:32

the surface area of object monitors seems to me much bigger than unix file system locks

favila23:02:53

you can always clear a file system lock problem

favila23:02:11

but a jvm with thread monitors in a busted state is just broken and leaking/deadlock prone forever

favila23:02:28

and every object is suspect that that thread ever referenced

andy.fingerhut23:02:58

You can delete the lock file, but problems of system invariants being violated can persist past deleting it, just as for object locks. Agreed if surface area here means JVM object locks are used far more often than Unix lock files, but not sure I see a fundamental difference between the kinds of problems that result.

ghadi23:02:29

breaking core.async channels is a big risk too

ghadi23:02:46

how do you stop (deref (promise)) ?

andy.fingerhut23:02:00

Maybe Thread.stop() is as useful/expedient in development situations, and has many possible sharp edges, just as def of a function in a running REPL is?

andy.fingerhut23:02:55

We don't think twice if we do it, because we know how to handle our chef's knives without cutting ourselves, but we are always worried others will be cutting themselves?

seancorfield22:02:13

I wonder how @mauricio.szabo does it in Chlorine? I'll go and ask...

seancorfield22:02:24

It relies on unrepl. When an evaluation is submitted, it is run in a future, and returns a function to call if you want to (try to) cancel the future.

ghadi22:02:06

that makes more sense

seancorfield22:02:06

@ghadi Do you think that is something that could be integrated into prepl? I get the impression prepl is purely synchronous at the moment, yes?

Alex Miller (Clojure team)22:02:41

No plans to work on prepl at this moment, but that doesn’t really mean anything. Could be picked up at anytime

4
seancorfield22:02:33

(there's also the issue of evaluation produces really large -- or even infinite results -- which I believe prepl just tries to return in full?)

seancorfield22:02:57

I guess both of those are "hard" problems to solve on top of the current prepl architecture?

ghadi22:02:17

@seancorfield not unless either 1) eval is parameterized or 2) the client always sends in a wrapper form for their eval form

ghadi22:02:22

(^ re: interruptibility, not re: printing)

ghadi23:02:03

oof even unrepl calls .stop on the thread

hiredman23:02:43

there isn't anything else

hiredman23:02:16

there is no other mechanism to stop a thread doing arbitrary things

ghadi23:02:18

there is abandoning the eval, or calling Thread/interrupt (which requires cooperation)

hiredman23:02:01

for eval, which is the most general of computations, there is no way to interrupt it other than calling thread.stop, as deprecated as that is

andy.fingerhut23:02:56

I said this in a thread earlier, but repeating here in case it is useful to anyone. I don't claim it is deep or anything: Maybe Thread.stop() is as useful/expedient in development situations, and has many possible sharp edges, just as def of a function in a running REPL is? We don't think twice if we do it, because we know how to handle our chef's knives without cutting ourselves, but we are always worried others will be cutting themselves?

Alex Miller (Clojure team)23:02:13

The plan is to actually remove it in the jvm soon

jumar11:02:23

That's interesting. I couldn't quickly find any details about that. The only thing I've found is the removal of destroy and stop(Throwable) [http://mail.openjdk.java.net/pipermail/jdk-dev/2018-June/001362.html] which also mentions: > > Note that the no-arg Thread.stop() method still works and won't be removed by > this changeset. It remains deprecated (though not for removal) and there are no > plans to remove it at this time.

andy.fingerhut23:02:48

Wow. For the JVM, that is really saying something.

Alex Miller (Clojure team)23:02:57

I think people don’t realize how dangerous it is

ghadi23:02:32

if you have a runaway computation initiated through the REPL, who cares? if you stop a Thread and your whole app breaks, that is worse IMHO

ghadi23:02:31

I can't count how many times have I hit Ctrl-C Ctrl-B (interrupt) in Emacs and everything breaks

Alex Miller (Clojure team)23:02:30

Actually they did already remove destroy() and one of the stop() methods in Java 11

andy.fingerhut23:02:41

I sure hope they leave kill -9 in Unix 🙂

jumar11:02:32

Which still doesn't mean you can always kill a process this way (hint: uninterruptible sleep).

ghadi23:02:48

Thread interrupts work fine for a large amount of usecases

ghadi23:02:12

but not something like infinite seqs (reduce + 0 (range))

ghadi23:02:00

(although if you tried to print an infinite seq, you could handle that through an interrupt)

andy.fingerhut23:02:43

good to know. So without Thread.stop, a thread that doesn't respond to Thread.interrupt(), and hasn't been written to poll some state asking it to quit, is only stoppable via killing the entire JVM process?

ghadi23:02:19

I think that's right

ghadi23:02:16

I'm more interested in: how can I give input control back to the initiating REPL? how can I surface a long computation? than stopping the long computation

sogaiu00:02:54

regarding input control back, i use additional socket repl connections. i guess that's not the initial repl -- what are the benefits in it being the initial repl over a new one?

ghadi23:02:55

the answer is to trick out your eval function

ghadi23:02:07

wrap clojure.core/eval

hiredman23:02:09

so you are saying, when you accidentally evaluate code that is copying the same data to a new file over and over again in infinite loop, you are more interested in "surfacing" and making that computation "visible", not stopping it?

ghadi23:02:38

"stopping" it

hiredman23:02:48

like, I think it is one thing to explain and understand why Thread.stop is a problem, but is kind of nuts to tell people they are wrong for wanting to stop bad loops running while doing iterative developement in a repl