This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-08-31
Channels
- # announcements (3)
- # beginners (139)
- # boot (28)
- # cider (40)
- # cljdoc (1)
- # cljs-dev (30)
- # clojure (61)
- # clojure-conj (1)
- # clojure-dev (113)
- # clojure-germany (4)
- # clojure-italy (29)
- # clojure-nl (3)
- # clojure-russia (2)
- # clojure-spec (38)
- # clojure-uk (53)
- # clojurescript (188)
- # core-async (4)
- # css (2)
- # cursive (7)
- # data-science (5)
- # datomic (14)
- # emacs (1)
- # figwheel-main (192)
- # fulcro (37)
- # jobs-discuss (1)
- # mount (4)
- # off-topic (47)
- # pedestal (7)
- # portkey (14)
- # re-frame (4)
- # reagent (22)
- # reitit (2)
- # remote-jobs (1)
- # ring (6)
- # shadow-cljs (65)
- # spacemacs (7)
- # specter (6)
- # yada (8)
I don't know what this unreachable athrow
is about, but it makes me happy
byte alignment?
the clojure compiler at times emits unreachable code because it doesn't do too much analysis for dead branches
o_O this seems clearly unreachable without thinking about branching it all
it seems like you'd have to go out of your way to put it there
@gfredericks if you give me the code it should be (relatively) easy to figure out why it's there
(if (.equals '~name 'clojure.core)
nil
(do (dosync (commute @#'*loaded-libs* conj '~name)) nil))
it's just an if
essentially
here's the whole method
public static void load();
Code:
0: getstatic #10 // Field const__0:Lclojure/lang/Var;
3: invokevirtual #16 // Method clojure/lang/Var.getRawRoot:()Ljava/lang/Object;
6: checkcast #18 // class clojure/lang/IFn
9: getstatic #22 // Field const__1:Lclojure/lang/AFn;
12: invokeinterface #26, 2 // InterfaceMethod clojure/lang/IFn.invoke:(Ljava/lang/Object;)Ljava/lang/Object;
17: new #28 // class user/empty_ns$loading__6515__auto____297
20: dup
21: invokespecial #31 // Method user/empty_ns$loading__6515__auto____297."<init>":()V
24: checkcast #18 // class clojure/lang/IFn
27: invokeinterface #33, 1 // InterfaceMethod clojure/lang/IFn.invoke:()Ljava/lang/Object;
32: getstatic #22 // Field const__1:Lclojure/lang/AFn;
35: checkcast #35 // class clojure/lang/Symbol
38: getstatic #38 // Field const__2:Lclojure/lang/AFn;
41: invokevirtual #42 // Method clojure/lang/Symbol.equals:(Ljava/lang/Object;)Z
44: ifeq 52
47: aconst_null
48: goto 67
51: athrow
52: new #44 // class user/empty_ns$fn__299
55: dup
56: invokespecial #45 // Method user/empty_ns$fn__299."<init>":()V
59: checkcast #47 // class java/util/concurrent/Callable
62: invokestatic #53 // Method clojure/lang/LockingTransaction.runInTransaction:(Ljava/util/concurrent/Callable;)Ljava/lang/Object;
65: pop
66: aconst_null
67: return
(I'm sure enough to bet $10 on it)
there's no exception table on the method
the dosync part is handled by calling runInTransaction
I'm stumped, I don't think the clojure Compiler emits that, it might be ASM
under the hood to make stack frames depth match but that's just a guess
ah ha, so I was right about byte alignment, sort of.
52 is a multiple of 4
does it potentially happen for all if
s?
"The Unboxed Bool Path" would be a great name for something
is there any particular reason for the use of $
instead of .
in the generated class names? Especially the first one, which seems to intentionally result in a package per parent namespace instead of a package per namespace
that's the thing I'm asking about, yes
in a way that .
is not?
example: (ns foo.bar) (class (fn yolo []))
returns foo.bar$eval325$yolo__326
I can imagine maybe foo.bar.eval325$yolo__326
where you have a package per namespace
is the only thing at play here how many packages you end up with, or is there some other aspect I'm not aware of?
I can see your point I think, your imagined example seems to reuse the package “foo.bar”
I'm not sure what you mean by "reuse"
$ is the jvm separator for inner classes
so the foo is the package, bar is the outer class, eval325 is an inner class of bar, and yolo__326 is an inner class of eval325
Yeah, that makes sense to me. foo
is the package, since the last segment (`bar`) represents a class.
but why generate that class instead of the class eval325
in the package foo.bar
?
“that” = ?
eval325$yolo__326 ?
there's a grouping decision being made here, I'm assuming -- if I have namespaces foo.bar
, foo.baz
, yolo.thomas
and yolo.wizard
, with whatever number of functions in them, in the current world by compiling these I get two packages -- foo
and yolo
. In an alternate reality that I'm asking about, you'd get four packages, one per namespace
it surprises me that we're in this world and not the alternate one
I think the grain of this is just easier in the classloader
because there is an assumption that clojure namespaces map to java packages. when in fact they map to classes
if I see namespace foo.bar, that means - go load some clj or class file
where should I find it?
if it’s a clj it could be foo/bar.clj - what else would make sense?
and it’s sane to make the class path the same foo/bar.class
yep; I see it now
packages (especially way back in Java 1.5 or whatever) were not reified in Java
they are now (kind of) to serve as a place to define various things
that makes sense; thanks
I guess you could also argue that you get a similar situation to java where there's not a 1 to 1 relationship between files and packages
there are fewer packages than files
I got stuck thinking about namespaces as groups of things, just like packages; but the other way of thinking of them is that they're groups of things, just like classes
that is better
its aggregation, not containment
(although AOT stretches that a little)
howdy! I get a “Pushback buffer overflow” using clojure.tools.reader.edn/read, with a clojure.lang.LineNumberingPushbackReader
. It looks like https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/LineNumberingPushbackReader.java#L37-L39 doesn’t pass size
to the super
constructor, so the pushback buffer is always 1, per https://docs.oracle.com/javase/8/docs/api/java/io/PushbackReader.html#PushbackReader-java.io.Reader-
some background: I had a unit test file that was getting unreasonably large, so I split it into multiple files and used load
to load individual files; one of the files had a syntax error, but nothing seemed to point to the line where the error was. I tried clojure.tools.reader.edn/read et al to try and debug this easier.
this sounds like a bug, but I'm wondering if I can reproduce it with a smaller example.
do you know of a way to make a smaller example or sanitize/deidentify the large input you already have?
I was curious for a test case exhibiting the problem, too, because I'm pretty sure many people have used that code for years without hitting that exception.
Not to say there is no bug, but it makes me wonder what about the file contents is causing that.
e.g. can we prove that 2 is always enough, or indeed any finite value, or is any such choice just kicking the can down the road?
I think the bug is that size
isn’t passed to the PushbackReader
constructor. Or possibly there should be another parameter for the pushback buffer size (since the size argument is passed as the buffer size for LineNumberReader).
also, if there is a good way (say in emacs) to anonymize a file, that would help with a test case
My belief is that the size parameter says how many "unread" method calls one can make in a row on the same PushbackReader, before you must either do a read, or else you get the "pushback buffer overflow" exception.
Even if the size parameter were passed, that just means it might be possible to pass down a size other than the default of 1. Looking through some of the EdnReader code in Clojure, I haven't yet found a code path where it obviously can call unread twice in a row.
Also, when I follow this link that you gave, I see a call to the superclass constructor that does pass the size parameter: https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/LineNumberingPushbackReader.java#L37-L39
or rather, it is passing it to the LineNumberReader constructor. I guess your point is that it isn't also a parameter to the super() call?
I am not aware of an anonymize capability in Emacs.
I have sometimes have reasonable success in reducing test cases in size by cutting it in half, then trying the first half, then the second, then repeating. I realize that might not work as easily for nested Clojure data structures
but LineNumberingPushbackReader feels broken to have no way to specify a larger pushback buffer size than 1
It might be broken. But suppose we pass it 2 and it still crashes? Then 3? Then it doesn't crash with 27. Do we commit 27 into Clojure and hope for the best, or maybe figure out why 1 doesn't work?
The exception was with tools.reader library's edn reader? Or the one built into Clojure?
Those are different code bases.
and as far as I can tell, the code you pointed at is using the size parameter to pass it on to construct a LineNumberReader with the same pushback buffer size.
tools.reader’s. I know they’re different code bases (I added the dependency for that lib to my project…)
I think I got mixed up, I thought the LineNumberingPushbackReader was an implementation of the protocols in clojure.tools.reader.reader-types
but no, the pushback buffer size is always 1, no matter the arg to LineNumberingPushbackReader — IMO the call may have been intended to be super(new LineNumberReader(r, size), size)
I did some quick REPL experiments with read/unread method calls on a LineNumberingPushbackReader, and it does seem that 2 unread's in a row raises an exception even if you call the 2-arg constructor with a size of 2.
Or perhaps even that call should be super(new LineNumberReader(r), size)
and again, this isn’t a problem with clojure itself (that I can tell); and it may not be worth changing
a cursory look at EdnReader and LispReader seems to indicate this can’t be a problem for clojure itself. I’ll back away slowly now…
It could be a bug in tools.reader. I'm taking a little bit of a look, but the code has enough cases in it that a test case to reproduce would make it a lot easier to find the problem than auditing the whole code base.
Was the exception message you saw perhaps "Pushback buffer is full" rather than "Pushback buffer overflow"? I see the former in the tools.reader code base.
I'll let you know if I think of anything else, but if you get a file you can share, plus version of tools.reader lib you are using, and which Clojure expressions you used to open the file, read it with tools.reader, and the exception you saw, that would be ideal for further debugging.
I dug for a while through tools.reader edn reader, at least the Clojure/JVM version, and have found a case where I can make it throw an exception with "Pushback buffer overflow" in the message: If you create a clojure.tools.reader.reader-types/indexing-push-back-reader from a file, and the file contains a sequence like "[a" then a carriage return character with no newline, then "b]".
If your file has carriage returns in it that are not followed by a newline or form feed character, and have a symbol, or probably one of several other things, immediately after the carriage return character, that might be the root cause of what you are seeing.
One workaround is to replace all carriage return chars with newlines.
well, or at least the ones that are not inside of strings, where you may wish to preserve them there, if you do not want to modify the contents of those strings.
I've created an issue with a small reproducing test case here: https://dev.clojure.org/jira/browse/TRDR-54
@csm If you can check whether your file that causes the problem contains some value like a symbol or number, immediately followed by a carriage return character (ASCII code decimal 13), and that is not immediately followed by a newline (ASCII decimal 10) or a formfeed (ASCII decimal 12), that would match the kind of problem I have found from code inspection and experiments. If you do not find something like that, you may have a distinct bug from the one I found.