Fork me on GitHub
#clojure
<
2022-03-20
>
respatialized16:03:19

Is there a reader or quote macro for Clojure code/forms that disables form expansion on dispatch characters like #"regex", #(+ 4 %), etc?

respatialized16:03:26

even double quoting still expands anonymous functions: (quote (quote #(+ 3 %))) ;; => '(fn* [p1__23236#] (+ 3 p1__23236#))

p-himik16:03:40

Are double quotes around the forms not an option? :) (while, of course, escaping all double quotes within)

respatialized16:03:33

Here's an example of what I'm thinking of. The existing behavior for dispatch (using anonymous functions):

(quote #(+ 4 %))
;; => (fn* [p1__23220#] (+ 4 p1__23220#))
What I am wondering about:
(no-dispatch-quote #(+ 4 %))
;; => (+ 4 %) 
or perhaps:
(no-dispatch-quote #(+ 4 %))
;; => #(+ 4 %)

p-himik17:03:15

What's the actual use case, what is the problem you're trying to solve?

respatialized17:03:58

writing clojure source code as data (as opposed to as a string) in a way that preserves what was originally written instead of that information being discarded by the reader.

p-himik17:03:44

So you need (no-dispatch-quote #(+ 4 %)) to return #(+ 4 %) and to also log that #(+ 4 %) in its original shape?

p-himik17:03:19

If that's the case, I don't think it's possible, unless no-dispatch-quote reads the source file during macro expansion. Which would of course prevent it from working in REPL or eval.

respatialized17:03:19

roughly, yes, but I don't quite know what you mean about "logging" the form apart from returning it as a value. I'd also be fine with it returning a list like (+ 4 %) if #(+ 4 %) is an unrepresentable value in clojure data.

lilactown17:03:56

I think reader macros go inside out, so I'm not sure you can do this in user land

😕 1
lilactown17:03:23

you would need a special form to do this and I don't think one exists

p-himik17:03:49

Right, #() does not exist as data. There's no point during reading when even just () exists that you could somehow reach without rewriting the reader.

p-himik17:03:44

If you don't need to use the object produced by evaluation of #(+ 4 %), then just use a string, "#(+ 4 %)". If you do need the object, either reconsider or use eval on that string.

respatialized17:03:24

the higher-level motivating use case here is treating example code as data: I would like to define a var that contains my example code and both save it for deferred evaluation and later print the example code as written. it seems like a shame to have to continually serialize and deserialize to strings when I'm trying to use code as data in my program.

andy.fingerhut17:03:55

It may be worth considering keeping the original string you read in, separate from, but "next to" and "associated with", any other forms you wish to manipulate that data.

respatialized17:03:19

FWIW this project uses both tools.reader and rewrite-clj, so I am fine with relying on non-default readers to try and support this, but as far as I can tell this specific idea is not supported.

andy.fingerhut17:03:12

I believe some of those readers have an option to return the original string that produced the read data returned, as a string, containing only those characters that produced the corresponding data.

andy.fingerhut17:03:26

Others who have used rewrite-clj can comment more precisely than I can, since I have not dug into that library, but IIRC one of its reasons for existing is to round trip from source code, edit it structurally in memory, then write out readable source code that preserves as much of the original as you want, in the parts you did not change.

respatialized18:03:48

yeah, I think ultimately I will have to leverage those capabilities to rewrite expanded anonymous functions to resemble their unexpanded counterparts. Since what I'm trying to do is effectively change how Clojure source is parsed into Clojure data, I can understand why it's not possible from the relevant section in https://download.clojure.org/papers/clojure-hopl-iv-final.pdf: > ... [reader macros] require the presence of specific code to read particular data, such requirement being in direct conflict with some of the benefits of independent read/print enumerated above. Thanks to everyone who weighed in for their thoughts.

Joshua Suskalo22:03:18

The clojure reader is inherently lossy. You could look at rewrite-clj for something like this though!

p-himik22:03:20

It has been mentioned.

👍 1
teodorlu18:03:45

Related question - how can I change the behavior of the #( reader literal? I tried a few things, but I'm doing something wrong:

user> (clojure.edn/read-string "#(+ 3 %)")
Execution error at user/eval9067 (REPL:14).
No dispatch macro for: (

user> (clojure.edn/read-string {:readers {"#" identity}} "#(+ 3 %)")
Execution error at user/eval9069 (REPL:17).
No dispatch macro for: (

user> (clojure.edn/read-string {:readers {'# identity}} "#(+ 3 %)")
Syntax error reading source at (REPL:20:55).
Unmatched delimiter: }

user> (clojure.edn/read-string {:readers {'#( identity}} "#(+ 3 %)")
                               )
Syntax error reading source at (REPL:23:56).
Unmatched delimiter: }

respatialized18:03:07

Clojure is a superset of edn; Clojure's reader does more than the edn reader by automatically expanding those anonymous function declarations into (fn* ...) forms.

respatialized18:03:32

the usual way to control these would be via *data-readers* which isn't supported in edn: (EDIT, I didn't read thoroughly enough, my mistake) https://clojuredocs.org/clojure.core/*data-readers*

👀 1
teodorlu18:03:54

The docstring says that the opt map of clojure.edn/read-string is the same as the opt map of clojure.core/read. Are you saying that doesn't apply to reader literals?

user> (doc clojure.edn/read-string)
-------------------------
clojure.edn/read-string
([s] [opts s])
  Reads one object from the string s. Returns nil when s is nil or empty.

  Reads data in the edn format (subset of Clojure data):
  

  opts is a map as per clojure.edn/read

teodorlu18:03:00

seems to work for #inst:

(clojure.edn/read-string "#inst \"1985-04-12\"")
;; => #inst "1985-04-12T00:00:00.000-00:00"

teodorlu18:03:56

(clojure.edn/read-string {:readers {'inst identity}} "#inst \"1985-04-12\"")
;; => "1985-04-12"

noisesmith18:03:53

reader macros are severely limited, you can't make one for #(...) because you need # to be followed by a valid token

👀 1
teodorlu18:03:21

So effectively you can't read reader-literal shorthand functions with clojure.edn/read-string? What I'm trying to do isn't possible?

noisesmith18:03:36

not using clojure's read or clojure.edn/read, no

noisesmith18:03:10

I mean it might technically be possible with insane hackery, but I think making your own lispy parser would be easier and less brittle

👍 1
teodorlu20:03:14

Makes sense. Thanks for explaining!

Nundrum20:03:18

I'm still hung up trying to call functions in sun.awt.X11. After adding the add-opens parameter, I can get setAccessible to work. The next step is invoking the function that connects to the X server - XOpenDisplay. But this code throws an error I can't figure out how to get around:

(def xlw (Class/forName "sun.awt.X11.XlibWrapper"))                                   
(def XOpenDisplay (. xlw getDeclaredMethod "XOpenDisplay" (into-array [Long/TYPE])) ) 
(. XOpenDisplay (setAccessible true))                                                 
(. XOpenDisplay (invoke nil (into-array Object [(long 0)])))                          
Error:
Execution error (UnsatisfiedLinkError) at sun.awt.X11.XlibWrapper/XOpenDisplay (XlibWrapper.java:-2).
'long sun.awt.X11.XlibWrapper.XOpenDisplay(long)'
Clojure: class java.lang.reflect.InvocationTargetException

phronmophobic20:03:51

UnsatisfiedLinkError is because the shared library referred to by XOpenDisplay hasn't been loaded yet.

p-himik21:03:34

If you don't know which library is needed exactly, you can wrap the call to your java app with strace and watch for ENOENT.

phronmophobic21:03:21

There's a tendency to avoid writing ffi code at all costs. Unless there's a wrapper that directly supports what you're trying to do, I think it's very difficult to coerce/hack an existing project to do the ffi part for you.

Nundrum22:03:25

I tried strace with strace -f lein run 2>&1 | grep ENOENT but can't find an obvious culprit. If I grep for lib in the output, there are 122 lines. They all look unrelated to AWT or X11.

Nundrum22:03:40

@U7RJTCH6J The only other option I see is that JNA has some sort of X11 component to it: http://java-native-access.github.io/jna/5.10.0/javadoc/ But I've had zero success with it thus far.

p-himik22:03:49

Can you share the strace file?

Nundrum22:03:51

From strace -f lein run 2>&1 | grep ENOENT > strace.out I suppose? Yeah one moment...

p-himik22:03:22

Nah, the full one.

p-himik22:03:29

Unless it's GBs in size.

p-himik22:03:13

The reason why you should not use grep in combination with -f:

181164 [pid 795074] <... newfstatat resumed>0x7f92c965dfe0, 0) = -1 ENOENT (No such file or directory)$

💡 1
p-himik22:03:32

Heh, and actually is simply tries to look up the already mentioned XOpenDisplay at a bunch of different locations. Although they for some reason don't respect the FQN and seem to only be looking at the classpath level directly. No clue why.

p-himik22:03:22

I don't know how AWT works, especially when it comes to native methods. But perhaps you have to generate the headers and C++ files, build, and link them yourself?

phronmophobic22:03:52

they should be shipped with the jvm, but not all the necessary AWT shared libraries are loaded unless they're "needed". I'm not sure which AWT calls lead to the shared libraries being loaded and it since it's an implementation detail, there may be subtle differences across jvm versions (even minor versions).

phronmophobic22:03:24

> The only other option I see is that JNA has some sort of X11 component to it: > http://java-native-access.github.io/jna/5.10.0/javadoc/ JNA just lets you call C functions in shared libraries. You would have to load the Xlib shared library and then use JNA to call its methods.

Nundrum23:03:02

I tried loading various names with System/loadLibrary but couldn't find anything that worked.

Nundrum23:03:56

I expect the AWT libs are shipped with the JRE since you can use them in Java code. I'm just digging in a layer deeper.

p-himik23:03:36

The thing is, that native XOpenDisplay(long) or whatever is trying to find XOpenDisplay.class on the classpath. Not a dynamic library, but a class for some reason. Maybe it's par for the course, but to me it seems weird.

Nundrum23:03:50

Then I must be invoking it incorrectly

Nundrum23:03:03

half the problem seems to be that it's a static method

Nundrum23:03:27

Also, either the param is either a string with the familiar "display:X" format, or it's NULL which tells Xlib to look at the $DISPLAY enviornment variable. I thought passing 0 would work, but I can't find some alternate way to pass NULL.

p-himik23:03:27

Well, that's definitely doing something wrong because you can't pass a number to something that expects String. But what do you actually want to achieve, if it's neither explicitly passing NULL nor passing "display:X"?

p-himik23:03:01

If you do want NULL and I just misunderstood you, then why not just pass nil?

Nundrum23:03:27

I'm guessing the Long it wants represents a pointer to a string. TBH I'm still figuring out JNA as I go

Nundrum23:03:43

If I pass nil, it fails to meet the requirement for passing a Long

Nundrum23:03:30

(. XOpenDisplay (invoke nil (into-array Object [nil]))) 
give me:
Execution error (IllegalAccessException) at jdk.internal.reflect.Reflection/newIllegalAccessException (Reflection.java:392).
class clojure.lang.Reflector cannot access a member of class sun.awt.X11.XlibWrapper (in module java.desktop) with modifiers "static native"
Clojure: class java.lang.IllegalAccessException

p-himik23:03:46

Oh, you're right about the pointer...

👍 1
Nundrum23:03:57

Which I swear wasn't the error I was getting before. hah

p-himik23:03:37

Given how much time you have already spent on this, I would've probably started writing some wrapper in C already if I were in your place.

Nundrum23:03:38

Ah yes I hadn't called setAccessible. The error now is:

Execution error (IllegalArgumentException) at jdk.internal.reflect.NativeMethodAccessorImpl/invoke0 (NativeMethodAccessorImpl.java:-2).
null
Clojure: class java.lang.IllegalArgumentException

Nundrum23:03:58

Again, I don't see how a wrapper in C gets me a structure I can use in AWT.

p-himik23:03:18

long is a primitive, you can't pass nil. But NULL in C is 0 - and that doesn't work for you as well.

Nundrum23:03:02

That's why I tried 0 hoping it would work

Nundrum23:03:00

It's like I need to inject a new capability into AWT, or else I need to call native C to get a X11 graphics context and pound that into something AWT can use.

Nundrum18:03:01

HA HA! I needed (System/load "/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libawt.so")

p-himik18:03:24

Does everything work now?

Nundrum18:03:28

Well the .invoke on XOpenWindow returns an integer now 😉 So that's not nearly everything, but it feels like I've broken through the barrier around XLibWrapper

🎉 2
Nundrum18:03:23

I have an ID representing the root window! Wow! The next hurdle is figuring out how to interact with the structs through JNA and pass them like pointers.

Richie22:03:54

Hey, I'm stuck getting this code to work; I'm getting an exception. I'd appreciate some help. Thanks!

(def data {:fun 'dialog
           :state {:prompt "your name:"
                   :response "enter text here"}
           :children [{:fun 'textarea
                       :state {:text ""}
                       :children []}]})

(defn textarea [{:keys [text]}]
  text)

(defn dialog [{:keys [prompt response]} textarea]
  [prompt
   (textarea {:text response})])

(defn make-my-component
  [tree]
  (fn [state]
    (eval
     (concat (list (resolve (:fun tree))
                   (merge (:state tree) state))
             (map make-my-component (:children tree))))))

((make-my-component data) {})

;; expected
;; ["your name:" "enter text here"]
exception
No matching ctor found for class examples.gui$make_my_component$fn__24015
Thanks again!

Richie22:03:01

Show: Project-Only All 
  Hide: Clojure Java REPL Tooling Duplicates  (19 frames hidden)

2. Unhandled java.lang.ExceptionInInitializerError
   (No message)

NativeConstructorAccessorImpl.java:   -2  jdk.internal.reflect.NativeConstructorAccessorImpl/newInstance0
NativeConstructorAccessorImpl.java:   77  jdk.internal.reflect.NativeConstructorAccessorImpl/newInstance
DelegatingConstructorAccessorImpl.java:   45  jdk.internal.reflect.DelegatingConstructorAccessorImpl/newInstance
          Constructor.java:  499  java.lang.reflect.Constructor/newInstanceWithCaller
          Constructor.java:  480  java.lang.reflect.Constructor/newInstance
             Compiler.java: 5000  clojure.lang.Compiler$ObjExpr/eval
             Compiler.java: 7180  clojure.lang.Compiler/eval
             Compiler.java: 7136  clojure.lang.Compiler/eval
                  core.clj: 3202  clojure.core/eval
                  core.clj: 3198  clojure.core/eval
                      REPL:  237  examples.gui.calculator/make-my-component/fn
                      REPL:  244  examples.gui.calculator/eval24036
                      REPL:  244  examples.gui.calculator/eval24036
             Compiler.java: 7181  clojure.lang.Compiler/eval
             Compiler.java: 7136  clojure.lang.Compiler/eval
                  core.clj: 3202  clojure.core/eval
                  core.clj: 3198  clojure.core/eval
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn/fn
                  AFn.java:  152  clojure.lang.AFn/applyToHelper
                  AFn.java:  144  clojure.lang.AFn/applyTo
                  core.clj:  667  clojure.core/apply
                  core.clj: 1977  clojure.core/with-bindings*
                  core.clj: 1977  clojure.core/with-bindings*
               RestFn.java:  425  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn
                  main.clj:  437  clojure.main/repl/read-eval-print/fn
                  main.clj:  437  clojure.main/repl/read-eval-print
                  main.clj:  458  clojure.main/repl/fn
                  main.clj:  458  clojure.main/repl
                  main.clj:  368  clojure.main/repl
               RestFn.java: 1523  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   84  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:   56  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:  152  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
                  AFn.java:   22  clojure.lang.AFn/run
               session.clj:  202  nrepl.middleware.session/session-exec/main-loop/fn
               session.clj:  201  nrepl.middleware.session/session-exec/main-loop
                  AFn.java:   22  clojure.lang.AFn/run
               Thread.java:  831  java.lang.Thread/run

1. Caused by java.lang.IllegalArgumentException
   No matching ctor found for class
   examples.gui.calculator$make_my_component$fn__24015

            Reflector.java:  288  clojure.lang.Reflector/invokeConstructor
           LispReader.java: 1317  clojure.lang.LispReader$EvalReader/invoke
           LispReader.java:  853  clojure.lang.LispReader$DispatchReader/invoke
           LispReader.java:  285  clojure.lang.LispReader/read
           LispReader.java:  216  clojure.lang.LispReader/read
           LispReader.java:  205  clojure.lang.LispReader/read
                   RT.java: 1879  clojure.lang.RT/readString
                   RT.java: 1874  clojure.lang.RT/readString
                      REPL:  244  examples.gui.calculator/eval24038
NativeConstructorAccessorImpl.java:   -2  jdk.internal.reflect.NativeConstructorAccessorImpl/newInstance0
NativeConstructorAccessorImpl.java:   77  jdk.internal.reflect.NativeConstructorAccessorImpl/newInstance
DelegatingConstructorAccessorImpl.java:   45  jdk.internal.reflect.DelegatingConstructorAccessorImpl/newInstance
          Constructor.java:  499  java.lang.reflect.Constructor/newInstanceWithCaller
          Constructor.java:  480  java.lang.reflect.Constructor/newInstance
             Compiler.java: 5000  clojure.lang.Compiler$ObjExpr/eval
             Compiler.java: 7180  clojure.lang.Compiler/eval
             Compiler.java: 7136  clojure.lang.Compiler/eval
                  core.clj: 3202  clojure.core/eval
                  core.clj: 3198  clojure.core/eval
                      REPL:  237  examples.gui.calculator/make-my-component/fn
                      REPL:  244  examples.gui.calculator/eval24036
                      REPL:  244  examples.gui.calculator/eval24036
             Compiler.java: 7181  clojure.lang.Compiler/eval
             Compiler.java: 7136  clojure.lang.Compiler/eval
                  core.clj: 3202  clojure.core/eval
                  core.clj: 3198  clojure.core/eval
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn/fn
                  AFn.java:  152  clojure.lang.AFn/applyToHelper
                  AFn.java:  144  clojure.lang.AFn/applyTo
                  core.clj:  667  clojure.core/apply
                  core.clj: 1977  clojure.core/with-bindings*
                  core.clj: 1977  clojure.core/with-bindings*
               RestFn.java:  425  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn
                  main.clj:  437  clojure.main/repl/read-eval-print/fn
                  main.clj:  437  clojure.main/repl/read-eval-print
                  main.clj:  458  clojure.main/repl/fn
                  main.clj:  458  clojure.main/repl
                  main.clj:  368  clojure.main/repl
               RestFn.java: 1523  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   84  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:   56  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:  152  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
                  AFn.java:   22  clojure.lang.AFn/run
               session.clj:  202  nrepl.middleware.session/session-exec/main-loop/fn
               session.clj:  201  nrepl.middleware.session/session-exec/main-loop
                  AFn.java:   22  clojure.lang.AFn/run
               Thread.java:  831  java.lang.Thread/run

Richie22:03:19

((make-my-component (first (:children data))) {}) returns ""

p-himik22:03:00

I'm not sure what's going on exactly, so I'm gonna say the most generic thing here: "Don't use eval". There doesn't seem to be a good reason for it there. You can just apply the function.

p-himik22:03:10

Yep, switching to apply produces the expected result.

🤯 1
Richie23:03:57

Woa. Thank you!

👍 1