Fork me on GitHub
#clojure
<
2018-11-13
>
borkdude08:11:42

Is it possible to write a macro that generates reader conditional code?

andy.fingerhut08:11:30

I'm not sure, but roughly reasoning from the idea that Clojure reads your code (and that is when reader conditional processing occurs), and then later compiles your code, which is when macro expansion occurs, I do not see how it is possible to write such a macro.

andy.fingerhut08:11:55

Unless you consider cases where the macro generates code that causes the reader to explicitly be invoked again on strings.

andy.fingerhut08:11:15

But don't take my guess here as authoritative on your question.

borkdude08:11:45

well, I don’t actually need to run the code in the same JVM, I only want to generate a form and emit it to a file, so I can run it later

borkdude08:11:15

e.g. this one:

(defn main-form []
  `(defn -main [& args]
     (time (run-tests 20))
     #?(:clj (shutdown-agents))))

borkdude08:11:34

where the number 20 must be variable

andy.fingerhut08:11:43

So you need to write a program that writes as its output the text of another program? You don't need a macro for that, do you?

borkdude08:11:08

the program writes a program that will be executed in a different process. let me rephrase the question: how can I generate a quoted list with a reader conditional in it

borkdude08:11:05

e.g. '#?(:clj (shutdown-agents)) doesn’t work

andy.fingerhut08:11:27

Try (reader-conditional '(:clj (shutdown-agents)))

andy.fingerhut08:11:05

Not sure if that is what you are looking for, but it may be.

andy.fingerhut08:11:07

Oops: I mean (reader-conditional '(:clj (shutdown-agents)) false)

borkdude08:11:15

figured that out 🙂

qrthey09:11:22

Once we have established that something (A) is equal to another thing (B), then why would a set containing just A not be equal with a set containing just B?

(let [jud (java.util.Date.)
      jst (java.sql.Timestamp. (.getTime jud))]
  (= jud jst))

;; => true

(let [jud (java.util.Date.)
      jst (java.sql.Timestamp. (.getTime jud))]
  (= #{jud} #{jst}))

;; => false

clj 12
andy.fingerhut17:11:20

I have spent hours writing an article and tracking down quirks in Clojure/Java's = and hash, and this one has me stumped after 3 minutes of thinking about it and experimenting at a REPL. Huh. You have me intrigued to try to find out what is going on here, because I'm not sure yet.

andy.fingerhut17:11:52

BTW, in case it might help you figure out the answer, here is that article: https://github.com/jafingerhut/thalia/blob/master/doc/other-topics/equality.md

andy.fingerhut17:11:40

The fact that the two objects are = according to clojure.core/=, and have equal clojure.core/hash return values, and yet #{jud} is not equal to #{jst}, has me baffled.

andy.fingerhut17:11:05

Another weird thing is this: (contains? #{jud} jst) returns false

andy.fingerhut17:11:04

It is definitely true that if you stick with immutable values and collections in Clojure, that = and hash behave the way you would want them to, mathematically, even for deeply nested collections.

andy.fingerhut17:11:11

at least most of the time. It seems you may have found another exception to that rule that I haven't seen before. ##NaN values in Clojure are not = to themselves, is another exception, and also make collections they are part of never =, but that exception a lot of people already know about.

andy.fingerhut19:11:17

Oh, here is another clue:

andy.fingerhut19:11:40

= is not symmetric for these two objects, probably because Java .equals method is not, either:

andy.fingerhut19:11:54

(def jud (java.util.Date.))
(def jst (java.sql.Timestamp. (.getTime jud)))

(= jud jst)
;; => true
(= jst jud)
;; => false
(.equals jud jst)
;; => true
(.equals jst jud)
;; => false
(= (hash jud) (hash jst))
;; => true
(= (.hashCode jud) (.hashCode jst))
;; => true

(= #{jud} #{jst})
;; => false
(= #{jst} #{jud})
;; => true
(contains? #{jud} jst)
;; => false
(contains? #{jst} jud)
;; => true

andy.fingerhut19:11:28

I think the root of this issue is that (.equals jst jud) is false, because of how the Java equals method is implemented for objects of type java.sql.Timestamp

andy.fingerhut19:11:48

Here is a quote from the Java documentation for class java.sql.Timestamp: "The Timestamp.equals(Object) method never returns true when passed an object that isn't an instance of java.sql.Timestamp, because the nanos component of a date is unknown. As a result, the Timestamp.equals(Object) method is not symmetric with respect to the java.util.Date.equals(Object) method. Also, the hashCode method uses the underlying java.util.Date implementation and therefore does not include nanos in its computation."

andy.fingerhut19:11:08

Clojure = does not attempt to mask the underlying behavior of Java .equals method for you, except for a few classes like Long, Integer, Byte, Short, etc.

qrthey08:11:39

Thanks for your elaborate research Andy. You are right, but it is scary that = isn't commutative in clojure for these cases. There are lots of advantages running on top of an existing large platform, but obviously som disadvantages as well...

qrthey08:11:41

This got me thinking about the ordering of associating data into the map, and shure enough;

(let [jud (java.util.Date.)
      jst (java.sql.Timestamp. (.getTime jud))]
  (assoc {jud 'ok} jst 'nok))

;; => {#inst "2018-11-14T08:24:01.039-00:00" ok,
 #inst "2018-11-14T08:24:01.039000000-00:00" nok}

(let [jud (java.util.Date.)
      jst (java.sql.Timestamp. (.getTime jud))]
  (assoc {jst 'ok} jud 'nok))
;; => {#inst "2018-11-14T08:22:49.234000000-00:00" nok}

qrthey08:11:30

And

...
(count (conj  #{jud} jst)))
;; => 2
while
...
(count (conj  #{jst} jud))
;; => 1
This all makes sense to me now, so thanks again.

andy.fingerhut18:11:57

No worries. Rich Hickey has said before something to the effect of "I can't fix Java equals behavior for you, while being a hosted language". Java equals is under the control of the decisions of others, and as Rich argues in multiple talks, equals is broken for mutable objects anyway, in deeper ways than this example you have found.

andy.fingerhut18:11:53

There isn't any reasonable way for Clojure = to be made symmetric for Java objects if Java equals is not symmetric for those objects. (I'm using symmetric as a synonym for commutative there)

orestis10:11:58

I think @qrthey that java mutable stuff might not have the same deep-equality guarantees for this.

qrthey10:11:36

both values hash to the same value

(let [jud (java.util.Date.)
      jst (java.sql.Timestamp. (.getTime jud))]
  (= (hash jud)
     (hash jst)))

;; => true

qrthey10:11:00

Same goes for list/vectors of these types

(let [jud (java.util.Date.)
      jst (java.sql.Timestamp. (.getTime jud))]
  (= [jud] [jst]))

;; => true

qrthey10:11:07

It is just that sets don't work for this. There is probably an explanation, and I would like to understand this better, but in any case this is rather unexpected.

mpenet10:11:34

it prolly defers to j.u.set .contains which will check the type

mpenet10:11:39

or something like that

qrthey10:11:09

The values are considered 'different' as they both will co-exist in a set.

(let [jud (java.util.Date.)
      jst (java.sql.Timestamp. (.getTime jud))]
  (count #{jud jst}))

;; => 2

mpenet10:11:38

yeah, diff types so that makes sense

qrthey10:11:11

And I thought functional programming would bring me closer to mathematics 😉

mpenet10:11:45

there's some trickery done with numbers to avoid this in some cases I think

mpenet10:11:12

but that's it

mpenet10:11:51

doesn't this go into: "garbage in-out" territory

borkdude10:11:06

well, probably people have looked at the source of merge and decided it’s a good conj replacement 🙂

borkdude10:11:25

but yeah, it’s probably not idiomatic use

qrthey10:11:31

Thanks for your input @mpenet and @orestis. It would be nice to have some insight from the language maintainers on the reasons for the set weirdness though.

clj 8
luposlip11:11:57

(= {:d jud} {:d jst})

;; => true

qrthey11:11:54

but as keys, they are not equal again

(let [jud (java.util.Date.)
      jst (java.sql.Timestamp. (.getTime jud))]
  (assoc {jud 'ok} jst 'nok))

;; =>
{#inst "2018-11-13T11:17:02.919-00:00" ok,
 #inst "2018-11-13T11:17:02.919000000-00:00" nok}

borkdude12:11:09

is 4clojure the website still maintained or more or less abandoned?

borkdude12:11:28

I wonder if it makes sense to move it to #clj-commons if the latter

dpsutton12:11:48

They are mutable. Imagine a set that behaved the way that you are wishing for. Have two dates that are different put into the set. Then mutate one of them so that they are equal. The set would then contain two values that are the same in violation of it's constraints

andy.fingerhut20:11:14

FYI, mutability wasn't the root cause in this case. It is that Java equals method is designed not to be symmetric for objects of type java.sql.Timestamp. They even document it as such on the Java doc page for that class.

andy.fingerhut20:11:12

It is certainly a good idea to warn people about putting mutable objects into Clojure immutable collections, of course.

kirill.salykin12:11:57

just curious, why (keys {:test "a"}) returns [:test], but not #{:test}? set semantics seems more valid in this context - no order, no duplication

borkdude12:11:03

(type (keys {:a 1}))
clojure.lang.APersistentMap$KeySeq

borkdude12:11:03

it’s a special type, probably optimized for some use cases

qrthey12:11:32

@kirill.salykin you are right, but there are some nice things about the implementation. If you also take vals from the map, you get vector semantics as well, which is nice as the order of the keys result will match the order of the vals result.

(let [v {:a 1 :b 2 :c 3}]
  (= v (zipmap (keys v) (vals v))))

kirill.salykin12:11:15

i am not sure order is applicable in this case

qrthey12:11:07

@dpsutton Thanks, that is a valid reason. It goes on to demonstrate that equality is undefined when dealing with mutable values.

kirill.salykin12:11:13

i mean I doubt order of keys garantied to be same as vals

qrthey12:11:55

The order is hard to predict, but it is stable.

Alex Miller (Clojure team)12:11:32

seq, keys, vals order are guaranteed to be the same on a map

kirill.salykin12:11:54

oh, didnt know it thanks

qrthey12:11:08

@dpsutton If only java could have been sane and dates etc had been immutable...

Alex Miller (Clojure team)12:11:19

It’s mentioned in the docstring I believe

kirill.salykin12:11:32

> “Returns a sequence of the map’s values, in the same order as (seq map).”

kirill.salykin12:11:38

this part I assume

dpsutton12:11:39

Tough thing about mutability and equality is you need to talk about "when" they are equal and when they become equal

burke13:11:28

I want to filter a list of elements in a xml file by iterating the list with zip/right and zip/remove the elements which I want to filter out. The problem is, that zip/remove will move the loc to the deepest element of the previous node. Is there an easy way to keep the loc at the same level it was before removing?

benoit14:11:03

@burke Do you know about clojure.data.zip.xml to filter zipped xml?

burke14:11:33

@benoit I thought this was only useful for "filtering" in the sense of "searching" for specific elements. I want to modify the zipper to then export it as a new xml document. I'll take a closer look to data.zip.xml, thanks

Chris19:11:57

Is there an idiomatic version of this construct? (fnil conj #{}) I have some code where I often call (update m k #((fnil conj #{}) % some-val)) to add some value to a set in a map. Is there a better way?

noisesmith19:11:47

you don't need to make the #() there - (update m k (fnil conj #{}) some-val) does the same thing

🙏 4
kaosko19:11:04

I'm trying to upgrade to clojure 1.9. I've successfully been using data.avl/sorted-maps but it returns two-elem vectors instead of actual mapentries, and 1.9 seems stricter, as (key/val %) doesn't anymore work on two-elem vectors. I can fix my code, but avl makes the same assumption internally, for example in its (subrange). is there a workardound or best practice to deal with this?

hiredman19:11:04

oof, that seems like a bad breaking change

kaosko19:11:16

yeah I was pretty surprised. I couldn't find anything about it though, but somebody else must have run into it

andy.fingerhut19:11:56

It is conceivable you are the first person to notice. I suspect Clojure's built in sorted sets are used a lot more than data.avl's.

ghadi19:11:06

going back to 1.7.0 (I didn't look further) - key doesn't work on [1 2]

ghadi19:11:26

I think it was changed temporarily during the tuple foray sha: ae7acfeecda1e70cdba96bfa189b451ec999de2e

ghadi19:11:29

so it's not a breaking change, in other words

ghadi19:11:01

is there a clojure version where key works on [1 2]?

ghadi19:11:15

ah, I should have kept looking

hiredman19:11:43

or, I dunno, I was pretty sure that it worked

kaosko19:11:45

hmm interesting. the project that uses data.avl/sortedmap has always used 1.8 AFAIK and it's worked fine, didn't think much of it

hiredman19:11:01

I must just be misremembering

hiredman19:11:32

and all the times I've done (for [x y] (key x)) y has been something that started as a map

ghadi19:11:33

for VER in 2 3 4 5 6 7 8 9; do echo 1.$VER.0; clojure -Sdeps "{:deps {org.clojure/clojure {:mvn/version \"1.$VER.0\"}}}" -e '(key [1 2])'; done

ghadi19:11:46

couldn't find one

noisesmith19:11:05

bikeshed of the shell code: $(seq 2 9)

ghadi19:11:49

thanks I changed it after having the 1.10-alphas 🙂

schmee20:11:02

is it possible to call default methods when implementing interfaces with reify?

kaosko21:11:21

sooo... sounds like key/val for 2-elem vectors is not coming back? why does (first (first {1 2})) work, or is that gonna go away as well? in the meantime, I forked and fixed data.avl for my purposes

andy.fingerhut21:11:23

From the discussion above, it isn't clear to me what "went away". Did anything change across Clojure versions here that is relevant to data.avl?

dpsutton21:11:28

Also the first first {1 2} uses the fact that mapentry implements seq which isn't going away since that would be a breaking change

andy.fingerhut21:11:50

@kaosko Ghadi's little shell script above is evidence that Clojure 1.8 also throws an exception if you try (key [1 2]) on a vector.

andy.fingerhut21:11:40

It isn't just Clojure 1.9. It is all Clojure versions since Clojure 1.2. Perhaps the root cause of the issue is something else?

kaosko21:11:49

@andy.fingerhut hmm how odd, must be something else then

andy.fingerhut21:11:16

It could be that something changed from Cloure 1.8 to Clojure 1.9 such that data.avl is now returning 2-element vectors, when it used to return map-entries?

ghadi21:11:40

I would love to see some context around the code.

ghadi21:11:44

error repro, etc.

kaosko21:11:15

yeah cljs repl prints: app:cljs.user=> (type (first (clojure.data.avl/sorted-map 0 0))) cljs.core/PersistentVector

kaosko21:11:01

whereas clj: (type (first (clojure.data.avl/sorted-map 0 0))) => clojure.data.avl.AVLNode

andy.fingerhut21:11:10

You may want to try asking on the #clojurescript or #cljs-dev channels to see if the ClojureScript developers are aware of any recent changes in that area.

andy.fingerhut21:11:26

If the issue you are seeing is specific to ClojureScript

kaosko21:11:44

yes seems so, thanks

schmee22:11:33

what a good pattern for conditionally inserting forms in a macro?

noisesmith22:11:58

[email protected](when (some-condition) ....) should work, if the ... expands to a list or vector

👍 4
schmee22:11:09

I’m trying to make a thing with reify that inserts methods if there is a key in the map passed in, so something like:

(defmacro reify-foo [opts]
  `(reify Foo
     (when ~(opts :foo)
       (foo [this]))))

schmee22:11:19

but I’m running into all kinds of issues with quoting

noisesmith22:11:31

I would have done [email protected](when (opts :foo) ...)

noisesmith22:11:04

because your version always has the foo inside the when in the output form, which won't work with reify

schmee22:11:21

here’s the actual thing:

(defn websocket-listener
  ([{:keys [on-binary on-close on-error on-open on-ping on-pong on-text]}]
   `(reify WebSocket$Listener
      [email protected](when on-binary
          (onBinary [this ws byte-buffer last?]
            (on-binary ws byte-buffer last?))))))

noisesmith22:11:21

then you might need quoting around (foo [this] ...) thanks to the surrounding unquote, but that's not hard to do

schmee22:11:48

I think the intention should be clear there, but the quoting is real

noisesmith22:11:15

yeah, I think you need another backtick around (onBinary ...)

noisesmith22:11:30

otherwise the macro tries to execute onBinary which is likely nonsense

schmee22:11:31

ahh, here we go! 😄

(defn websocket-listener-fast
  ([{:keys [on-binary on-close on-error on-open on-ping on-pong on-text]}]
   `(let [ob# ~on-binary]
     (reify WebSocket$Listener
       [email protected](when on-binary
           `(onBinary [this# ws# byte-buffer# last?#]
              (ob# ws# byte-buffer# last?#)))))))

schmee22:11:01

that is some incomprehensible code right there

noisesmith22:11:32

sometimes breaking some of it out into a helper function can make it clearer what's happening

noisesmith22:11:13

minimal example of the principle though

Clojure 1.9.0
(ins)user=> (defmacro foo-object [with-bar?] `(reify Object [email protected](when with-bar? `[(toString [this] "bar")])))
#'user/foo-object
(ins)user=> (foo-object true)
#object[user$eval150$reify__151 0x23fb172e "bar"]
(ins)user=> (foo-object false)
#object[user$eval154$reify__155 0x5e1fa5b1 "[email protected]"]

lilactown22:11:53

I wish reify was more data-y

noisesmith22:11:27

@schmee are you sure that onBinary inside the when isn't missing another layer of nesting for the @ to consume?

noisesmith22:11:33

@lilactown at least macros are 100% data-y :D

schmee22:11:30

this one is tested and actually works 😂

(defn websocket-listener-fast
  ([{:keys [on-binary on-close on-error on-open on-ping on-pong on-text]}]
   `(let [~'ob ~on-binary]
     (reify WebSocket$Listener
       [email protected](when on-binary
           [`(~'onBinary [_# ws# byte-buffer# last?#]
              (~'ob ws# byte-buffer# last?#))])))))

noisesmith22:11:10

I wouldn't have noticed except that same issue tripped me up in my minimal example (and the error you get when expanding the macro is really unhelpful...)

mhuebert23:11:07

@schmee I don't know if I understand the whole context of what you want to do, but you may be able to use a function instead of a macro and conditionally specify! the protocols/methods onto a plain object at runtime

noisesmith23:11:09

is specify! new?

mhuebert23:11:38

Or... maybe it is just a ClojureScript thing

schmee23:11:44

seems like it

mhuebert23:11:49

In which case never mind :)