Fork me on GitHub
#clojure
<
2017-07-23
>
beders00:07:54

Going back to my question from this morning to introspect a Java object, here's a fn I've written to expose the fields of an object via reflection (clojure.reflect really didn't help). I've crammed it all into a single fn and it looks quite ugly. Any suggestions on how to improve the code style. I'm still a clnewb?

(defn java->map
  "Turns fields of a Java object into a map, up to 'level' deep"
  ([obj] (java->map obj 1))
  ([obj level]
   (when (some? obj)
     (let [c (class obj)]
       (cond
         ;;;(.isPrimitive c) obj never works because clojure implicitly wraps primitives
         (contains?
           #{java.lang.Long java.lang.Character java.lang.Byte
             java.lang.Double java.lang.Float java.lang.Short java.lang.Integer} c) obj
         (= 0 level) (.toString obj)
         (instance? java.lang.String obj) obj
         (.isArray c) (concat
                        (->> obj
                             (take 5)
                             (map (fn [e] (java->map e (dec level)))))
                        (when (> (count obj) 5) [:more (count obj)]))
         :else
         (assoc (into {} (->> (concat (.getDeclaredFields c)
                                      (.getFields c))
                              (filter #(= (bit-and (.getModifiers %) java.lang.reflect.Modifier/STATIC) 0)) ;;; ignore static fields
                              (map
                                #(do (.setAccessible % true)
                                     [(keyword (.getName %))
                                      (java->map (.get % obj) (dec level))]))))
           :-type c  ;;; add the type as well
           ))))))
Examples:
(java->map (java.util.Date.) 4)
=> {:fastTime 1500771066642, :cdate nil, :-type java.util.Date}

(java->map (java.text.AttributedString. "bubu") 2)
=>
{:text "bubu",
 :runArraySize 0,
 :runCount 0,
 :runStarts nil,
 :runAttributes nil,
 :runAttributeValues nil,
 :-type java.text.AttributedString}

sophiago02:07:07

I'm having trouble adding metadata inside a macro. It's a bit weird (as usual) since it overrides defn in order to capture the ast after compilation, but works otherwise:

(defmacro defn [name & decls] `(def ^{:ast ~decls} ~name (fn ~decls)))

noisesmith02:07:47

defmacro never sees the reader macro, it's applied before it sees the form

noisesmith02:07:05

you need to use with-meta or vary-meta to add metadata to the symbol you emit

sophiago02:07:35

I did try that, but perhaps didn't do it correctly

sophiago02:07:55

(and good call because I definitely don't want this done at runtime)

noisesmith02:07:00

^{:foo :bar} baz translates to (with-meta baz {:foo :bar})

sophiago02:07:18

lemme give it another go and post the results if it doesn't work

sophiago02:07:53

ok, so yeah so for some reason the actual fn part doesn't compile when i write it like that:

(defmacro defn [name & decls] `(def ~(with-meta ~name {:ast ~decls}) (fn ~decls)))

noisesmith02:07:32

nested ~ isn't valid like that

noisesmith02:07:47

it needs to be one of those positions or the other, not both

sophiago02:07:44

oh. i tried unquote splicing it, but then got an error on the macro

noisesmith02:07:59

splicing isn't valid there either

sophiago02:07:12

is it enough to just unquote the expression? that works

sophiago02:07:23

but same error when calling the macro

noisesmith02:07:05

the next problem is that decls isn't going to be a valid data literal

noisesmith02:07:16

so it needs to be escaped or quoted in some manner

sophiago02:07:01

hmm. it worked fine without the metadata

noisesmith02:07:29

right but the decls are a data literal in the metadata - so they need to be a valid one

noisesmith02:07:36

+user=> (defmacro defn [name & decls] `(def ~(with-meta name {:ast (cons 'quote decls)}) (fn ~decls)))
#'user/defn
+user=> (defn baz [])
#'user/baz
+user=> (meta #'baz)
{:ast [], :line 19, :column 1, :file "NO_SOURCE_PATH", :name baz, :ns #object[clojure.lang.Namespace 0x373ebf74 "user"]}
user=> 

sophiago02:07:37

oh that one

sophiago02:07:47

i confused the error message

noisesmith02:07:20

wait I think cons ins the wrong way to do that... checking

noisesmith02:07:56

yeah, my bad

+user=> (defn baz ([]) ([_]))
CompilerException clojure.lang.ExceptionInfo: Wrong number of args (2) passed to quote {:form (quote ([]) ([_]))}, compiling:(NO_SOURCE_PATH:21:1) 

noisesmith02:07:35

this fixes an issue with multi arities in the original too

+user=> (defmacro defn [name & decls] `(def ~(with-meta name {:ast (cons 'quote (list decls))}) (fn [email protected])))
#'user/defn
+user=> (defn baz ([]) ([_]))
#'user/baz
+user=> (meta #'baz)
{:ast (([]) ([_])), :line 25, :column 1, :file "NO_SOURCE_PATH", :name baz, :ns #object[clojure.lang.Namespace 0x373ebf74 "user"]}

noisesmith02:07:23

there's probably a better way to rewrite`(cons 'quote (list decls))` - that's super ugly

sophiago02:07:43

it works if i just do (list 'quote decls)

noisesmith02:07:52

oh, right, much nicer

sophiago02:07:31

wow, thanks for explaining that to me as always

sophiago02:07:41

definitely would not have picked up on that on my own

noisesmith02:07:05

yeah, there's an art to this stuff, and seeing simple examples makes a difference in learning it

sophiago02:07:51

tbh i never cared much about macros before the project i'm using this for

sophiago02:07:04

it's funny because in scheme i'm pretty sure i'd do some fancy transformation like closure conversion, but that's not possible since clojure is JIT compiled at the function level. but otoh, i don't think i could do it this way with scheme-style macros

sophiago02:07:35

now i get to play around with techniques for parsing the asts. i'll check out the muir library, but suspect i'll end up having to hack away at it myself

chrisbroome06:07:21

is there a "built-in" absolute value function? I couldn't find it on https://clojure.org/api/cheatsheet . I came up with

(if (neg? x) (- x) x)

chrisbroome06:07:47

lol of course

chrisbroome06:07:55

(I'm not a java programmer)

spangler07:07:11

Non-obvious legacy weirdness. abs would be nicer

spangler07:07:50

along with sin, cos, sqrt, pow, exp etc etc

chrisbroome07:07:01

i was definitely expecting abs

chrisbroome07:07:39

that would be nice because it could work on longs, floats, rationals, and bignums

noisesmith15:07:28

@chrisbroome @spangler the rationale of just using java.lang.Math is most people using those things extensively are doing performance critical stuff where the overhead of a clojure wrapper that dispatched on and preserved types would be a problem

chrisbroome19:07:35

that makes sense. one thing it might be useful to have a clojure-defined abs, sin, etc for is cross-target support. yeah they might be slower but at least the code would work without modification

chrisbroome19:07:37

by cross-target I mean clj, cljs, cljn (is that the .net suffix)?

noisesmith19:07:47

Math/foo works for clj and cljs already - not sure what the .net scenario is (since js already wanted to be like java it implemented the math functions with the same api)

gmercer12:07:44

cljr for Clojure CLR

hmaurer15:07:21

Does anybody have a good online resource / talk / book to recommend to learning clojure’s internals?

hmaurer15:07:42

Implementation details, how it interacts with the JVM, etc

hmaurer15:07:57

What will this teach me? Doesn’t it just contain the std lib?

noisesmith15:07:26

it's everything implementing the language, except the special forms and the parser and the bytecode emitter

schmee15:07:38

It’s an entry point to learn about the internals

schmee15:07:49

see where it calls into java / compiler stuff and dive in from there

hmaurer16:07:09

@U3L6TFEJF ok perfect then, thanks! 7000LOC isn’t so bad actually

hmaurer16:07:33

Out of curiosity, where is Clojure’s STM implemented? (for refs, etc)

schmee16:07:07

here’s another good one about namespaces: https://www.youtube.com/watch?v=8NUI07y1SlQ

hmaurer17:07:07

I have a function (`refresh`) which returns either :ok or an exception (without throwing it). I am calling refresh in a do block. If refresh returns :ok, I would like to continue the execution, but if refresh returns an exception I would like to return that exception

hmaurer17:07:49

I could do this with a (let [result (refresh)] (if (= result :ok) ... but am wondering if there is a more idiomatic way

hmaurer17:07:08

What the refresh function actually does is I think irrelevant to answer my question, but in case anyone is wondering, it’s the one from https://github.com/clojure/tools.namespace

tdantas19:07:39

hey guys, I’m building a bot application as a service (websocket based , slack bots ) using clojure. I’m struggling on how to manage the sockets for multiple users. using the repl/exploratory code I’m able to connect, receive message and post message to slack, no problem at all, but I’m stuck on how to make it able to multiple users. that is my function to connect on slack websocket ( will return the ws socket )

(slack/ws-connect token
            :on-message dispatch
            :on-error #(.getMessage %1))

tdantas19:07:24

how can manage multiple sockets on my app ( creating a global map and assoc the sockets there ? )

noisesmith19:07:08

a global map with a unique ID per websocket (and the id also attached to client account, if you have any such concept) is the normal way

noisesmith19:07:13

that's what eg. sente has under the hood

tdantas19:07:58

thanks , will do that

noisesmith19:07:33

if you have minimal state to track, you can also just have handler functions that take a request and a websocket object, and call them from the listener you attach to the socket

noisesmith19:07:50

but if things get stateful at all the map is good

tdantas19:07:07

> you can also just have handler functions that take a request and a websocket object, and call them from the listener you attach to the socket can you pseudo-code that ? didn’t get

noisesmith19:07:30

(defn do-something-for-socket [connection args] (let [result (frob args)] (send-to-connection connection result))

noisesmith19:07:25

assuming you get something from the request that you can send data to by some method or function (that's usually how this is set up)

tdantas19:07:53

connection is the ws socket, right ? how the client of do-something-for-socket will grab the ws socket ?

beders22:07:09

I'm trying to create an uberjar for yada and are getting interesting error messages like: 'namespace 'cheshire.factory' not found'

beders22:07:19

I can compile the code fine

beders22:07:27

anything obvious I'm missing? (this is with leiningen)

beders22:07:31

only happens when using {:aot :all} though

qqq22:07:15

in clojure/jdbc, are all sql statements compiled down to STRINGS, sent to SQL, then reparsed ?

qqq22:07:50

I'm wondering if there is a way to send an AST or something, instead of directly sending the strings

hmaurer22:07:50

@beders total newbie here, but is your cheshire dependency in “dependencies”, or under dev dependencies?

beders22:07:15

it is brought in by yada indirectly, but also when adding it explicitly it barfs when trying to create the uberjar

beders22:07:30

(and it is in dependencies)

hmaurer22:07:11

I am operating blind here, but lookign at Yada’s project.clj it is under the “test” profile dependencies

hmaurer22:07:26

Those dependencies won’t be pulled when running uberjar

beders22:07:52

yes, looked at that too, but the dependency tree brings it in via yada/json

beders22:07:34

the weird thing is, that error is triggered when aot compiling cheshire.core

beders22:07:53

Exception in thread "main" java.lang.Exception: namespace 'cheshire.factory' not found, compiling:(cheshire/core.clj:1:1)

beders22:07:13

even weirder: the JAR file for cheshire in the requested version does have a factory.clj

hmaurer22:07:45

so I assume you ran lein deps :tree and checked that cheshire was there?

hmaurer22:07:17

and you checked that the right version was the first one in the output?

beders22:07:18

yup, it's there

hmaurer22:07:27

(not overriden by another version without factory.clj)

beders22:07:37

only happens when doing {:uberjar with {:aot :all}}

hmaurer22:07:58

weird. I don’t know how aot works so I am afraid I can’t help you further 😞

beders22:07:43

yeah, I'm feeling pretty clueless myself. But running lein deps :tree has interesting warnings. Looking at those now. Thank you!

hmaurer22:07:37

@beders make sure that the right version of cheshire appears first in the tree

hmaurer22:07:49

from what I have been told lein will pick the first version it finds in the list

beders22:07:50

there's only one

hmaurer22:07:29

@mention me if you find a solution please, I am curious 🙂

beders22:07:05

I will, thanks again

hmaurer22:07:49

@beders can you Gist me the output of lein deps :tree ?

beders22:07:08

@hmaurer I was able to get it working. I had to exclude cheshire manually and then re-add

beders22:07:12

I have no idea why that fixed things

beders22:07:23

i.e.

[cheshire "5.6.3"]
[yada "1.2.6" :exclusions [ring-swagger cheshire]]

hmaurer22:07:29

odd. Maybe some dependency cache going on

hmaurer22:07:36

no idea how those things work in clojure/java

beders22:07:51

uberjar seems a bit brittle

beders22:07:04

but it looks like I can soon deploy something to Heroku and so my Sunday is complete

beders22:07:44

thanks for your help

hmaurer22:07:53

You’re welcome! Enjoy the rest of your evening

seancorfield23:07:27

@beders AOT is the root of a lot of problems and is best avoided, if possible. As for dependency conflicts, that's pretty common the JVM world so I'm not surprised you had to add :exclusions -- Cheshire is a popular library and lots of other libraries depend on it so you can get any one of a whole bunch of different versions pulled in if you're not careful.

beders23:07:53

weird thing here: only yada really asked for it