Fork me on GitHub
#clojure
<
2021-09-14
>
hlship03:09:27

Is there a way, inside a macro, to get type information about a local reference (that is, from &env). I'm trying to write some code w/ maximum efficiency, and I need to invoke .toString():

(decompile
    (let [a 6]
      (.toString a)))

// Decompiling class: com/walmartlabs/lang/string$fn__5082
package com.walmartlabs.lang;

import clojure.lang.*;

public final class string$fn__5082 extends AFunction
{
    public static Object invokeStatic() {
        final long a = 6L;
        return Reflector.invokeNoArgInstanceMember(Numbers.num(a), "toString", false);
    }
    
    @Override
    public Object invoke() {
        return invokeStatic();
    }
}

=> nil

hlship03:09:55

Here, Clojure knows that a is a long, but to invoke .toString() it passes it Number.num() and then, for some reason, the compiler uses reflection to invoke toString().

hlship03:09:04

Ideally, I could special case a type of java.lang.Long and invoke Long.toString(long) and get maximum efficiency code.

hlship03:09:37

Hm, Maybe I can do something with clojure.lang.Compiler$LocalBinding.

Alex Miller (Clojure team)03:09:27

I don’t trust the decompiler. Did you look at the bytecode?

Alex Miller (Clojure team)03:09:36

Decompilation is inherently lossy and you should never trust that it’s telling you the real truth

hlship15:09:44

I'll check the decompiler, but I also leveraged LocalBinding/getJavaClass() which let me generate the correct code.

hlship15:09:32

FWIW:

(disassemble
    (let [a 6]
      (.toString a)))

// Decompiling class: user$fn__10124
class user$fn__10124
    Minor version: 0
    Major version: 52
    Flags: PUBLIC, FINAL, SUPER
    
    public void <init>();
        Flags: PUBLIC
        Code:
                  linenumber      2
               0: aload_0        
               1: invokespecial   clojure/lang/AFunction.<init>:()V
               4: return         
    
    public static java.lang.Object invokeStatic();
        Flags: PUBLIC, STATIC
        Code:
                  linenumber      2
               0: ldc2_w          6
               3: lstore_0        /* a */
               4: lload_0         /* a */
               5: invokestatic    clojure/lang/Numbers.num:(J)Ljava/lang/Number;
                  linenumber      3
               8: ldc             "toString"
              10: iconst_0       
              11: invokestatic    clojure/lang/Reflector.invokeNoArgInstanceMember:(Ljava/lang/Object;Ljava/lang/String;Z)Ljava/lang/Object;
              14: areturn        
    
    public java.lang.Object invoke();
        Flags: PUBLIC
        Code:
                  linenumber      2
               0: invokestatic    user$fn__10124.invokeStatic:()Ljava/lang/Object;
               3: areturn        
    
    static {};
        Flags: PUBLIC, STATIC
        Code:
                  linenumber      2
               0: return         

=> nil
Mostly, I've seen decompilers get tripped up by locals clearing. I'm liking using the decompiler to ensure I'm not generating code that uses reflection deep in my macro.

Alex Miller (Clojure team)15:09:58

what's your actual code where you're having a problem?

hlship17:09:43

It's app code that I can't share currently. I'm working on a limited, but fast macro replacement for clojure.core/format. Shaving those microseconds!

Alex Miller (Clojure team)17:09:52

well, I'm doubtful this is a good idea

Martynas Maciulevičius07:09:18

Hey guys. I want to know if there is a way to not use reflection and use with-open when you have java.lang.AutoCloseable and java.lang.Closeable. Like this:

(with-open [my-closeable (create-new-closeable)])
Are there any ways to not have this error?
Reflection warning, /my_test.clj:_:_ - reference to field close can't be resolved.
This is a var in project.clj but I want to not remove it: :global-vars {*warn-on-reflection* true}

lsenjov07:09:54

Type hint it? (with-open [^java.lang.AutoCloseable my-closeable ...]

Martynas Maciulevičius07:09:34

Great! It worked! I was expecting to add type hint but I didn't think it could work 😄 Thanks a lot

🎉 2
West10:09:53

Is there something like python’s venv for JVM versions?

Lennart Buit10:09:40

sdkman or jenv spring to mind

p-himik10:09:34

Ah funny introduction: > jEnv is a command line tool to help you forget how to set the JAVA_HOME environment variable I would rather keep that knowledge, thank you very much. :)

West10:09:22

jenv is kinda janky right now so imma check out sdkman

flowthing10:09:06

There's also Jabba (https://github.com/shyiko/jabba), which I've used with some success.

West10:09:48

@U04V4KLKC I had to remove certain jdk versions from jenv in order to get them to show up in jenv, even though I had just installed jenv. Also you don’t necessarily pick by the specific jdk distribution, you can pick by distribution(if that’s the right word for it), for example oracle-jdk16 or openjdk16, AND by the version (16, 11, 8, etc ) regardless of the distribution. That made things a bit confusing. Now maybe it’s because my shell is fish, and I’m seeing they’ve had problems implementing jenv so it works properly with fish.

West10:09:55

$ jenv versions              
  system
  1.8
  1.8.0.302
  11
  11.0
  11.0.12
  16
  16.0
  16.0.1
  16.0.2
  graalvm64-11.0.10
  graalvm64-16.0.1
  openjdk64-1.8.0.302
  openjdk64-11.0.12
* openjdk64-16.0.2 (set by /Users/main/.jenv/version)

nbardiuk10:09:24

My team mates use http://asdf-vm.com for both java and node

schmee11:09:45

I used to use jenv but it makes your shell slow as molasses, these days I just have a bunch of aliases like this in my zshrc:

java14() {
  export JAVA_HOME='/Library/Java/JavaVirtualMachines/jdk-14.0.2.jdk/Contents/Home'
}

borkdude11:09:27

I should start doing that as well, I'm using jenv but it doesn't always work well for me

p-himik11:09:32

Just out of interest - what kind of scenarios prompt frequent changes to the Java version that you use? I understand the case with Python - it doesn't have .m2 that it could use, so you kinda have to have such a mechanism.

borkdude11:09:11

I want to ensure I'm using java8 when I'm compiling sources

borkdude11:09:34

I know you can set properties for this, but even then it's not guaranteed that some things aren't being pulled in that don't work on 8

borkdude11:09:43

also I switch graalvm versions sometimes

p-himik11:09:39

You work on a lot of tools, so you're somewhat of an outlier. :) But what about "regular" app development? Just making sure I don't have a blind spot here.

hansbugge12:09:27

At my work I regularly switch between projects with different java version requirements. Many are still on java 8 because of dependencies, while others require java >=11 (e.g. because of java.net.http). I use jenv with .java-version files in the different projects, and that works well for me.

👍 2
Karol Wójcik14:09:58

How can I check if x is type of .$JarURLInputStream in Clojure?

dpsutton14:09:06

(doc instance?)

Karol Wójcik14:09:39

Eh yea. Actually my issue was different. I forgotten how to import inner class 😄

dpsutton14:09:23

ah yeah. that gets confusing.

Karol Wójcik14:09:04

Oh wait.

Exception in thread "main" java.lang.IllegalAccessError: failed to access class sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream from class com.vethelp.api_lambda.core$AWSApiGatewayProxy (sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream is in module java.base of loader 'bootstrap'; com.vethe

Karol Wójcik14:09:15

This is the first time I'm seeing something like this 😮

Karol Wójcik14:09:28

(instance? JarURLConnection$JarURLInputStream x)

borkdude14:09:57

this is because this class is an implementation class, not a public class

borkdude14:09:07

you should not be using this class directly

Karol Wójcik14:09:15

This is what I thought.

Karol Wójcik14:09:29

However I need to check for the type of this class.

borkdude14:09:37

probably look at (supers (class obj))

borkdude14:09:42

and then pick a public interface

Karol Wójcik14:09:10

I need to differentiate the JarURLInputStream and regular InputStream

emccue14:09:27

looking at the docs for java 7, there are only 2 classes in its hierarchy

emccue14:09:26

you could check for FilterInputStream if you need that in field, but other than that its identical to an InputStream & FilterInputStream & Closeable

Karol Wójcik15:09:15

Actually my bad. I don't need to differentiate between those two facepalm

Karol Wójcik15:09:24

Sorry guys! And thanks for all your help!

zhuxun215:09:31

Sometimes when working with lots of map forwarding (esp. in ReactJS), I wish clojure had an object construction short-hand like the ES2015 notation { foo, bar } === { foo: foo, bar: bar } https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Object_initializer#new_notations_in_ecmascript_2015

zhuxun215:09:27

Is there any similar tricks in clojure that let me avoid typing the same name twice? It really helps when doing refactoring

zhuxun215:09:31

I can probably write myself a macro, but it will probably make my code confusing to anyone else. I was wondering in case there's any idiomatic way

p-himik15:09:23

What do you mean by "map forwarding"? And no, you can't have {a b c} be turned into {:a a, :b b, :c c} because map literals require an even number of members even at the read time.

zhuxun215:09:41

@U2FRKM4TW Basically, take a large map with many fields, add a few fields, delete a few fields, change a few fields, and send the result to another function

zhuxun215:09:22

Ok, so I guess the answer is no. Fair enough.

p-himik15:09:29

All that is trivially solved by core functions that work with maps. One extra thing that makes it easier is avoiding assigning long names in small scopes.

p-himik15:09:47

+ threading macros, namely -> and cond-> in this case.

👍 2
zhuxun215:09:51

Thanks, guys. These suggestions are all currently what I use daily. Just had to wonder whether I missed anything. Guess not.

Ed15:09:20

(-> m
      (dissoc :remove :some-keys)
      (rename-keys {:rename :key})
      (assoc :add-key value))
yeah ... find that much more readable than the es2015 notation

👍 2
dpsutton16:09:16

what would the equivalent es2015 notation look like?

zhuxun216:09:53

@U11BV7MTK I think the tweeted solution @U0CL38MU1 linked is very close to what I imagined, except that it is customized and will appear confusing to other people reading my code.

dpsutton16:09:06

(defmacro nmap [& xs] 
    (cons 'hash-map 
              (interleave (map (comp keyword name) xs) xs)))

(let [a 1, b 2, c 3] 
  (nmap a b c))

{:c 3, :b 2, :a 1}
i'd go with this one

👍 2
dpsutton16:09:45

from that clojureverse thread. maybe extend it to be aware of vector pairs to allow normal key value pairs.

lilactown16:09:14

I would just repeat the key and the symbol. it's not so bad IMO and if you're working on a team it doesn't seem worth it to add the macro

6
emccue16:09:06

I had one that was (m a b c | :d 1 :e 3)

👍 2
zhuxun216:09:24

I tend to agree with @U4YGF4NGM and the farthest I would go is probably @U11BV7MTK's suggestion of defining a named macro, in which case at least it's obvious I'm using a macro and their IDE could lead them to the definition.

dpsutton16:09:03

yeah. and really take the temperature of the team. if everyone is on board, go for it. if not, don't mix and match

👍 4
zhuxun216:09:11

In that clojureverse thread, @U04V70XH6 brought up a good point that maybe that ES2015 feature has it's own downside by being a "visual overload" of the braces, not necessarily a net plus for mental mapping

🔥 2
Joshua Suskalo15:09:05

For destructuring you can use {:keys [foo bar]}, and for checking equality on components you can use rename-keys and select-keys. There is no shorthand for object construction though, and you'd need your own macro for it, and even then you couldn't use the {} syntax because it requires an even number of forms.

Max18:09:15

Is there any reasonable way to co-mingle tests/code/resources? e.g. instead of the standard project structure:

src/
  a/
    b.clj
  c/
    d.clj
test/
  a/
    b_test.clj
  c/
    d_test.clj
resources/
  foo.txt
  bar.csv
Is it possible to do something more like this?
a/
  b.clj
  b_test.clj
  resources/
    foo.txt
c/
  d.clj
  test/
    d_test.clj
  resources/
    bar.csv
My instinct is that’d require wildcard patterns in path specifications, and I didn’t see anything about those being supported

p-himik18:09:51

It is possible. For tests - by having the test runner to scan files by a glob/RegEx, so depends on your test runner I suppose. For resources - by listing the directories out explicitly, assuming you want for them to be on the classpath as e.g. foo.txt and not a/resources/foo.txt.

p-himik18:09:39

But not sure if anybody actually does that, and FWIW I would certainly not like to find such a structure in a library I need to read or, let alone, support.

p-himik18:09:15

If a and c are isolated that much, then perhaps it makes sense to split them into different projects?

lukasz18:09:30

this ☝️ bundling test code, also means pulling in test dependencies (if there are any) or removing test code before building the jar - so yeah, a bit... unorthodox approach

p-himik18:09:18

Not necessarily - you can bundle the code, but if it's not required anywhere, it won't be evaluated so you won't need the dependencies. But perhaps it depends on the bundling approach - I don't really have experience there.

Max18:09:21

Cool, so sounds like it’s in the realm of “technically possible, but impractical in practice”

lukasz18:09:22

Yes, you'd be going against how most projects are laid out right now

lukasz18:09:50

@U2FRKM4TW true, it would kinda work for libraries, but not for applications

lukasz18:09:32

actually, scratch that - I missed the "if you don't require" part

lukasz18:09:58

so yes, doable, but impractical IMHO

👍 2
mkvlr19:09:38

happy to see https://github.com/clojure/clojure/commit/f96eef2892eaef55dbbdbf7f3b4b818be06f6ddd, thanks @alexmiller and everyone else involved in making it happen! 🙏👏

💯 8
4
🔑 4
Alex Miller (Clojure team)19:09:14

if I did my job, that should be a better explanation

Alex Miller (Clojure team)19:09:32

so if you are using spec you might have a spec :my.cool.domain/account where "my.cool.domain" is not a namespace you can load, it's just a useful qualifier for specs

Joshua Suskalo19:09:47

awesome, I'm glad you took into account the case where the namespace is loaded later and has real content.

Alex Miller (Clojure team)19:09:01

before you could just use the full kw but lots of typing

Alex Miller (Clojure team)19:09:41

you can't do this:

(ns foo
  (:require [my.cool.domain :as d]))   ;; fails - can't load my.cool.domain

Alex Miller (Clojure team)19:09:23

you can do this:

(create-ns 'my.cool.domain)
(alias 'd 'my.cool.domain)
but, it's annoying and invisible to many analyzers as it's outside ns

Alex Miller (Clojure team)19:09:55

now you can do:

(ns foo
  (:require [my.cool.domain :as-alias d]))   ;; works! (doesn't load)

::d/account ;; valid

👍 2
Alex Miller (Clojure team)19:09:55

so :as-alias is like :as (creates an alias) but doesn't force a load of the namespace

seancorfield20:09:18

I'm looking forward to replacing this code at work as soon as the new Alpha drops:

(alias 'm  (create-ns 'ws.domain.member))
(alias 'mu (create-ns 'ws.domain.member.update))
(alias 'mv (create-ns 'ws.domain.member.validation))

vncz20:09:42

This specific use case is only to have aliases instead of writing full names every time, right?

seancorfield20:09:00

Yes, the above allows me to write ::m/whatever instead of :ws.domain.member/whatever

seancorfield20:09:02

Pretty sure I've got some code that I can replace with update-vals too.

jjttjj20:09:21

I'll have to retrain my muscle memory to use update-vals instead of medley.core/map-vals (thrilled to do it though!)

djblue20:09:02

Would :as-alias also work for lazy loading deps too?

Joshua Suskalo20:09:00

in theory if the namespace has been loaded, you could access vars behind it, but I would think there'd be some potential issues with AOT compilation.

Joshua Suskalo20:09:31

specifically with direct-linking turned on

seancorfield20:09:29

@djblue Not sure what you're asking there... :as-alias is a way to create aliases for namespaces without needing to have code behind the actual namespace name.

seancorfield20:09:35

It's useful for keyword resolution, like ::m/whatever based on the alias calls above, but you couldn't access any code (unless you had a "normal" require for that namespace).

djblue20:09:13

Yeah, sometimes I want to wait to load a large dep until I need it. But I'm assuming I would still need to use requiring-resolve . So I could do (requiring-resolve m/whatever)`

Alex Miller (Clojure team)20:09:34

nothing here changes that

Alex Miller (Clojure team)20:09:42

if you want to use vars, you have to load

Alex Miller (Clojure team)20:09:30

techniques exist for dynamic loading and this does not give you anything new in that regard

djblue20:09:51

But now I can use m/whatever` instead of the.full.m/whatever` when passing a symbol to requiring-resolve

Joshua Suskalo20:09:17

seems odd that you'd need to write that in code

Joshua Suskalo20:09:31

isn't requiring-resolve usually just used for user input for command line utilities and similar?

Joshua Suskalo20:09:48

cool that you found a usecase, but it seems odd

Alex Miller (Clojure team)20:09:54

I think I'd rather just type it out to make it easier to find :)

djblue21:09:45

True, but the same can be said for keywords

seancorfield21:09:06

Just replaced a pair of nested reduce-kv calls with update-keys and update-vals 🙂

vemv22:09:24

Checking out the commit, can't :as-alias result in namespaces being misteriously empty? e.g. ns x requires y under :as-alias and ns z requires y normally, and also requiring x normally i.e. there's the assumption (but not enforcement) that namespace y isn't backed by an actual file? If I put actual contents under it, those can get hidden by an unlucky loading order

potetm22:09:49

That path doesn’t appear to update *loaded-libs*

👍 2
potetm22:09:45

which is the cache which normally prevents a namespace from being loaded.

seancorfield23:09:16

It's easy enough to try it out, to convince yourself it works "as expected".

seancorfield23:09:21

If you look at the commit, I think it specifically has a test for a ns that is required :as-alias followed by a later (regular) require of that same ns, to verify the code is loaded at that point.

👍 2
vemv23:09:16

ty both!

Alex Miller (Clojure team)01:09:12

If you load the namespace it will define the vars in the existing runtime namespace

👍 2
Alex Miller (Clojure team)01:09:48

So the rules haven’t changed - if you want to use vars from the namespace, load it

vemv09:09:30

Thanks again. I was thinking, I'm pretty used to use find-ns to determine whether a ns has been loaded: https://github.com/jonase/eastwood/blob/8a6655f592004a7d6e028380144cb1210bb7a9f5/src/eastwood/util.clj#L1144 https://github.com/clojure-emacs/refactor-nrepl/blob/1b646079ef3e3c4438c3eeb20514e4d2731f444e/src/refactor_nrepl/core.clj#L338 Looks like I should change this code to use *loaded-libs* instead (edit: the linked code in particular wouldn't necessarily be wrong today... I simply used it to reflect how I'm using this pattern in the wild)

West23:09:08

I haven’t delved into the polymorphic functionality in Clojure. For the most part, I have only built applications using pure functions. When is it helpful to use things like defrecord, defprotocol, deftype, defmulti, defmethod, reify, make-hierarchy, derive, etc.?

West00:09:25

Ah thank you.

potetm00:09:28

Here be opinions, but: They’re for open dynamic dispatch (i.e. you have a bunch of different functions, and you need to figure out which one to call.) They’re great for libraries. They should be used often in libraries. End-user applications (i.e. what most people write) don’t actually need them very often. In most cases, you’d be fine using a lookup map (e.g. {:foo (fn …), :bar (fn …)})

potetm00:09:25

Reason being: in end-user applications, you have total control over all the code. You don’t need open dispatch. Closed is fine.

potetm00:09:49

(Unless you have a large team or multiple teams. Then you might need open dispatch.)

potetm00:09:24

(But that’s basically the same rationale as using open dispatch for a library.)

West00:09:35

I’m actually writing a blog post about this. Would you like me to give a shoutout to you guys? @U2FRKM4TW @U07S8JGF7

seancorfield00:09:27

Probably worth mentioning that the plumatic article predates protocols being extended via metadata -- so the need for defrecord is (even) less now since plain ol' Clojure data with metadata can satisfy a protocol.

💯 2
West00:09:17

@U04V70XH6 Thank you for mentioning that. I forgot metadata was a thing.

potetm00:09:19

That’d be awesome @U01KQ9EGU79. Thanks for asking.

West01:09:00

Where does this concept of open dispatch/closed dispatch come from? Are there more resources that you can point me to? @U07S8JGF7

potetm01:09:57

open = you can add dispatch entries without altering existing code closed = you must alter existing code to add dispatch entries

potetm01:09:23

I dunno what resources exist. Rich talks about it in Simple Made Easy.

potetm01:09:08

To be concrete: defprotocol and defmulti both define dynamic dispatch functions. You add to their lookup tables via extend-protocol, extend-type, defrecord, and defmethod. You can call those macros in any namespace and add to their lookup tables at any time.

potetm01:09:05

If you use a map like I suggested (`{:foo (fn …), :bar (fn …)}`), you have to go to that specific map in the specific namespace, add a new entry, and reload that code.

potetm01:09:28

@U01KQ9EGU79 Does that make more sense?

West02:09:08

I still don’t really get it.

West02:09:43

Like I don’t see why this isn’t just a layer of indirection to a function.

potetm02:09:22

Of course it’s “just a layer of indirection to a function!”

potetm02:09:53

But that statement is so broad it’s almost meaningless 🙂

potetm02:09:30

Method calls on objects are just indirection to functions. Vars are layers of indirection to functions.

West02:09:50

I guess I’ve never had to make a map of functions before, or map types to functions. I’ve never used this pattern, {:foo (fn …), :bar (fn …)}.

potetm03:09:55

I can almost guarantee that you’ve done it, but you just didn’t recognize it :)

potetm03:09:29

How familiar are you with polymorphism from OOP? (e.g. Java interfaces, JavaScript prototypes, etc)

West03:09:30

I do not come from an OOP background at all. Clojure was my first real language and before that I played with python and couldn’t wrap my head around objects then.

West03:09:26

My understanding stopped at “Ok, methods are just functions, and objects are just variables with variables and functions inside.” I had no idea why it would be useful to use objects rather than just using functions and variables.

West03:09:00

(def ^:private table-columns
  (into []
        (for [each schema/table-schema]
          {:fx/type            :table-column
           :text               (:column-name each)
           :cell-value-factory identity
           :cell-factory       {:fx/cell-type :table-cell
                                :describe     (fn [x]
                                                {:text ((:file-data-key each) x)})}})))

(def table-schema
  ;; This vector of maps generates the table.
  ;; `:name` is the string that will show at the top of the column.
  ;; `:key` is the keyword that will be used in `file-data` to pair with its value.
  ;; `:fn` is the function that will execute on each value within each kv-pair in `file-data`.
  [{:column-name     "Full Path"
    :file-data-key   :full-path
    :data-display-fn helpers/full-path}
   {:column-name     "Movie Name"
    :file-data-key   :display-name
    :data-display-fn display-name}
   {:column-name     "File Type"
    :file-data-key   :file-ext
    :data-display-fn helpers/extension}
...
])
Now that you mention it, I actually have created a map with functions. This was my solution for a UI based on cljfx. Could protocols and records simplify this somewhat?

potetm03:09:10

Yeah, this is exactly the thing. You’ve built a dispatch table with different implementations of (i.e. different ways of doing) :data-display-fn.

West03:09:58

Ok, I’m pretty close to grokking this. How might I go about refactoring this?

potetm03:09:21

I personally think that what you have is fine (though I can’t really tell everything you’re using it for).

West03:09:17

Hmmmm. Ok, so far I’m only using this lookup table for one thing, a table within a window.

potetm03:09:05

Where is :data-display-fn called?

potetm03:09:07

I gotta get to bed for now. You can almost certainly get this done with mulitmethods as an exercise, but I doubt the code would be any clearer or better as a result.

potetm03:09:22

But it’s probably worth the learning experience.

West03:09:23

(defn map-file-data [dirs]
  (mapv
    (fn [dir]
      (into {}
            (for [schema-entry schema/table-schema]
              [(:file-data-key schema-entry) ((:data-display-fn schema-entry) dir)])))
    (mapcat file-seq dirs)))

potetm03:09:51

Right, you could def do this with multimethods. Use :file-data-key as a dispatch fn and provide different implementations for :full-path :display-name and :file-ext.

potetm03:09:55

I’ll try to check back tomorrow and see if you have any questions. Feel free to add on here or DM me in the meantime.

mpenet08:09:11

derivation (hierarchies) are great, think having a way to express relations between things without baking the definition of these relations to the values they link. It can be very handy

mpenet08:09:38

hierarchies are first class in clojure, there's a global one yes, but you can also create your own and pass them around

p-himik09:09:42

@U01KQ9EGU79 It would make more sense to shout-out Plumatic for that article and Sean Corfield for that addendum. :)