Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2018-08-13

Channels

#clojure
<
2018-08-13
>
aengelberg01:08:34

If I define a function like so: 

(defn foo ^double [^double x ^double y] (+ x y))
What combination of type hints / primitive casts / feng shui do I need on my calls to foo such that the compiled bytecode will call foo's .invokePrim method and use the unboxed value coming back?

aengelberg01:08:31

I've been experimenting with clj-java-decompiler: 

user> (decompile (foo 1.0 2.0))

// Decompiling class: cjd__init
...
        ((IFn)const__0.getRawRoot()).invoke(const__1, const__2);
...

user> (decompile (+ ^double (foo (double 1.0) (double 2.0)) 3.0))
...
        Numbers.add(RT.doubleCast(((IFn)const__1.getRawRoot()).invoke(RT.doubleCast(1.0), RT.doubleCast(2.0))), 3.0);
...
but it's possible that the decompiler is incorrectly inferring the compiled logic in practice.

schmee08:08:37

absolutely invaluable when trying to do efficient numerics in clojure

martinklepsch08:08:36

Does anyone know of a tool like https://github.com/benedekfazekas/mranderson that instead operates on plain Jar files instead of some higher level abstraction like Leiningen's project.clj? Or could point me in the right direction for doing that with mranderson? Thanks

dominicm09:08:54

I looked into this, couldn't find anything. Let me know if you find it 🙂

dominicm09:08:13

There was something that seemed to wrap a maven plugin, but I struggled to understand quite what it was doing, and it was a lein plugin too, but less coupled.

valerauko09:08:26

iirc the new clojure dep does something like this

dominicm09:08:57

I don't think it does anything like mr anderson. What do you mean by that?

valerauko09:08:17

i recall a podcast where they talked about dep including jars as source but i might be remembering wrong

Alex Miller (Clojure team)13:08:21

you may be referring to the add-lib stuff (not yet merged)

Alex Miller (Clojure team)13:08:06

but I think it would helpful to state what you want as I’m not sure

dominicm14:08:04

That is very different to what Mr. Anderson provides though 🙂

Alex Miller (Clojure team)14:08:49

yeah, that’s why I think it would be helpful to actually start from the problem

martinklepsch15:08:09

I want to take jars (clojure/java), take their source files and move all namespaces/packages according to some predictable scheme (e.g. adding a prefix)

martinklepsch15:08:03

The problem I'm trying to solve by doing this is loading individual dependencies in potentially multiple versions. Specifically the cljdoc analyzer environment where I already have some dependencies that I currently cannot analyze in a version different than the one I have loaded.

martinklepsch15:08:31

Hope that makes sense. It's also not a huge issue for cljdoc yet, I was just looking around what might be out there.

dominicm15:08:15

dependencies of your analyzer are colliding with the dependencies of the projects you're analyzing?

martinklepsch15:08:31

not with their dependencies but dependencies of the analyzer might also be projects to be analyzed. Eg. I need org.clojure/tools.namespace but I won't be able to build docs for it in a different version than the one I have currently loaded.

dominicm15:08:07

and tools.analyzer is run against the lib?

martinklepsch15:08:37

@U09LZR36F not sure I understand the question? the analyzer is a slightly more complex mixture than tools.analyzer but yes, it might be that cljdoc wants to build docs for a new (or old) release of tools.analyzer. Currently it can't because it already uses a specific version of tools analyzer.

dominicm15:08:14

I was thinking that if tools.analyzer let you override how to find source files, then that would solve the problem

martinklepsch15:08:21

I could just say "use the version that should get analyzed" but that might break things, especially when trying to build for an old version

dominicm15:08:54

it doesn't, from a glance at the api

martinklepsch15:08:39

The cljdoc analyzer takes a directory of source files as input, is that what you mean?

dominicm15:08:44

@U050TNB9F those files are on the classpath though? why?

martinklepsch15:08:43

if they were not, requiring transitive namespace dependencies would fail, no?

dominicm15:08:36

So, if you could control how require worked in tools.analyzer, you could use an isolated classloader.

martinklepsch15:08:46

yeah, maybe. sounds like a lot of work 😅

dominicm15:08:48

Less than you'd expect ime.

dominicm15:08:00

I've done the classloader thing a few times. It's even done in cider to support boot.

Alex Miller (Clojure team)16:08:00

in java land, there are tools like Jar Jar Links (oof, I know) or Maven Shade to do stuff like this, but I’m not familiar with anything in Clojure

Alex Miller (Clojure team)16:08:13

in autodoc, the project is split into the collector (which has no deps and merely analyzes and builds a data file), and a renderer project that takes the data file. That helps minimize the deps problem.

Alex Miller (Clojure team)16:08:53

not sure how wide the issue is, but seems like you could maybe do this manually if the number of affected libs is small enough

martinklepsch16:08:22

Cljdoc ended up with the same approach autodoc seems to be taking.

martinklepsch17:08:32

But still, some deps (Clojure, ClojureScript, tns, Java cp, ...) for which it will be trickier to analyze all versions of. I’ll think of something :)

dominicm19:08:14

@U064X3EF3 mr anderson is a wrapper of jar jar links, the api is horrible

andy.fingerhut19:08:04

I have not used mranderson, but have done something intended to solve the same problem for Eastwood, called 'dolly', but it requires manually fixing up some changed require statements by hand in some cases: https://github.com/jafingerhut/dolly I hesitate to even mention it because it is a barely-working piece of code that you have to treat it just right or it useless.

andy.fingerhut19:08:41

It doesn't do much more than copy a namespace's source files from one project into another, attempting to rename all occurrences of one or more other namespaces as it goes, in Eastwood's case by prepending something like 'eastwood.copieddeps.proj<n>.'

Andrea Imparato10:08:59

hallo! anybody has a recommendation for a library to use for a slack bot?

bronsa13:08:38

@aengelberg the decompiler is correct, you're just forcing boxing because decompil is wrapping the entire expression into a non-primitive returning function

bronsa13:08:44

try decompiling e.g. (Long/toString (let ..)) -- you'll see everything being unboxed until the call to Long.toString(long)

fantomofdoom14:08:01

Hi, smb use manifold (https://github.com/ztellman/manifold), i can't find how create something like core.async go-loop only for manifold stream? (and smb known if use manifold.deffered/loop & recur, thread are parked or it alway take message from source)

aengelberg16:08:11

@bronsa hmm I just tried that, doesn't seem to be working.

bronsa16:08:55

oh sorry I didn't read your actual code

aengelberg16:08:57

that's sort of what I was going for with the (+ ^double (foo (double 1.0) (double 2.0)) 3.0) example above

bronsa16:08:06

the problem is that clojure can only invokePrim on primitive functions bound to Vars

aengelberg16:08:39

ok, I just combined the two tips (make sure I'm calling a defned function, and wrap it in TheType/toString) and it seems to be working now: 

user> (decompile (Double/toString
                   (let [d (double 1.0)]
                     (foo d d))))

// Decompiling class: user$fn__17583
import clojure.lang.*;

public final class user$fn__17583 extends AFunction
{
    public static final Var const__2;
    
    public static Object invokeStatic() {
        final double d = 1.0;
        return ((DDD)const__2.getRawRoot()).invokePrim(d, d);
    }
    
    public Object invoke() {
        return invokeStatic();
    }
    
    static {
        const__2 = RT.var("user", "foo");
    }
}
nil

aengelberg16:08:58

interestingly I had to wrap it in a let to get it to work; (Double/toString (foo 1.0 2.0)) didn't work

bronsa16:08:13

with 

(defn foo ^long [^long a ^long b]
  a)

(defn bar []
  (Long/toString (foo 1 2)))
I'm definitely getting clojure/lang/IFn$LLL.invokePrim:(JJ)J

bronsa16:08:42

same with doubles

bronsa16:08:32

I don't see why the let would make any difference

bronsa16:08:42

and the double call should be useless too

bronsa16:08:08

are you sure you weren't doing a long -> double cast by mistake? (that's what I just did)

skrat16:08:45

Can I query datascript/datomic for entities that DO NOT have certain attribute?

dominicm19:08:36

Datascript doesn't have not

arrdem19:08:55

negation requires the closed world assumption and is not part of traditional datalog