Fork me on GitHub
#clojure
<
2018-03-02
>
rgorrepati00:03:44

Hi, (type (int 10)) returns java.lang.Integer. How do i create a primitive int in clojure?

schmee00:03:11

int will actually create a primitive int, but those can only exist in local scope, ie let

schmee00:03:42

when you call int in the REPL, the value will be boxed before it is returned and printed

schmee00:03:28

methods can be tagged to return primitives, but only longs and doubles

rymndhng01:03:46

let's say I parsed some json, i.e. {"First Name": "bar"} and I keywordized the keys print like: {:First Name "bar"}. Would be a safe way of printing this to EDN such that it could be read safely?

zentrope01:03:20

Although it’s possible to create keywords with spaces, they’re not something you can use as literals.

hiredman01:03:57

have you considered not keywordizing?

rymndhng03:03:19

it's coming from user input

hiredman01:03:09

{"First Name" "bar"}

mfikes01:03:05

FWIW, ClojureScript itself does what @hiredman is suggesting when it needs to print a JavaScript object like that

#js {"First Name" "bar"}
specifically so you can copy and paste it in the REPL, etc., and it can be read.

mfikes01:03:29

This is as compared to

#js {:name "bar"}

zentrope01:03:46

I guess you can create keywords with spaces in them, just not use them as literals.

mfikes03:03:19

You can use keyword to make objects that are not legal keywords.

Alex Miller (Clojure team)03:03:41

they are legal keywords

Alex Miller (Clojure team)03:03:50

they are not print-read round-trippable

Alex Miller (Clojure team)03:03:25

they are not valid literal keywords

rymndhng03:03:14

aye, my use-case is it's coming from user input, and i wanted to try round-trippin' the data using EDN instead of JSON

zentrope03:03:43

You can do that, just don’t keywordize the keys.

rymndhng04:03:15

definitely can, was more so wondering if folks know something that i might've missed

rymndhng04:03:29

so let's say i'm leaning towards a serialization format that gives me all of clojure's flexibility, would that be to lean towards transit?

zentrope04:03:29

I do mostly prototypes, so for me, just edn itself is fine.

zentrope04:03:22

On the Clojure side, something like: (clojure.edn/read-string (slurp (:body request)) or variants, depending on your web server.

zentrope04:03:19

When sending in ClojureScript: #js { “method” “POST”, “body” (pr-str data)}.

qqq07:03:59

I know about using reader macros in cljc:

(try
 (dangerous!)
 (catch #?(:clj Exception
           :cljs :default)
        e
   (solve-it)))
Now, I need to define a macro which expands to a try/catch. How do I get this macro to expand to Exception in clj and :default in cljs ?

benzap08:03:30

is there a way to tell clojure not to concern itself with whether or not a pulled in function is private from a separate namespace?

benzap08:03:06

It errors out over a method that was set private, however, it doesn't error out when I pull it into clojurescript

seancorfield08:03:03

@benzap you can use the #' prefix on the name to override the private access.

danielstockton09:03:40

I'm looking to build a very high performance web server. nginx-clojure looks like a good choice. Am I right?

gklijs09:03:03

@danielstockton I had good experience using it, it still in progress, so no benchmarks, but it's nice to have both your static resources and some handler in the save instance. I use this to be able to upload images, which then gets used in the html, and served by nginx. Combined with cashing, pages should load very close to just static files.

danielstockton09:03:03

@gklijs In my case, i'm not planning on serving any static files. It's an API service which will receive a lot of traffic.

gklijs09:03:13

Ok, nginx uses multiple JVM do you probably want to use redis to keep start and/or only call other services in the handler, both should be fine.

mpenet09:03:55

@danielstockton just wondering, what are your constraints/numbers?

danielstockton09:03:03

I'd like to use Clojure, that's the only constraint really. It will have to scale horizontally, so i'm not trying to get a certain number of requests from a single instance. I just want to get the most bang for my buck and minimize response times.

danielstockton09:03:28

@gklijs Sorry, what do you mean by 'use redis to keep start'?

gklijs09:03:47

Meant state, not start. But really depends on what kind of API it is. The second call to the API might gets handled by a different JVM instance then the first one, that's all.

danielstockton10:03:26

Oh right, ok. Yes, I will be making calls to other data stores within the handler (most likely redis is one of them).

danielstockton10:03:49

I won't assume that the Clojure process has access to state shared between requests, that's for sure.

gnejs11:03:35

@danielstockton any reason that Undertow (through Immutant) couldn’t provide enough bang for the bucks?

gnejs11:03:00

heh - the same one I’m currently have open in my browser 😉

mpenet11:03:16

anybody is using jsonista in prod yet?

mpenet11:03:29

looks good on paper

qqq14:03:21

is it same to assume that jvm exceptions are highly optimized so that if there is no exception thrown, the cost of negilible ?

dpsutton14:03:53

i think exceptions are just longjumps and are "cheap". But I'm not a jvm guy just what i've heard

qqq14:03:42

ah, so in theory, we can imagine "global pointer handler; // points to the catch block to call on exception" then, the cost of a "try" is: 1. save old value of handler 2. set handler to new catch block ?

qqq14:03:03

That sounds both computationally inexpensive and sufficient for implementing try/catch blocks.

firthh14:03:40

A quick google suggests that try/catch is relatively quick, it’s the throwing of the exception that’s expensive

schmee14:03:24

AFAIK what’s expensive is to construct the stack trace, so to get maximum performance you should throw a pre-allocated exception, in that case it’s basically just a jump instruction

schmee14:03:29

this blog post is excellent and tells you everything you need to know 🙂 https://shipilev.net/blog/2014/exceptional-performance/

qqq14:03:37

@schmee: a quick search of the blog post seems ot suggest it doesn't covedr this 'pre allocation' trick; can you point me at where that trick is explained ?

schmee14:03:35

he calls them “static” exceptions, if you really want to know what’s going on just read the whole thing

Alex Miller (Clojure team)14:03:58

I’ve used that trick in Java :)

schmee14:03:24

@alexmiller since you’re around I’ll ask a completely different question I’ve been thinking about: how long does Clojure intend to support Java 1.6? has there been any discussion around moving the oldest version up to, say, Java 8?

Alex Miller (Clojure team)15:03:07

I would like to move to Java 8 as the min bar. We haven’t had a discussion about it for Clojure 1.10 yet

schmee15:03:26

roger, thanks for the info

dominicm15:03:53

Wouldn't changing be a form of breaking backwards compatibility?

Alex Miller (Clojure team)15:03:03

it sounds like you’re trying to begin a semantic argument that I’m totally uninterested in :)

Alex Miller (Clojure team)15:03:41

realistically, very few people are using 1.6 (or even 1.7 now) and it’s getting difficult to even support in our build infrastructure

schmee15:03:36

also, invokedynamic, which will totally solve all our problems and make clojure 1000x faster with almost zero effort

schmee15:03:20

jokes aside, I’m really excited about that stuff and I’m looking at ways to get it into the Clojure compiler, hence my question about the version 🙂

dominicm15:03:02

@alexmiller I am genuinely confused by the the notion that java version compatibility isn't considered part of the API of Clojure. I accept the difficulties of course, I personally would love to see the minimum bar raised for my own selfish needs, because seeing better java 8 time integration would be awesome. I sometimes hear something which questions my understand of something Rich has said, so this is really an attempt to gain insight into that.

Alex Miller (Clojure team)15:03:48

it is of course considered

Alex Miller (Clojure team)15:03:30

generally upgrading java is an additive experience - you get access to new stuff and existing stuff continues to work, same as Clojure

Alex Miller (Clojure team)15:03:45

so the upgrade path is still maintaining backwards compatibility

dominicm15:03:20

> Upgrade path There's a nuance here about the "API" of upgrading Clojure here that's interesting, and completely pivots your ability to shift the requirement. This was useful, thank you 🙂

mikerod15:03:45

My understanding of exceptions was that it was supposed to be fairly cheap to take the normal flow brancch, but not the exception branch. Also, I believe that try-catch blocks may have some affect on JVM optimizations

mikerod15:03:33

I forget now, maybe it is in that post that I need to read above. I was thinking it may have some affect on certain memory barriers and/or inlining

mikerod15:03:59

Also, If I remember correctly, I thought that Clojure seemed to generate heavier byte-code for try-catch than from what I saw in Java… Maybe I’m crazy though. Will have to check again

mikerod15:03:26

(just wondering if this triggers anyone’s memory on the topic)

Alex Miller (Clojure team)15:03:38

try/catch is not actually bytecode per se - it’s really a handler table attached to the bytecode

Alex Miller (Clojure team)15:03:52

Clojure’s output should not really be any different Java’s afaik

Alex Miller (Clojure team)15:03:56

there are effects on inlining, but generally I find there is far more mythology about this stuff than reality and it is constantly improving

Alex Miller (Clojure team)15:03:45

Shipilev is exactly the guy to tell you the real answer and not the mythology

Alex Miller (Clojure team)15:03:05

so I’d believe him :)

mikerod15:03:10

@alexmiller I’m reading through that post above, so far already good. So will continue. I don’t tend to get worked up about the effect it has on perf either. I just have wondered before, especially in Java code, when I saw try-catch control flow being used for whatever reason and it happened to be potentially a method called in a hot path. Something like loading classes having to use ClassNotFoundException errors for flow. I want to relook at my experience with what Clojure generated. I thought it had more bytecode operations involved than I expected and saw for Java

mikerod15:03:46

I wasn’t implying that the try/catch was just a bytecode or something like that. I just thought the branching that clj output costs more time

Alex Miller (Clojure team)15:03:57

one possible difference is that because we are still using java 5 compatible bytecode, I don’t think we’re using the “new” exception style handling they moved to in java 6 bytecode (I think those are the right numbers, haven’t looked at that stuff in a while), but that’s pretty deep in the technical weeds. we go through asm and generate standard bytecode in the same way as all the other jvm languages in this regard.

borkdude15:03:47

hmm, I get a null pointer exception on (pos? ...) in this code:

processed (reduce + (map second (get grouped :processed)))
processed (when (pos? processed)
                            (str "/ Processed: " processed))
When can (reduce + ...) return nil?

sundarj15:03:37

user=> (reduce + nil)
0
user=> (reduce + nil nil)
nil
user=> (reduce + [nil nil])
NullPointerException   clojure.lang.Numbers.ops (Numbers.java:1018)
user=> (reduce + [nil])
nil

sundarj15:03:25

should probably be using the init arity of reduce

sundarj15:03:52

user=> (reduce + 0 nil)
0
user=> (reduce + 0 [nil nil])
NullPointerException   clojure.lang.Numbers.ops (Numbers.java:1018)
user=> (reduce + 0 [nil])
NullPointerException   clojure.lang.Numbers.ops (Numbers.java:1018)

borkdude15:03:26

This particular expression returns nil: (reduce + (map ...))

borkdude15:03:02

I don’t miss a type system often, but this is one of them

carocad15:03:40

(reduce + (map identity [nil]))

carocad15:03:03

the collections that map operates on has a single nil inside

borkdude15:03:53

thanks, makes sense now

mikerod15:03:51

I’m curious now. This is something I looked at pretty long back. I can check it out and see if I can rediscover whatever I thought I did before

Qambar16:03:46

never mind

polymeris17:03:48

Testing a function that might get stuck in an infinite loop, so wrapping it in a future:

(defn happy-test-case [[input expected]]
  (is
    (let [f (future (function-under-test input))
          result (deref f 1000 :timeout)]
      (future-cancel f)
      (= expected result))))
After running the tests I am left with high CPU usage, so I suspected some futures where left orphaned, and added future-cancel. However it does not fix the issue. Ideas?

noisesmith17:03:44

@polymeris future-cancel only interrupts certain specific methods

noisesmith17:03:54

if those methods never get called, the cancellation does nothing

polymeris17:03:31

is there a future-kill or something similar?

noisesmith17:03:50

nothing that is sane to use without shutting down the entire vm

noisesmith17:03:14

the smart thing to do is write the code in the future so it periodically does something cancellable (or explicitly checks cancellation status of its thread)

noisesmith17:03:13

(.isInterrupted (Thread/currentThread)) - returns true if someone called future-cancel on the future you are in

schmee17:03:46

one thing you can do is to return a tuple of [cancel-fn future] where the cancel-fn flips a condition in the loop to false to it stops

noisesmith17:03:10

@schmee you alread get that for free via isInterrupted and future-cancel

polymeris17:03:02

Hmm, would have to contaminate function-under-test with testing logic

schmee17:03:35

well, the question is: why can it get stuck in an infinite loop in the first place?

noisesmith17:03:05

cancellation from the outside is not safe in the jvm, period - if you need to be cancellable from the outside that needs to be done in a cooperative way

noisesmith17:03:54

if this just happens in testing, note the still-running threads (a status you can check on a future), print a message, shut down the vm

polymeris17:03:52

it's basically iterate + a transducer, with an optional count limit. But under some circumstances the transducer filters out all values produced by iterate, and count is never reached

polymeris17:03:06

I could add an until-fn parameter, and check that fn every iteration. Then pass (.isInterrupted ... from the test suite

noisesmith17:03:12

does it really hurt to check for interruption? it's a cheap check, and even if you don't plan on doing it, it's a sane thing to put in a loop that's in a future

noisesmith17:03:47

I'm tempted to write a version of loop that implicitly stops on isInterrupted actually... - would be an easy macro to write

polymeris17:03:50

Typically the loop would not be in a future, that's only for testing

noisesmith17:03:52

while we are thinking outside the box, if the test hangs, kill the process, and call it failed :P

noisesmith17:03:03

then you don't even need a future

polymeris17:03:01

Yeah, I think that makes the most sense. Just restart the REPL when running tests from it during development

polymeris17:03:13

Or live with the high CPU

bajrachar17:03:43

@noisesmith @alexmiller - Just discovered that you can't reduce over a transient map - wondering why this is not supported on transient?

Alex Miller (Clojure team)17:03:55

I think transients are not seqable?

noisesmith17:03:14

what's the use case where you would want to reduce over a transient without making it persistent?

Alex Miller (Clojure team)17:03:26

I’m pretty sure that’s intentional but I don’t know that I have a detailed reasoning for why

bajrachar17:03:27

yes that is the case - examples I've seen show reduce over persistent map

noisesmith17:03:44

my impression is they aren't meant to be a replacement for the persistent data, but an optimization specifically for filling them while creating less gc churn

noisesmith17:03:54

(and therefore better perf)

noisesmith17:03:30

in that sense, seq would be outside the intended scope maybe?

bajrachar17:03:40

well - I was trying to mostly work with transient map inside a fold reduce method - I have a nested map and I am basically making it transient on the onset - and trying to work with it before making it persistent

noisesmith17:03:06

yeah, I have never heard of nested transients working out nicely - I bet if you lifted the nested data to a denormalized form where the data was lifted to the top level you'd also get better space usage(?)

Alex Miller (Clojure team)17:03:23

I don’t know that it would make sense to build an immutable seq out of a mutating collection

hiredman17:03:33

I would do it without transients, and if it is too slow, come up with a faster way to do it, vs trying to make the way you are doing it faster with transients

bajrachar18:03:37

@hiredman it isn't too slow - it is just taking up too much memory for the amount of data I have

bajrachar18:03:46

this is without transients that is

hiredman18:03:50

if all you are doing is building a single map concurrently, maybe use a concurrenthashmap

bajrachar18:03:29

Let me post what I've done - (this isn't working 🙂

bajrachar18:03:05

as you can see -- trying to use transients in nested map -

bajrachar18:03:30

got the error in the reduce - saying it can't reduce over transient

hiredman18:03:07

have you gotten it working at all with transients?

hiredman18:03:30

(have you seen it reduce the memory usage)

bajrachar18:03:12

I can't tell at the moment since it's not working

hiredman18:03:07

it looks like you may be mutating at least one of the transients in place

bajrachar18:03:25

yes that is the innermost in the nested map

hiredman18:03:40

you are not allowed to mutate transients in place

hiredman18:03:19

you have to pass the return value around just like the non-transient datastructures

bajrachar18:03:43

yes it recursively makes it persistent when it returns up

hiredman18:03:58

no, I mean, it is hard to see where parens balance in an image, but it looks like you have (do (if ... bang-on-transient) (assoc! ...))

hiredman18:03:29

so you are not keeping the return value of the if, which means the mutating of the transient there violates the contract of transients

bajrachar18:03:14

hmm - not sure what you mean - yes I do have two assoc! inside the do block

bajrachar18:03:27

one is within the if

hiredman18:03:08

(let [some-transient-map ...] (dotimes [i 10] (assoc! some-transient-map i i)))

bajrachar18:03:08

At this time - not married to using transients - wanted to try it out

hiredman18:03:37

that kind of thing is explicitly prohibited in the documentation for transients

noisesmith18:03:24

@bajrachar that will reduce memory usage by dropping your data on the floor :)

hiredman18:03:54

you have to use something like loop/recur and pass around the return value the call to assoc! just like you would have to pass around the return value from assoc

bajrachar18:03:48

Hmm - yes it does warn about that 🙂

noisesmith18:03:32

the thing is that assoc! does incidentally update the input in place sometimes - just often enough to lead people to make the wrong assumptions from repl experiments

bajrachar18:03:56

yes that is exactly what happened - I tested it on repl first

noisesmith18:03:14

as soon as an assoc! requires a new leaf on the internal tree of vectors, you simply drop the reshuffled data if you don't use the return value

noisesmith18:03:50

err - not leaf, I can't say that specifically - I can say there are certain operations that cause data to be shuffled then lost

bajrachar18:03:43

Well thanks for pointing that out @hiredman

bajrachar18:03:24

do you think I am better off using something like a java hashmap in this case instead of clojure data structure?

noisesmith18:03:10

yeah, I think I mentioned that the other day as the thing that would actually reduce your memory usage

noisesmith18:03:14

and you can also bash it in place

hiredman18:03:30

hard to say, you were in hear the other day saying you are trying to come with an alternative to some in memory database thing, but everything was using too much memory right?

bajrachar18:03:54

well the in-memory database is working -

bajrachar18:03:02

it's also not taking up much space

bajrachar18:03:09

but it's a bit slower

hiredman18:03:16

I would be super surprised if any data structure would beat a "database" for memory usage

bajrachar18:03:18

and seems like a lot of effort to tune it

carocad18:03:23

hey guys, does anybody know a multi-key map implementation? I mean a map where multiple keys map to the same value?

carocad18:03:52

I can cope with a multi-value map but I cannot get my head around a multi-key one 😞

hiredman18:03:02

a datastructure like a map will just have a pointer to your object, where as a database is more likely to copy it to some kind of storage optimized format

hiredman18:03:18

@carocad use clojure.set/index

carocad18:03:02

thanks @hiredman. That does the job, though I was looking something more like data.priority-map, i.e. with automatic book-keeping in case a value is updated. But your solution gets me 90% there 🙂

bajrachar18:03:29

yes - however I guess the put and get on the db does take up a bit of time - maybe due to the encoding/de-coding involved - and at this time I don't have enough knowledge to tune that stuff.

bajrachar18:03:43

well - @noisesmith @hiredman- thank you for your suggestions

qqq18:03:13

(def x (fn foo [])) given the above, is there some function that goes from x to foo ?

noisesmith18:03:58

@qqq - no, functions don't know or store their names

noisesmith18:03:14

the actual object name will be based on the name, but the solutions there are hacky

qqq18:03:49

@noisesmith: why is it that stack frames can pull the name out

qqq18:03:05

that seems to imply the information is 'somewhere in there'

Alex Miller (Clojure team)18:03:12

that’s the class name

Alex Miller (Clojure team)18:03:42

you can call demunge to go from the class name to the function name but the other direction is lossy, so you won’t always get the original thing back

qqq18:03:06

I see; thanks;

noisesmith18:03:16

@qqq you get something like peregrine.circle$eval47604$foo__47605 which has the name in it, but no part of it is the actual name

noisesmith18:03:21

yeah, you can process the string

Alex Miller (Clojure team)18:03:27

by demunge, I specifically mean clojure.lang.Compiler/demunge

schmee19:03:48

this is potentially a dumb question so apologies in advance: I’m trying to understand the Clojure compiler, and each expr has two methods: eval and emit. As far as I understand, eval, well, evals the code, and emit emits bytecode. what I don’t understand is: in what context are these functions called? why do both exist? does emit emit bytecode for what eval does? if not, what’s the difference?

gtrak19:03:23

things need to get emitted as classes to be eval'd and loaded by the JVM, except for like the 'do' block, literals, etc. You should be able to trace it.

schmee19:03:16

yeah, I’m trying to wrap my head around it, but I’m confused by the difference between compile, analyze, eval, emit… a short high-level explanation about how these relate would make it easier to explore on my own 🙂

bronsa19:03:31

eval is a very small interpreter that clojure uses for simple expressions at the repl

bronsa19:03:41

just read emit and you'll be fine

gtrak19:03:51

basically any code in a function (perhaps all code) needs to be code-gen'd (emitted) before it can be eval'd, and not-everything typed at the REPL needs to be emitted to run.

Alex Miller (Clojure team)19:03:43

@bronsa I’m still waiting for you to write the comprehensive book explaining the Clojure compiler

the2bears20:03:23

Would buy for sure, and actually read it, too 🙂 . Seriously, love the idea.

schmee19:03:01

okay, so if I’m running a Clojure program from a JAR, eval will not be involved at all, it’s all emitted?

noisesmith19:03:12

you can run uncompiled source from a jar - that's normal when using libraries in a repl for example

bronsa19:03:52

@alexmiller a book for a dozen interested people doesn't sound like a hit :)

schmee19:03:09

I’d pay good money for it! 😄

bronsa19:03:10

maybe some blogposts some time

bronsa19:03:30

but I've been saying that for the past 5 years

bronsa19:03:13

@schmee what if I tell you it's more fun to figure it out on your own :)

gtrak19:03:02

the CLJS one is fun too, if you like huge recursive print statements.

schmee20:03:44

yeah it is fun to explore on your own, I just feel like if I could sit down with someone who knows the compiler and talk with them for 30 minutes I could spare myself soooo much confusion 😛

bronsa20:03:18

feel free to ask when you have questions

dpsutton20:03:42

@tbaldridge does some video series. if you just sat down and did some screencasts and put them behind a paywall of you explaining things, how you develop patches, etc, i'd subscribe and pay for sure

bronsa20:03:36

I can think about that

schmee20:03:29

if I type (fn [a b] (+ 1 2)) at the REPL, does that get evaled or emited?

bronsa20:03:45

there's no eval for fns

bronsa20:03:52

there's a small evaluation part that just instantiates the newly emitted class

schmee20:03:30

okay, so a function is a class that’s compiled at runtime and loaded dynamically

schmee20:03:25

how does that compare to a MethodHandle? Am I correct that a MethodHandle is basically a “native” JVM way to express the same thing?

bronsa20:03:40

note that functions unlike most other contructs are compiled at analysis time

bronsa20:03:25

clojure compilation predates indy and methodhandles

schmee20:03:46

yeah, I just want to see if I have the right idea about what’s going on 🙂

bronsa20:03:14

using modern jvm bytecode we could avoid creating so function classes in some cases, but for AOT it'd be necessary anyway in most cases

hiredman20:03:55

methodhandles are more similar to the clojure.lang.IFn interface

bronsa20:03:10

yep and they still need a concrete class behind them

schmee20:03:39

okay, this is the stuff I’ve been curious about lately, does this even make any sense at all? 😄 Clojure -> method resolution: emit class as bytecode + dynamic classloader, method invokation: static method call Hypothetical Java 1.8+ Clojure -> method resolution: invokedynamic + bootstrap method, method invokation: MethodHandle combinators?

ghadi20:03:20

you still need bytecode to grab a method handle from

ghadi20:03:02

emit a function, load its class, grab its invoke(...) method as a MethodHandle

schmee20:03:35

would you still need the IFn interface if you had MethodHandles?

bronsa20:03:02

would be useful for interop with java

hiredman20:03:22

john rose did a very early experiment with method handles where he implement scheme as a single classfile. where, if I recall, function values where methodhandles

schmee20:03:03

@ghadi what do you gain by using a MethodHandle to call invoke instead of calling invoke directly?

hiredman20:03:25

you get rid of invoke

hiredman20:03:48

you can create a method handle that directly points to a static method

hiredman20:03:45

you can create a method handle that when called matches a regex against a string

hiredman20:03:10

(not a great example)

schmee20:03:35

ahh, right, because now all fn calls is a virtual method that does some loads and the calls a static method?

hiredman20:03:27

right now for an object to be callable as a function in clojure it must implement IFn

hiredman20:03:22

I suspect you are getting confused because of the newish direct linking bits in the compiler

hiredman20:03:48

in the general case calling a function invokes an instance method on the function object

hiredman20:03:26

the compiler has some optimizations for cases where a function doesn't close over any values

hiredman20:03:00

those can be turned in to static methods on the generated class

hiredman20:03:31

but you still need to be able to create and instance of the class and invoke it the general way

schmee20:03:57

that might be it, when I disassemble (fn [a b] (+ a b)) there is both public java.lang.Object invoke(java.lang.Object arg0, java.lang.Object arg1); and public static java.lang.Object invokeStatic(java.lang.Object a, java.lang.Object b);, is that what you’re talking about?

hiredman20:03:02

because IFn is the general calling convention

schmee20:03:16

ahh so that = direct linking?

schmee20:03:37

awesome, I’ve learned something new 🙂

hiredman20:03:50

IFn basically solves the problem (to some degree) that methodhandles would solve, so if you really wanted a reason to use methodhandles, you would rip out IFn and instead of using instances of IFn as functions you would use methodhandles

schmee20:03:26

I see, thanks a lot for your explanations hiredman, appreciate it 👍

staypufd20:03:58

Hello all. Has anyone used Schema to define a schema for validation that a File is being returned from a GET? For like a file that has been uploaded and then someone is using a GEat api to retrieve it. I’m trying to use the :responses

ghadi20:03:22

But methodhandles can do a lot more than replace IFn -- think inline caches, etc.

schmee20:03:45

yeah, I’m trying to fit all these pieces together somehow, basically what I’m trying to figure out is “how can I use modern JVM features to make Clojure faster?” 🙂

schmee20:03:40

ghadi I saw the experiment you did with invokedynamic btw, I’m def going to take a closer look at that! 🙂

tbaldridge20:03:43

@schmee if you want the answer to that I'd dig into Graal and Truffle. With the stock JVM you aren't going to make it much faster

tbaldridge20:03:13

But with Truffle/Graal/SubstrateVM I think there's some cool situations there.

tbaldridge20:03:04

Most of that is because as it stands today Clojure is very sympathetic to the JVM. What Truffle allows is to make the JVM more sympathetic to the language.

hiredman20:03:48

clojure is surprisingly static for a dynamic language(I think at one point a jruby developer said it was cheating because it is so static) so the jvm does pretty well with it

tbaldridge20:03:33

Stuff like transducers and other uses of HOF are weak places for this rule though. Run a transducer over primitives in CLJS vs CLJ and the JS JIT is way faster.

Alex Miller (Clojure team)20:03:57

I wonder if they ever get the value object stuff working if that would be a way around that

ghadi20:03:57

there are large pockets of static calling behavior but let's not underestimate how dynamic it all is

schmee20:03:12

@tbaldridge I’m actually looking at Graal/Truffle right now! thing i,s, rewriting the whole compiler in Truffle is waaaay out of my league, and even if it wasn’t, I don’t see a Truffle compiler replacing the official one any time soon. so to get these improvements to users it needs to happen in the existing compiler 🙂

ghadi20:03:56

direct linking came in with a tradeoff on dynamicity. indy would allow you to have static calls without trading off dynamicity

ghadi20:03:21

i had a branch somewhere that linked vars with indy

Alex Miller (Clojure team)20:03:26

I think that’s a good way to put it

tbaldridge20:03:15

Truffle doesn't replace anything. It's a library that hooks into the JVM's JIT. Those hooks exist in Graal and JVM9.

ghadi20:03:18

yeah somewhere in there

hiredman20:03:29

https://gist.github.com/hiredman/d6ea3e30be757fef6da6 someone just needs to write a library of invokePrim transducers

tbaldridge20:03:00

@hiredman I don't think that keeps the comp callsites from going megamorphic though

schmee20:03:52

@ghadi do you think the indy stuff in your branch a good starting point for future work in that direction?

ghadi20:03:28

No it predates direct linking

ghadi21:03:22

though you can cherry-pick the indy stuff on top of my changes on CLJ-2284

tbaldridge21:03:32

@ghadi does the indy stuff support deoptimization?

ghadi21:03:31

which indy stuff

tbaldridge21:03:06

just invokedynamic in general, is it possible to setup a condition that causes the callsite to be re-initialized?

ghadi21:03:27

yeah. that's essential for correctness when you do an inline cache

ghadi21:03:46

if you redefine a var, the code I wrote re-links it

ghadi21:03:55

at all call sites

dpsutton21:03:46

I hope that there might be four blog posts written from this discussion

schmee21:03:06

@ghadi ooohh, does CLJ-2284 contain a version bump of asm as well?

schmee21:03:10

I see now that it does, sry for noise

dominicm21:03:30

Could I perform batching/timeouts in a transducer? For example, if nothing new comes in for a while, it should add it's state?

tbaldridge21:03:09

@dominicm no, since transducers are single-threaded and push oriented

dominicm21:03:32

I wasn't certain if they were single threaded. A bit of a shame, the idea of a multi-purpose batch system appealed to me. Especially as I wouldn't need to break my transducer pipelines up with an intermediate step. Thanks @tbaldridge!