Fork me on GitHub

Hi, (type (int 10)) returns java.lang.Integer. How do i create a primitive int in clojure?


int will actually create a primitive int, but those can only exist in local scope, ie let


when you call int in the REPL, the value will be boxed before it is returned and printed


methods can be tagged to return primitives, but only longs and doubles


let's say I parsed some json, i.e. {"First Name": "bar"} and I keywordized the keys print like: {:First Name "bar"}. Would be a safe way of printing this to EDN such that it could be read safely?


Although it’s possible to create keywords with spaces, they’re not something you can use as literals.


have you considered not keywordizing?


it's coming from user input


{"First Name" "bar"}


FWIW, ClojureScript itself does what @hiredman is suggesting when it needs to print a JavaScript object like that

#js {"First Name" "bar"}
specifically so you can copy and paste it in the REPL, etc., and it can be read.


This is as compared to

#js {:name "bar"}


I guess you can create keywords with spaces in them, just not use them as literals.


You can use keyword to make objects that are not legal keywords.

Alex Miller (Clojure team)03:03:41

they are legal keywords

Alex Miller (Clojure team)03:03:50

they are not print-read round-trippable

Alex Miller (Clojure team)03:03:25

they are not valid literal keywords


aye, my use-case is it's coming from user input, and i wanted to try round-trippin' the data using EDN instead of JSON


You can do that, just don’t keywordize the keys.


definitely can, was more so wondering if folks know something that i might've missed


so let's say i'm leaning towards a serialization format that gives me all of clojure's flexibility, would that be to lean towards transit?


I do mostly prototypes, so for me, just edn itself is fine.


On the Clojure side, something like: (clojure.edn/read-string (slurp (:body request)) or variants, depending on your web server.


When sending in ClojureScript: #js { “method” “POST”, “body” (pr-str data)}.


I know about using reader macros in cljc:

 (catch #?(:clj Exception
           :cljs :default)
Now, I need to define a macro which expands to a try/catch. How do I get this macro to expand to Exception in clj and :default in cljs ?


is there a way to tell clojure not to concern itself with whether or not a pulled in function is private from a separate namespace?


It errors out over a method that was set private, however, it doesn't error out when I pull it into clojurescript


@benzap you can use the #' prefix on the name to override the private access.


I'm looking to build a very high performance web server. nginx-clojure looks like a good choice. Am I right?


@danielstockton I had good experience using it, it still in progress, so no benchmarks, but it's nice to have both your static resources and some handler in the save instance. I use this to be able to upload images, which then gets used in the html, and served by nginx. Combined with cashing, pages should load very close to just static files.


@gklijs In my case, i'm not planning on serving any static files. It's an API service which will receive a lot of traffic.


Ok, nginx uses multiple JVM do you probably want to use redis to keep start and/or only call other services in the handler, both should be fine.


@danielstockton just wondering, what are your constraints/numbers?


I'd like to use Clojure, that's the only constraint really. It will have to scale horizontally, so i'm not trying to get a certain number of requests from a single instance. I just want to get the most bang for my buck and minimize response times.


@gklijs Sorry, what do you mean by 'use redis to keep start'?


Meant state, not start. But really depends on what kind of API it is. The second call to the API might gets handled by a different JVM instance then the first one, that's all.


Oh right, ok. Yes, I will be making calls to other data stores within the handler (most likely redis is one of them).


I won't assume that the Clojure process has access to state shared between requests, that's for sure.


@danielstockton any reason that Undertow (through Immutant) couldn’t provide enough bang for the bucks?


heh - the same one I’m currently have open in my browser 😉


anybody is using jsonista in prod yet?


looks good on paper


is it same to assume that jvm exceptions are highly optimized so that if there is no exception thrown, the cost of negilible ?


i think exceptions are just longjumps and are "cheap". But I'm not a jvm guy just what i've heard


ah, so in theory, we can imagine "global pointer handler; // points to the catch block to call on exception" then, the cost of a "try" is: 1. save old value of handler 2. set handler to new catch block ?


That sounds both computationally inexpensive and sufficient for implementing try/catch blocks.


A quick google suggests that try/catch is relatively quick, it’s the throwing of the exception that’s expensive


AFAIK what’s expensive is to construct the stack trace, so to get maximum performance you should throw a pre-allocated exception, in that case it’s basically just a jump instruction


this blog post is excellent and tells you everything you need to know 🙂


@schmee: a quick search of the blog post seems ot suggest it doesn't covedr this 'pre allocation' trick; can you point me at where that trick is explained ?


he calls them “static” exceptions, if you really want to know what’s going on just read the whole thing

Alex Miller (Clojure team)14:03:58

I’ve used that trick in Java :)


@alexmiller since you’re around I’ll ask a completely different question I’ve been thinking about: how long does Clojure intend to support Java 1.6? has there been any discussion around moving the oldest version up to, say, Java 8?

Alex Miller (Clojure team)15:03:07

I would like to move to Java 8 as the min bar. We haven’t had a discussion about it for Clojure 1.10 yet


roger, thanks for the info


Wouldn't changing be a form of breaking backwards compatibility?

Alex Miller (Clojure team)15:03:03

it sounds like you’re trying to begin a semantic argument that I’m totally uninterested in :)

Alex Miller (Clojure team)15:03:41

realistically, very few people are using 1.6 (or even 1.7 now) and it’s getting difficult to even support in our build infrastructure


also, invokedynamic, which will totally solve all our problems and make clojure 1000x faster with almost zero effort


jokes aside, I’m really excited about that stuff and I’m looking at ways to get it into the Clojure compiler, hence my question about the version 🙂


@alexmiller I am genuinely confused by the the notion that java version compatibility isn't considered part of the API of Clojure. I accept the difficulties of course, I personally would love to see the minimum bar raised for my own selfish needs, because seeing better java 8 time integration would be awesome. I sometimes hear something which questions my understand of something Rich has said, so this is really an attempt to gain insight into that.

Alex Miller (Clojure team)15:03:48

it is of course considered

Alex Miller (Clojure team)15:03:30

generally upgrading java is an additive experience - you get access to new stuff and existing stuff continues to work, same as Clojure

Alex Miller (Clojure team)15:03:45

so the upgrade path is still maintaining backwards compatibility


> Upgrade path There's a nuance here about the "API" of upgrading Clojure here that's interesting, and completely pivots your ability to shift the requirement. This was useful, thank you 🙂


My understanding of exceptions was that it was supposed to be fairly cheap to take the normal flow brancch, but not the exception branch. Also, I believe that try-catch blocks may have some affect on JVM optimizations


I forget now, maybe it is in that post that I need to read above. I was thinking it may have some affect on certain memory barriers and/or inlining


Also, If I remember correctly, I thought that Clojure seemed to generate heavier byte-code for try-catch than from what I saw in Java… Maybe I’m crazy though. Will have to check again


(just wondering if this triggers anyone’s memory on the topic)

Alex Miller (Clojure team)15:03:38

try/catch is not actually bytecode per se - it’s really a handler table attached to the bytecode

Alex Miller (Clojure team)15:03:52

Clojure’s output should not really be any different Java’s afaik

Alex Miller (Clojure team)15:03:56

there are effects on inlining, but generally I find there is far more mythology about this stuff than reality and it is constantly improving

Alex Miller (Clojure team)15:03:45

Shipilev is exactly the guy to tell you the real answer and not the mythology

Alex Miller (Clojure team)15:03:05

so I’d believe him :)


@alexmiller I’m reading through that post above, so far already good. So will continue. I don’t tend to get worked up about the effect it has on perf either. I just have wondered before, especially in Java code, when I saw try-catch control flow being used for whatever reason and it happened to be potentially a method called in a hot path. Something like loading classes having to use ClassNotFoundException errors for flow. I want to relook at my experience with what Clojure generated. I thought it had more bytecode operations involved than I expected and saw for Java


I wasn’t implying that the try/catch was just a bytecode or something like that. I just thought the branching that clj output costs more time

Alex Miller (Clojure team)15:03:57

one possible difference is that because we are still using java 5 compatible bytecode, I don’t think we’re using the “new” exception style handling they moved to in java 6 bytecode (I think those are the right numbers, haven’t looked at that stuff in a while), but that’s pretty deep in the technical weeds. we go through asm and generate standard bytecode in the same way as all the other jvm languages in this regard.


hmm, I get a null pointer exception on (pos? ...) in this code:

processed (reduce + (map second (get grouped :processed)))
processed (when (pos? processed)
                            (str "/ Processed: " processed))
When can (reduce + ...) return nil?


user=> (reduce + nil)
user=> (reduce + nil nil)
user=> (reduce + [nil nil])
NullPointerException   clojure.lang.Numbers.ops (
user=> (reduce + [nil])


should probably be using the init arity of reduce


user=> (reduce + 0 nil)
user=> (reduce + 0 [nil nil])
NullPointerException   clojure.lang.Numbers.ops (
user=> (reduce + 0 [nil])
NullPointerException   clojure.lang.Numbers.ops (


This particular expression returns nil: (reduce + (map ...))


I don’t miss a type system often, but this is one of them


(reduce + (map identity [nil]))


the collections that map operates on has a single nil inside


thanks, makes sense now


I’m curious now. This is something I looked at pretty long back. I can check it out and see if I can rediscover whatever I thought I did before


never mind


Testing a function that might get stuck in an infinite loop, so wrapping it in a future:

(defn happy-test-case [[input expected]]
    (let [f (future (function-under-test input))
          result (deref f 1000 :timeout)]
      (future-cancel f)
      (= expected result))))
After running the tests I am left with high CPU usage, so I suspected some futures where left orphaned, and added future-cancel. However it does not fix the issue. Ideas?


@polymeris future-cancel only interrupts certain specific methods


if those methods never get called, the cancellation does nothing


is there a future-kill or something similar?


nothing that is sane to use without shutting down the entire vm


the smart thing to do is write the code in the future so it periodically does something cancellable (or explicitly checks cancellation status of its thread)


(.isInterrupted (Thread/currentThread)) - returns true if someone called future-cancel on the future you are in


one thing you can do is to return a tuple of [cancel-fn future] where the cancel-fn flips a condition in the loop to false to it stops


@schmee you alread get that for free via isInterrupted and future-cancel


Hmm, would have to contaminate function-under-test with testing logic


well, the question is: why can it get stuck in an infinite loop in the first place?


cancellation from the outside is not safe in the jvm, period - if you need to be cancellable from the outside that needs to be done in a cooperative way


if this just happens in testing, note the still-running threads (a status you can check on a future), print a message, shut down the vm


it's basically iterate + a transducer, with an optional count limit. But under some circumstances the transducer filters out all values produced by iterate, and count is never reached


I could add an until-fn parameter, and check that fn every iteration. Then pass (.isInterrupted ... from the test suite


does it really hurt to check for interruption? it's a cheap check, and even if you don't plan on doing it, it's a sane thing to put in a loop that's in a future


I'm tempted to write a version of loop that implicitly stops on isInterrupted actually... - would be an easy macro to write


Typically the loop would not be in a future, that's only for testing


while we are thinking outside the box, if the test hangs, kill the process, and call it failed :P


then you don't even need a future


Yeah, I think that makes the most sense. Just restart the REPL when running tests from it during development


Or live with the high CPU


@noisesmith @alexmiller - Just discovered that you can't reduce over a transient map - wondering why this is not supported on transient?

Alex Miller (Clojure team)17:03:55

I think transients are not seqable?


what's the use case where you would want to reduce over a transient without making it persistent?

Alex Miller (Clojure team)17:03:26

I’m pretty sure that’s intentional but I don’t know that I have a detailed reasoning for why


yes that is the case - examples I've seen show reduce over persistent map


my impression is they aren't meant to be a replacement for the persistent data, but an optimization specifically for filling them while creating less gc churn


(and therefore better perf)


in that sense, seq would be outside the intended scope maybe?


well - I was trying to mostly work with transient map inside a fold reduce method - I have a nested map and I am basically making it transient on the onset - and trying to work with it before making it persistent


yeah, I have never heard of nested transients working out nicely - I bet if you lifted the nested data to a denormalized form where the data was lifted to the top level you'd also get better space usage(?)

Alex Miller (Clojure team)17:03:23

I don’t know that it would make sense to build an immutable seq out of a mutating collection


I would do it without transients, and if it is too slow, come up with a faster way to do it, vs trying to make the way you are doing it faster with transients


@hiredman it isn't too slow - it is just taking up too much memory for the amount of data I have


this is without transients that is


if all you are doing is building a single map concurrently, maybe use a concurrenthashmap


Let me post what I've done - (this isn't working 🙂


as you can see -- trying to use transients in nested map -


got the error in the reduce - saying it can't reduce over transient


have you gotten it working at all with transients?


(have you seen it reduce the memory usage)


I can't tell at the moment since it's not working


it looks like you may be mutating at least one of the transients in place


yes that is the innermost in the nested map


you are not allowed to mutate transients in place


you have to pass the return value around just like the non-transient datastructures


yes it recursively makes it persistent when it returns up


no, I mean, it is hard to see where parens balance in an image, but it looks like you have (do (if ... bang-on-transient) (assoc! ...))


so you are not keeping the return value of the if, which means the mutating of the transient there violates the contract of transients


hmm - not sure what you mean - yes I do have two assoc! inside the do block


one is within the if


(let [some-transient-map ...] (dotimes [i 10] (assoc! some-transient-map i i)))


At this time - not married to using transients - wanted to try it out


that kind of thing is explicitly prohibited in the documentation for transients


@bajrachar that will reduce memory usage by dropping your data on the floor :)


you have to use something like loop/recur and pass around the return value the call to assoc! just like you would have to pass around the return value from assoc


Hmm - yes it does warn about that 🙂


the thing is that assoc! does incidentally update the input in place sometimes - just often enough to lead people to make the wrong assumptions from repl experiments


yes that is exactly what happened - I tested it on repl first


as soon as an assoc! requires a new leaf on the internal tree of vectors, you simply drop the reshuffled data if you don't use the return value


err - not leaf, I can't say that specifically - I can say there are certain operations that cause data to be shuffled then lost


Well thanks for pointing that out @hiredman


do you think I am better off using something like a java hashmap in this case instead of clojure data structure?


yeah, I think I mentioned that the other day as the thing that would actually reduce your memory usage


and you can also bash it in place


hard to say, you were in hear the other day saying you are trying to come with an alternative to some in memory database thing, but everything was using too much memory right?


well the in-memory database is working -


it's also not taking up much space


but it's a bit slower


I would be super surprised if any data structure would beat a "database" for memory usage


and seems like a lot of effort to tune it


hey guys, does anybody know a multi-key map implementation? I mean a map where multiple keys map to the same value?


I can cope with a multi-value map but I cannot get my head around a multi-key one 😞


a datastructure like a map will just have a pointer to your object, where as a database is more likely to copy it to some kind of storage optimized format


@carocad use clojure.set/index


thanks @hiredman. That does the job, though I was looking something more like data.priority-map, i.e. with automatic book-keeping in case a value is updated. But your solution gets me 90% there 🙂


yes - however I guess the put and get on the db does take up a bit of time - maybe due to the encoding/de-coding involved - and at this time I don't have enough knowledge to tune that stuff.


well - @noisesmith @hiredman- thank you for your suggestions


(def x (fn foo [])) given the above, is there some function that goes from x to foo ?


@qqq - no, functions don't know or store their names


the actual object name will be based on the name, but the solutions there are hacky


@noisesmith: why is it that stack frames can pull the name out


that seems to imply the information is 'somewhere in there'

Alex Miller (Clojure team)18:03:12

that’s the class name

Alex Miller (Clojure team)18:03:42

you can call demunge to go from the class name to the function name but the other direction is lossy, so you won’t always get the original thing back


I see; thanks;


@qqq you get something like$eval47604$foo__47605 which has the name in it, but no part of it is the actual name


yeah, you can process the string

Alex Miller (Clojure team)18:03:27

by demunge, I specifically mean clojure.lang.Compiler/demunge


this is potentially a dumb question so apologies in advance: I’m trying to understand the Clojure compiler, and each expr has two methods: eval and emit. As far as I understand, eval, well, evals the code, and emit emits bytecode. what I don’t understand is: in what context are these functions called? why do both exist? does emit emit bytecode for what eval does? if not, what’s the difference?


things need to get emitted as classes to be eval'd and loaded by the JVM, except for like the 'do' block, literals, etc. You should be able to trace it.


yeah, I’m trying to wrap my head around it, but I’m confused by the difference between compile, analyze, eval, emit… a short high-level explanation about how these relate would make it easier to explore on my own 🙂


eval is a very small interpreter that clojure uses for simple expressions at the repl


just read emit and you'll be fine


basically any code in a function (perhaps all code) needs to be code-gen'd (emitted) before it can be eval'd, and not-everything typed at the REPL needs to be emitted to run.

Alex Miller (Clojure team)19:03:43

@bronsa I’m still waiting for you to write the comprehensive book explaining the Clojure compiler


Would buy for sure, and actually read it, too 🙂 . Seriously, love the idea.


okay, so if I’m running a Clojure program from a JAR, eval will not be involved at all, it’s all emitted?


you can run uncompiled source from a jar - that's normal when using libraries in a repl for example


@alexmiller a book for a dozen interested people doesn't sound like a hit :)


I’d pay good money for it! 😄


maybe some blogposts some time


but I've been saying that for the past 5 years


@schmee what if I tell you it's more fun to figure it out on your own :)


the CLJS one is fun too, if you like huge recursive print statements.


yeah it is fun to explore on your own, I just feel like if I could sit down with someone who knows the compiler and talk with them for 30 minutes I could spare myself soooo much confusion 😛


feel free to ask when you have questions


@tbaldridge does some video series. if you just sat down and did some screencasts and put them behind a paywall of you explaining things, how you develop patches, etc, i'd subscribe and pay for sure


I can think about that


if I type (fn [a b] (+ 1 2)) at the REPL, does that get evaled or emited?


there's no eval for fns


there's a small evaluation part that just instantiates the newly emitted class


okay, so a function is a class that’s compiled at runtime and loaded dynamically


how does that compare to a MethodHandle? Am I correct that a MethodHandle is basically a “native” JVM way to express the same thing?


note that functions unlike most other contructs are compiled at analysis time


clojure compilation predates indy and methodhandles


yeah, I just want to see if I have the right idea about what’s going on 🙂


using modern jvm bytecode we could avoid creating so function classes in some cases, but for AOT it'd be necessary anyway in most cases


methodhandles are more similar to the clojure.lang.IFn interface


yep and they still need a concrete class behind them


okay, this is the stuff I’ve been curious about lately, does this even make any sense at all? 😄 Clojure -> method resolution: emit class as bytecode + dynamic classloader, method invokation: static method call Hypothetical Java 1.8+ Clojure -> method resolution: invokedynamic + bootstrap method, method invokation: MethodHandle combinators?


you still need bytecode to grab a method handle from


emit a function, load its class, grab its invoke(...) method as a MethodHandle


would you still need the IFn interface if you had MethodHandles?


would be useful for interop with java


john rose did a very early experiment with method handles where he implement scheme as a single classfile. where, if I recall, function values where methodhandles


@ghadi what do you gain by using a MethodHandle to call invoke instead of calling invoke directly?


you get rid of invoke


you can create a method handle that directly points to a static method


you can create a method handle that when called matches a regex against a string


(not a great example)


ahh, right, because now all fn calls is a virtual method that does some loads and the calls a static method?


right now for an object to be callable as a function in clojure it must implement IFn


I suspect you are getting confused because of the newish direct linking bits in the compiler


in the general case calling a function invokes an instance method on the function object


the compiler has some optimizations for cases where a function doesn't close over any values


those can be turned in to static methods on the generated class


but you still need to be able to create and instance of the class and invoke it the general way


that might be it, when I disassemble (fn [a b] (+ a b)) there is both public java.lang.Object invoke(java.lang.Object arg0, java.lang.Object arg1); and public static java.lang.Object invokeStatic(java.lang.Object a, java.lang.Object b);, is that what you’re talking about?


because IFn is the general calling convention


ahh so that = direct linking?


awesome, I’ve learned something new 🙂


IFn basically solves the problem (to some degree) that methodhandles would solve, so if you really wanted a reason to use methodhandles, you would rip out IFn and instead of using instances of IFn as functions you would use methodhandles


I see, thanks a lot for your explanations hiredman, appreciate it 👍


Hello all. Has anyone used Schema to define a schema for validation that a File is being returned from a GET? For like a file that has been uploaded and then someone is using a GEat api to retrieve it. I’m trying to use the :responses


But methodhandles can do a lot more than replace IFn -- think inline caches, etc.


yeah, I’m trying to fit all these pieces together somehow, basically what I’m trying to figure out is “how can I use modern JVM features to make Clojure faster?” 🙂


ghadi I saw the experiment you did with invokedynamic btw, I’m def going to take a closer look at that! 🙂


@schmee if you want the answer to that I'd dig into Graal and Truffle. With the stock JVM you aren't going to make it much faster


But with Truffle/Graal/SubstrateVM I think there's some cool situations there.


Most of that is because as it stands today Clojure is very sympathetic to the JVM. What Truffle allows is to make the JVM more sympathetic to the language.


clojure is surprisingly static for a dynamic language(I think at one point a jruby developer said it was cheating because it is so static) so the jvm does pretty well with it


Stuff like transducers and other uses of HOF are weak places for this rule though. Run a transducer over primitives in CLJS vs CLJ and the JS JIT is way faster.

Alex Miller (Clojure team)20:03:57

I wonder if they ever get the value object stuff working if that would be a way around that


there are large pockets of static calling behavior but let's not underestimate how dynamic it all is


@tbaldridge I’m actually looking at Graal/Truffle right now! thing i,s, rewriting the whole compiler in Truffle is waaaay out of my league, and even if it wasn’t, I don’t see a Truffle compiler replacing the official one any time soon. so to get these improvements to users it needs to happen in the existing compiler 🙂


direct linking came in with a tradeoff on dynamicity. indy would allow you to have static calls without trading off dynamicity


i had a branch somewhere that linked vars with indy

Alex Miller (Clojure team)20:03:26

I think that’s a good way to put it


Truffle doesn't replace anything. It's a library that hooks into the JVM's JIT. Those hooks exist in Graal and JVM9.


yeah somewhere in there

hiredman20:03:29 someone just needs to write a library of invokePrim transducers


@hiredman I don't think that keeps the comp callsites from going megamorphic though


@ghadi do you think the indy stuff in your branch a good starting point for future work in that direction?


No it predates direct linking


though you can cherry-pick the indy stuff on top of my changes on CLJ-2284


@ghadi does the indy stuff support deoptimization?


which indy stuff


just invokedynamic in general, is it possible to setup a condition that causes the callsite to be re-initialized?


yeah. that's essential for correctness when you do an inline cache


if you redefine a var, the code I wrote re-links it


at all call sites


I hope that there might be four blog posts written from this discussion


@ghadi ooohh, does CLJ-2284 contain a version bump of asm as well?


I see now that it does, sry for noise


Could I perform batching/timeouts in a transducer? For example, if nothing new comes in for a while, it should add it's state?


@dominicm no, since transducers are single-threaded and push oriented


I wasn't certain if they were single threaded. A bit of a shame, the idea of a multi-purpose batch system appealed to me. Especially as I wouldn't need to break my transducer pipelines up with an intermediate step. Thanks @tbaldridge!