This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-07-03
Channels
- # arachne (1)
- # beginners (71)
- # boot (14)
- # cider (17)
- # cljs-dev (3)
- # clojure (169)
- # clojure-gamedev (35)
- # clojure-greece (1)
- # clojure-russia (30)
- # clojure-spec (3)
- # clojurescript (16)
- # core-async (2)
- # cursive (1)
- # datomic (3)
- # dirac (1)
- # hoplon (95)
- # off-topic (10)
- # onyx (1)
- # parinfer (1)
- # pedestal (2)
- # re-frame (2)
- # reagent (2)
- # untangled (1)
I’m an admin @kirked — you can PM me if you want
Oh, thanks Sean! I need to have my 2FA disabled for Clojurians, please! It went away with Google Authenticator on my last phone.
I don’t know that admin’s have that power but I’ll take a look...
…Yup, done. Looks like a relatively new feature? I hadn’t seen that before.
For future reference (to everyone), Slack-related assistance is often easier to get in #C04V84U7G — and the admins are all listed on the Code of Conduct for this Slack.
oh, thanks. I looked through the channel list, but it's easy to miss one when there are so many!
(I was going to add a link to the CoC — https://goo.gl/VsJ8q8 — since this is the second most popular channel behind the announce-to-everyone unpartable channel)
@seancorfield: you could use ctrl-o to do that, like in emacs
Doesn't work for me, but thanks for updating it like that. Cool.
sometimes some types share the same implementation for a protocol. Is there anything like this:
`(extend-protocol Protocol AType BType (foo [x] ...) (bar [x y] ...) CType (foo [x] ...) (bar [x y] ...))`
(without using hash-maps
+ merge
)
Hm, I’m seeing a really interesting problem where a call only succeeds with type hints (reflector can’t find appropriate method with). Is that necessarily a reflector bug?
@lvh: I don't know what your case is without looking at the code, but sometimes there's ambiguity that can't be figured out at runtime. Say you have 2 interfaces, A and B, and a class C that implements both. Now say you have two a static method overloaded to take either an instance of A or an instance of B, and you invoke it with an instance of C. both methods might apply, which one should the reflector pick?
@bronsa: hm, so, the code I have now annotates with ^ByteBuffer or ^bytes, and the interface supports a bunch of different permutations for those arguments, so at least that particular problem doesn’t apply
without knowing exactly the signatures of the method you're trying to invoke and how you're invoking it I can only make guesses
relying on runtime reflection is a bad idea anyway as it's orders of magnitude slower than a call resolved at compile time
(I wouldn't be surprised if it turned out to be a bug in Reflector tho, the method matching algorithm can be non-deterministic and non-consistent and there are a number of edge-cases that don't behave as you'd expect I've found over the years)
@bronsa: So, I’m still debugging this further (and I don’t know if you’re asking for the code or not; I just feel bad imposing on your time) and I’m 99% sure it’s about DirectByteBuffer
One of the warnings I’m getting under cloverage is:
Reflection warning, caesium/crypto/secretbox.clj:1:146 - call to method crypto_secretbox_easy on caesium.binding.Sodium can't be resolved (argument types: [B, [B, unknown, [B, [B).
Reflection warning, caesium/crypto/secretbox.clj:1:146 - call to method crypto_secretbox_easy on caesium.binding.Sodium can't be resolved (argument types: java.nio.ByteBuffer, [B, unknown, [B, [B).
Reflection warning, caesium/crypto/secretbox.clj:1:146 - call to method crypto_secretbox_open_easy on caesium.binding.Sodium can't be resolved (argument types: [B, [B, unknown, [B, [B).
^ByteBuffer seems like a valid type hint; does that just not do enough evaluation to resolve static variables, or is there some weirdness around primitives I’m missing
it's.. complex.. clojure resolves type hints via Class/forName
+ special cases long
/`longs`/`object`/..
because calling long seems to fix the problem, but I can’t figure out how to type hint that
>Also is it just me or is type hinting for primitives super weird type hinting in general is super weird and inconsistent
OK, so I have a fn:
(defn secretbox-open-easy-to-buf!
"Decrypt with `crypto_secretbox_open_easy` into the given byte array.
You only want this to manage the output byte array yourself. Otherwise,
you want [[secretbox-open-easy]]."
[^bytes out ^bytes ctext ^bytes nonce ^bytes key]
(let [clen (long (buflen ctext))
res (.crypto_secretbox_open_easy sodium out ctext clen nonce key)]
(if (= res 0)
out
(throw (RuntimeException. "Ciphertext verification failed")))))
buflen is a fn from a protocol, which I ended up annotating ^Long because I couldn’t get any permutation of ^’long to workHere’s the protocol and the two impls:
(defprotocol BufLen
(^Long buflen [this]))
(extend-protocol BufLen
(Class/forName "[B")
(buflen [this] (alength ^bytes this))
ByteBuffer
(buflen [this] (.remaining ^ByteBuffer this)))
right, so protocols can't return primitive types, that ^Long
cast is useless and so would ^long
using the long
function there is the right way to do it -- you don't want to type-hint (buflen ctext)
as long
because that would be lying to the compiler (it's a boxed Long
, not a primitive long
)
so the only reason I have a protocol is because I’m expecting all calls to be hinted, so I wanted them to be resolved at compile time
also remember that type hints should not be used to cast, so if you have an int
and type-hint it as long
weird things might happen
I was just messing around with it trying to get the refl warning to go away at all in an attempt to understand hitning
your best option is to either not use protocols and just use a function & switch on the argument class, or wrap the protocol function in a type-hinted function returning long
, but you're going to pay the price of boxing+unboxing in that case
which might not be a big overhead, don't know how performance sensitive your code has to be
I mean, I guess there’s the distinction between Integer as returned by alength and the int returned by ByteBuffer/remaining
I’m guessing there’s a weird method in Arrays or something that will tell me the array length as a primitive
@lvh I can tell you that generally Java developers don't worry about un/boxing of primitives from a performance perspective. If your application is that time-sensitive you'll want to make a thin interface over native code. Usually other things will dominate when you profile.
@kirked: boxed vs unboxed math can make huge performance differences in hot code paths
sure if you're doing tons of IO you're probably not going to notice much of a difference, but in algorithmic code boxed vs unboxed does matter
as an example, tools.reader
is about twice as slow as the clojure reader written in java because clojure can't pass around unboxed chars between functions
@bronsa: Yes, I know all about boxing, having been a Java developer for 15 years.
As I said, it depends upon what you're doing whether it matters or not. I've found performance in most applications to be dominated by string manipulation or IO, not numeric algorithms.
Thanks for the info about tools.reader
. I do find some of the compilation strategies in Clojure to be strange, and think that I would've made some different choices. Then again, I never worked with traditional lisps (so wouldn't carry any baggage from that), and I'm currently happy enough to not even think about performance most of the time. I leave that to Intel, Apple, and Google.
Generally speaking, if you're running on the HotSpot JVM, it's good enough to optimise all of the hot code paths for you.
@kirked: I wasn't questioning your knowledge of java/boxing FWIW if that's how it came across
@bronsa: np, I just wanted to let you know some of my background. I've done some amazing things on the JVM in that time (some of which "couldn't be done"). 🙂 I find it a bit strange that I keep seeing performance coming up in Clojure discussions, tho. My overall perception is that Clojure is plenty fast enough. But then again, I'm not controlling medical or nuclear devices 😄
generally I'd agree with you that focusing too much on performance of algorithmic code might be a premature/useless optimization -- if we were talking about application code. But @lvh's use case is an algorithmic library code as far as I can tell, and as a rule of thumb I tend to write my libraries to be as fast as they can. Sure the difference between .4ms and 2ms might be completely irrelevant to most use cases, but for others (which as a library writer, you can't foresee!) might mean the difference between your library being usable or not
(OT with this discussion, but I'd like to know what compilation strategies of clojure you find strange, as that's an argument that interests me quite a bit)
Hah! Ironically enough, my perception of that has to do with performance optimisation! (I do think it matters for compiler output, for one example). Just on cursory glance (I haven't made any deep study), I would've chosen to emit functions differently. For pure functions, there's no reason not to emit them as JVM static methods, which dispatch faster than instance methods. I'd also emit a module as a single JVM class.
If you're interested in this topic, there's a really nice old book Programming for the Java Virtual Machine (Joshua Engel, Addison Wesley) that gives good coverage of the topic. It's too old to include some of the bytecode additions they've made since 1.5 (e.g., frames), but I believe nearly everything in the book is still applicable to today's VMs.
I'm trying to connect to a remote nREPL with a given ip/port, but cider-connect
is trying to open an SSH connection. Any ideas how to make it connect directly?
@frozenlock: Which version of CIDER are you using? That sounds like an issue that was resolved a few versions back...
0.11 on the server side.
I was on 0.11 in Emacs, but just updated and I still have the same issue 😞
seancorfield: I think it's still present in the latest version https://github.com/clojure-emacs/cider/issues/1544
Having read all of that (both tickets) I'm trying to think what we did when we ran into this problem...
I read them both, but I don't see any obvious solution.
Yeah, me neither... Sorry 😐
In my context, this is crypto code, so I do particularly care about perf and especially about consistent, predictable perf.
I mostly know the relevant parts of the JLS; I’m just confused by the Clojure-specific parts 🙂
(Someone talked about this with me earlier, but it passed the 10,000 msg threshold)
bronsa: Does the suggestion with Interfaces work for existing types? I’m counting the length of byte arrays (as in [B
) and java.nio.ByteBuffers; my understanding was that JVM classes are immutable, so you can’t teach them a new interface (which is part of the reason protocols are cool)
Also, do those type hints cross fn borders? So:
(defprotocol BufLen (-buflen [this])
(defn buflen ^long [this] (long (-buflen this)))
(defn f [^ByteBuffer b]
(-buflen b) #_hinted
(buflen b) #_hinted? )
part of the problem is that I’m implementing new crypto using these tools, and, for better or for worse, the crypto community doesn’t care much unless you get reasonable cpb (cycles per byte)
the -buflen
call isn't hinted, only the buflen
one will return a long
-- if you want to return a primitive long from f
you'll have to type-hint f
aswell
so (defn buflen ^long [b] (long (case (class b) .. ))
will avoid the boxing + unboxing
@bronsa: I’m assuming that explicit type dispatch like that, while still slower than compile-time dispatch, is a lot faster than Clojure’s Reflector?
but yeah, I'd go with the protocol + wrapping fn & profile if the additional boxing is an issue, the overhead might be negligible
maybe case + map type is appropriate as well, because I know not all of them really exist, it’s just a handful
primitive support in clojure fns is limited to 4 arguments unfortunately, if you need more than that I don't think you're going to have much luck avoiding boxing
there are a bunch of tricks involving creating a deftype + using an interface that can get you around that but it's not pretty and often unnecessary unless you really need no boxing at all
because slamhound wants to delete that import and I can’t tell if it’s because it doesn’t understand type hinting or if I don’t understand type hinting 😄
I’m hoping to do some autogeneration for this because a lot of the code is very repetitive
yeah but type hinting has proven confusing enough before trying to do it in a macro and remembering why ^bytes doesn’t work and you really wanted vary-meta or something 😄
but it appears that maybe jnr-ffi + type hinting + macros are too many chainsaws to juggle at once
AFAICR cloverage does some weird code-rewriting thing, which might be dropping the type-hints
@seylerius I don't think I wrote my example down anywhere other than here, but I can whip up another example for you when I get home tonight
@bronsa: The hints you gave were super helpful, https://camo.githubusercontent.com/0dbd922ca8c6406767e8768ce29aea8c29a979ce/68747470733a2f2f636f6465636f762e696f2f67682f6c76682f6361657369756d2f6272616e63682f6d61737465722f67726170682f62616467652e737667
<S> = token*
<token> = b | i | u | text
b = <'*'> token* <'*'>
i = ...
u = ...
<text> = #'[^*_/]+'
@seylerius something like thatAlso see: #instaparse channel