Fork me on GitHub
#clojure
<
2018-02-10
>
metacritical00:02:10

Anyone looked at implementing a scheme using Graalvm ? or maybe porting clojure to it?

andy.fingerhut02:02:02

@metacritical You can search through the Clojure Google group for "graal" and find a very few mentions of it in messages there, mostly from 3+ years ago. I don't know any more than that, but if you haven't done that, it may give you a lead or two (perhaps to a cold trail)

metacritical02:02:17

@andy.fingerhut Ohh, it would be interesting to see clojure run on graal, atleast the startup characteristic would improve.

qqq05:02:24

Does it make sense to talk about a JVM that "runs on a cluster, presenting the entire cluster as a single machine", or is this smoething that does not exist for tech_reason_xyz ?

qqq05:02:08

In particular, I have a local network of four machines, I want to control them all via my clojure program, and am wondering if I could somehow treat them as one JVM (they're all on local network, connected by 1Gbit ethernet), or I have to treat them as four separate JVMs

andy.fingerhut05:02:48

It definitely makes sense if you have multiple CPUs physically connected to the same CPU<->memory interconnect, like PCI express

andy.fingerhut05:02:20

And all running a common OS that manages those multiple CPU cores, with common memory address space between them.

andy.fingerhut05:02:19

I have heard of PCI express-over-Ethernet tunneling, but I do not know if that can be used to make 4 CPUs run a common OS, though. I think that is more commonly used to access PCI slave devices like a GPU over an Ethernet from a CPU.

andy.fingerhut05:02:15

I have never heard of a single JVM running across CPUs connected via Ethernet.

gonewest81805:02:29

Well, going back a few years there were commodity Linux clusters like Beowulf for supercomputing type problems. But you said Clojure and JVM... take a look at Onyx, Apache Ignite, or even Apache Mesos? Three different approaches to interacting with a cluster.

andy.fingerhut05:02:04

It is the "single JVM process" part that sounds difficult. Running separate JVMs on each CPU+OS instance is straightforward.

qqq06:02:19

Maybe 'cluster' is not the right word, it's four machines connected via gigabit ethernet

qqq06:02:28

right now, I have cider + connected to the jvm via hnrepl

qqq06:02:37

but I don't want four separate nrepls

hiredman06:02:29

cliff click was working with a startup that was modeling a jvm on top of a cluster a few years ago

Alex Miller (Clojure team)13:02:37

I ran about half the dev team at Terracotta. I first used Clojure trying to help Paul fix some bugs he found building that. :)

qqq06:02:49

Current mindset: * four separate machines ==> have to use four separate JVMs * maybe one nrepl, maybe four nrepls * how to control all this via cider ?

gklijs07:02:52

Not sure if it's overkill, but you could use something like redis to coordinate between instances. So when you enter something in the repl on one, the other read the value from redis, and give the output back. You could also use RMI, or a shared disk or something.

andy.fingerhut07:02:58

I don't use cider, and you would want to ask in a cider-specific channel if there is one, but several Github issues for cider mention the ability to (and sometimes issues with) creating multiple nrepl sessions from cider, e.g.: https://github.com/clojure-emacs/cider/issues/1122

andy.fingerhut07:02:20

The discussion there makes it look like it should normally work to create multiple nrepl sessions.

andy.fingerhut07:02:38

nrepl typically uses TCP for transport, which is point-to-point communication, so you would need multiple separate TCP connections from your dev machine to each JVM process.

hiredman08:02:21

https://github.com/hiredman/crepl/blob/master/src/com/manigfeald/crepl.clj is an unfinished thing I was working on a while ago. it does something not entirely unlikely swim to do gossip based cluster membership, and adds the ability to proxy your nrepl session to another nrepl session

qqq08:02:09

I failed to explain my problem: the problem is NOT how do I get cider to connect to 4 different repls; the issue is how it breks my work flow, if things ilke C-x C-e now becomes C-x C-e {1, 2, 3, 4} and I have to keep track of which repl to send each command

qqq08:02:27

on one hand, it's very weird, on the othedr hand, if I have four different repls, it's not clear how I can get around specifying which nrepl to talk to

dominicm09:02:07

Fireplace scopes a connection to a directory.

qqq08:02:46

(and this issue isn't cider specific -- if I have 4 nrepls, regarldess of what editoride I use, I have to specify which nrepl it talks to)

robert-stuttaford10:02:26

just got a full-stack clj/cljs project set up with clj and deps.edn (and figwheel-sidecar). really impressed - fully running nrepl and figwheel service running in about 15 seconds! (i have datomic and jetty on the classpath, which likely contributes to a chunk of that time)

dominicm10:02:12

I wasn't surprised to find that our version of that still took 1m30s πŸ˜‚

robert-stuttaford11:02:23

ahh, the ole monolith πŸ™‚

magra11:02:54

Hi, I startet an app with luminus. Until now I did everything server side. Now I want to add cljs and something reactive. I added figwheel and sidecar to :project/dev in project.clj ` :project/dev {:dependencies [[prone "1.5.0"] [ring/ring-mock "0.3.2"] [ring/ring-devel "1.6.3"] [pjstadig/humane-test-output "0.8.3"] [figwheel-sidecar "0.5.14"] [com.cemerick/piggieback "0.2.2"]] :plugins [[com.jakemccrary/lein-test-refresh "0.19.0"]] :source-paths ["env/dev/clj"] :resource-paths ["env/dev/resources"] :repl-options {:init-ns user :nrepl-middleware [cemerick.piggieback/wrap-cljs-repl]} :injections [(require 'pjstadig.humane-test-output) (pjstadig.humane-test-output/activate!)]}` Now I get this error: Error loading cemerick.piggieback: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;)V, compiling:(closure.clj:100:1) Exception in thread "main" java.lang.RuntimeException: Unable to resolve var: cemerick.piggieback/wrap-cljs-repl in this context, compiling:(/tmp/form-init6908219750721856608.clj:1:8532) at clojure.lang.Compiler.analyzeSeq(Compiler.java:7010) How do I go about hunting that down?

magra11:02:51

I tried different versions of piggiback.

tstelzer12:02:35

@magra for multi-line snippets you need to use `

tstelzer12:02:45

or the snippet feature πŸ™‚

magra12:02:33

yeah... formatting is not exactly what I intended πŸ˜‰

schmee13:02:59

what’s the maximum number of elements in a Clojure hash map?

gklijs13:02:41

even if the keys have the same hash, they get added

schmee13:02:48

nice, thanks for that link!

schmee13:02:18

but if my understanding is correct, the tree that backs the map never gets deeper than 7 levels, right?

schmee13:02:57

since the hashcode is partitioned into blocks of 5?

qqq15:02:10

@nikki: I am reading https://clojurians-log.clojureverse.org/clojure/2016-10-18.html -- do you have regl + cljs bindings for live coding ?

bhauman17:02:18

OK I've opened my new readline library and deployed snapshot jars.

bhauman17:02:03

I would absolutely love it if folks could kick the tires and see if it works in their terminal of choice.

noisesmith17:02:29

that looks amazing!

noisesmith17:02:33

I'm going to try it today

noisesmith17:02:45

this is great, so far

noisesmith17:02:52

(just using lein trampoline run for now)

noisesmith17:02:15

I guess expecting it to respect my readline config that asks for vi mode keys would have been too much haha

noisesmith17:02:42

(this is gnome-terminal, with tmux)

bhauman17:02:36

@noisesmith that might be something that is already in JLine3. I haven't explored it yet.

noisesmith17:02:00

cool - with any luck it might just be something you can plug in

bhauman17:02:21

also I made vi mode available

bhauman17:02:44

but I can't tell if its working well, I did my best as a non vi user

noisesmith17:02:59

I'll be sure to stress test it if I can figure out how to turn it on haha

bhauman17:02:33

:repl/help

bhauman17:02:51

:repl/set-key-map :viins

noisesmith17:02:22

oh this works great

noisesmith17:02:32

so the only thing left is respecting inputrc I guess?

bhauman17:02:01

yeah in a cross-platform way

jsa-aerial17:02:26

Seems pretty neat - but what is the expected use case for this?

jsa-aerial17:02:17

Also, I tried setting the color to dark screen but nothing happened - no error, no change. On gnome terminal on ubuntu

bhauman17:02:01

hmmm it does seem there is no feedback

bhauman17:02:07

but the colors did change

jsa-aerial17:02:15

Well, if the color changed for you, that is different from what happened for me. No change, no error.

noisesmith17:02:22

cool! vi mode, d% will actually delete a multi line balanced form, as expected

noisesmith17:02:17

@bhauman for the themes, that should be changing colors of foreground colors for things typed after that input, and not previous output, right?

bhauman17:02:45

it will not change the terminal background color

noisesmith17:02:47

because if so, it works - @jsa-aerial just making sure that you are testing the right thing - it shouldn't change the scrollback

noisesmith17:02:59

@jsa-aerial I am also in gnome-terminal in ubuntu, but inside tmux

cjsauer17:02:04

How "heavy" is creating a watch on a ref type (atom, ref, etc) in terms of time/space? Is anyone aware of a benchmark for this?

bhauman17:02:16

@noisesmith also there is an issue here https://github.com/bhauman/rebel-readline/issues/70 where I tried to provide visual feedback for vi modes, but its super sensitive to terminal control chars in insert mode

noisesmith17:02:06

real working multi line text objects in a repl is awesome btw

bronsa17:02:09

you should try chez scheme :)

bronsa17:02:27

it has that and only takes 0.1s to start

jsa-aerial17:02:35

Yeah, for whatever reason, after I set to dark screen nothing changes - input is still same color, known tokens (map, range, etc), same color, scroll back lines are same color. Β―\(ツ)/Β―

noisesmith17:02:31

it starts as dark, and then light is the alternative

bhauman17:02:49

type :repl/set-color-theme :light-screen-theme and the type :repl/set-color-theme :dark-screen-theme repeatedly

bhauman17:02:11

well I can't upload that screenshot of expected behavior

noisesmith17:02:40

looks fine from here

noisesmith17:02:26

OK - some of the vi text objects (eg. %) work very nicely, others act weird or don't work at all

jsa-aerial17:02:58

It starts as dark? For me it doesn't actually change my background color (terminal) which is lighter (kind of beige like). OK, setting to light does change things. I think the issue was misreading the meaning of 'light' and 'dark' in this context. I was taking it as 'set the color to dark/light' not 'set colors according to having a light/dark terminal background

noisesmith17:02:09

it never changes the background

noisesmith17:02:14

the themes change foregrounds

jsa-aerial17:02:36

Exactly - it changes foreground to fit a light or dark terminal

bhauman17:02:37

yeah better wording around that is probably necessary

jsa-aerial17:02:11

Wow, that color change helps a lot!

bhauman17:02:29

@noisesmith if you audit the vi stuff please takes notes in an issue, I can at least disable some of the wonky features until someone implements them properly

noisesmith17:02:02

nothing is outright terrible compared to other vi bindings - just "oh that thing isn't really there"

bhauman17:02:16

oh well thats good to hear

noisesmith17:02:13

I'm going to start making some compatibility notes - biggest one so far is that d only destroys, it doesn't put the destroyed thing into the paste buffer as I'd expect (but y / p do copy / paste)

jsa-aerial17:02:32

In the help output, might be nice to indicate the defaults

bhauman17:02:48

thanks for taking the time to try it out, I'm heading to lunch

schmee17:02:38

ooooohh shiet, I’m going to try it out right now! πŸ˜„

noisesmith17:02:18

this looks like it's going to make the frontend person on my team (who spends a lot of time in her figwheel repl) quite happy

rauh18:02:56

Did my first short screencast about (yet another) tiny spy macro: https://youtu.be/38BTviEdi64

qqq22:02:35

clojure.lang.IFn -- applyTo (clojure.lang.ISeq arglist) <-- how do I convert this arg to an Object [] ?

noisesmith22:02:36

probably object-array

qqq22:02:42

context: I'm writing, in JVM bytecode, a function that implements interface cllojure.lang.IFn

noisesmith22:02:00

if you are already working on a bytecode level, going up to call something like object-array from in there seems a bit odd - maybe I don't understand your task though

justinlee22:02:06

I don’t know what @qqq has been up to over the last few weeks, but whatever it is, it seems very interesting

justinlee22:02:41

i keep waiting for him to start asking about rocket telemetry or genome encoding using the jvm on a cluster πŸ˜›

noisesmith22:02:32

"leinencoin ICO"

qqq22:02:56

Let me rephrase the problem: I want to define a macro asm-fn so I can write things like:

((asm-fn [a :float, x :float-array, y :float-array]
  .. insn bytecode that does ...
  .. pertend this next section is written using insn bytecode ...
  (doseq [i (range n)]
    (aset y i (+ (aget y i) (* a (aget x i)))))
  ...) 2.0 (float-array [1 2 3]) (float-array [2 3 4])))

qqq22:02:43

so right now, I have to do: (.go (asm-fn ....) 2.0 (float-array [1 2 3]) (float-array [2 3 4]))) and I want to extend clojure.lang.Ifn to get rid of the .go

noisesmith22:02:09

depending on why you are in the asm-fn level, you might want to do a more streamlined ISeq -> array of object transform - but maybe the calling overhead is negligible compared to the work inside the function itself so you don't care

qqq22:02:47

it's a bunch of numerical / computer vision algorithms, for which type hinting in Clojure has not worked well for me, so it's reached the 'fuck it, I'll write this in asm' level

qqq22:02:02

(defn my-wrap [c]
  (fn [& args]
    (.go c (object-array args))))
I guess I could do that, and then all my functions can take an Object [] as the args.

qqq22:02:58

(defn bc->obj [ret code]
          (let [co         (ic/visit {:name    "my.dyn.TestClass"
                                      :fields  []
                                      :methods [{:name  "go", :desc [[Object] ret] , :emit code,
                                                 :flags #{:public }}]})
                obj (ic/new-instance co)]
            (fn [& args]
              (.go obj (object-array args)))))
        
        ((bc->obj Object
                  [[:aload 1]
                   [:ldc 0]
                   [:aaload]
                   (comment
                     ;; how do I cast this to an integer?
                     ;; already tried [:chekcast Integer]
                     [:ldc 22]
                     [:iadd] ;; then we can add 22
                     [:ireturn])
                   [:areturn]])
         20)
this is almost working; I can return the object, but I can't cast it to an integer yet

bronsa22:02:36

you want an int not an Integer

bronsa22:02:55

you can't areturn a primitive

bronsa22:02:08

or ireturn when the signature expects an Object

bronsa22:02:18

and you cant iadd an int and an Integer

qqq22:02:55

@bronsa:

((bc->obj int
                  [[:aload 1]
                   [:ldc 0]
                   [:aaload]
                   [:checkcast int] 
                   [:ireturn]])
         (int 20))

        (class (int 20))                ;; java.lang.Integer
        
is not working yet

qqq23:02:02

how do I create an int instead of an java.lang.Integer ?

bronsa23:02:05

you can't cast an Integer to an int

bronsa23:02:13

checkcast operates on objects

bronsa23:02:17

ints are primitives

bronsa23:02:26

@qqq you unvoke intValue

bronsa23:02:39

from the Integer

bronsa23:02:45

or you operate directly on primitive ints

bronsa23:02:57

I suggest you read a bit more on the jvm data types before attempting to write bytecode

qqq23:02:06

okay, I found sample code for that; let me try that

qqq23:02:40

or I can learn to swim by jumping into the ocean, and ask here for help when I see sharks

qqq23:02:52

πŸ™‚

bronsa23:02:16

sure, but you'll never learn to be self sufficient that way

bronsa23:02:40

you can go just so far by trying stuff and asking for help

bronsa23:02:43

documentation is there for a reason

qqq23:02:59

((bc->obj int
                  [[:aload 1]
                   [:ldc 0]
                   [:aaload] 
                   [:checkcast Integer]
                   [:invokevirtual Integer "intValue" [:int]] 
                   [:ldc 22]
                   [:iadd]
                   [:ireturn]])
         (int 20))
woot, it works!

bronsa23:02:02

that's doing way more work then a simple (+ 20 22) FYI

qqq23:02:09

@bronsa: I acknowledge I may be in te wrong here, but in practice, I find dry documentation much easier to digest after I get a few working examples. Until I get a few working examples, there's no "skeleton" to attach the knowledge to, and everything just goes over my head.

qqq23:02:45

Yeah, but input args are only cast once, so it's not . big deal.

bronsa23:02:51

you've got 3 conversions for that input argument

bronsa23:02:58

long to int, int to Integer, Integer to int

qqq23:02:48

Here's my full code (with bc->obj serving as a function to test out all future isntrs); how would you optimzie stuff away

(defn bc->obj [ret code]
          (let [co         (ic/visit {:name    "my.dyn.TestClass"
                                      :fields  []
                                      :methods [{:name  "go", :desc [[Object] ret] , :emit code,
                                                 :flags #{:public }}]})
                obj (ic/new-instance co)]
            (fn [& args]
              (.go obj (object-array args)))))
        
        ((bc->obj int
                  [[:aload 1]
                   [:ldc 0]
                   [:aaload] 
                   [:checkcast Integer]
                   [:invokevirtual Integer "intValue" [:int]] 
                   [:ldc 22]
                   [:iadd]
                   [:ireturn]])
         (int 20))

bronsa23:02:59

I would write it in java

qqq23:02:24

Are you suggesting:

((bc->obj int
                  [[:aload 1]
                   [:ldc 0]
                   [:aaload] 
                   [:checkcast Long]
                   [:invokevirtual Long "intValue" [:int]] 
                   [:ldc 22]
                   [:iadd]
                   [:ireturn]])
         20)
?

qqq23:02:28

Why would you write it in Java ?

bronsa23:02:42

because it'd be shorter and easier

qqq23:02:30

That boat has sailed; jvm bytecode is quite elegant and worth learning.

bronsa23:02:31

and harder to mess up if you don't fully understand how JVM bytecode works

bronsa23:02:41

ok, your time, your choice

qqq23:02:29

insn = all the raw performance of jvm bytecode + as a "macro language", all the power of clojure ... it's just not clear what Java has to offer over this

bronsa23:02:59

and C's __asm has all the power of raw assembly + all the power of the C preprocessor so why would you use C over __asm

qqq23:02:25

If C's preprocessor was as nice as Clojure fn's, I'd probably still be using C.

bronsa23:02:36

I was making a parallel

qqq23:02:49

I'm pointing out that C's macro system realy isn't all that nice.

qqq23:02:48

Anyway, let's not argue over this.

joelsanchez23:02:48

qqq is having fun with this and java is not fun at all

qqq23:02:58

I really appreciate all the help you've provided these past few days.

bronsa23:02:09

that's not what I was saying, I was pointing out how using a low level language + a macro system doesn't make much sense when you can achieve the same in a higher level language which is just a dsl for the lower level one

qqq23:02:34

In fact, @bronsa, I think it was you who suggested insn when I first wanted to do "clojure dsl -> *.java -> load class" πŸ™‚

bronsa23:02:52

no, I suggested insn when you asked about ways to write jvm bytecode

bronsa23:02:05

I never said I'd write jvm bytecode directly if I had the choice of writing java instead

bronsa23:02:32

@joelsanchez sure and he can have all the fun he wants, but if his goal is being productive, this is just not the way to go

bronsa23:02:24

there's good use cases for reaching for writing jvm bytecode, writing performant arithmetic in clojure is not one I'd consider such

qqq23:02:01

@bronsa: do you believe that *.java would generate faster numerical code?

dpsutton23:02:07

@bronsa any chance you're going to be in new orleans next week for clojure sync?

bronsa23:02:34

@qqq java is essentialy a stupid dsl over jvm bytecode, there's no way that you can write faster jvm bytecode than what javac would emit for an equivalent java program

bronsa23:02:42

modulo emitting bytecode that's not expressible in java

bronsa23:02:06

@dpsutton as much as I'd love to, no unfortunately

bronsa23:02:18

that conference looks sick

dpsutton23:02:40

yeah it seems like its gonna be great. wanted to buy you a beer/tea/coffee if you were gonna be in town

bronsa23:02:00

ha, thanks! some other time :)

bronsa23:02:42

I honestly don't think I'll be to the US for a conference if I'm not speaking (I could make an exception for strangeloop), we'll see tho

qqq23:02:13

@bronsa: suppose you were writing num-clj, the clj equiv of numpy, would you write the primitives in java instead of having clj gen bytecode ? the later seems like it would be lower barrier to add new 'primitives'

dpsutton23:02:03

where are you based?

bronsa23:02:13

I don't undertsand what people have against writing class Util { static int add (int a, int b) { return a + b} } if all they want is integer addition

qqq23:02:15

@bronsa: but I want to do more than addition; I want to implement things ilke J-style loopless code: http://www.jsoftware.com/help/jforc/loopless_code_i_verbs_have_r.htm#_Toc191734331

qqq23:02:19

insn gives me the power to build my own abstractions on top of jvm bytecode, using the full flexibility of Clojure -- why would I want the pre-defined language 'Java' instead ?

bronsa23:02:19

@qqq I can't make a decision for your hypotetical scenario, I don't know what you are trynig to do or why you need this -- all I'm saying is that writing bytecode is not pleasant and it's easy to get wrong, while writing java is trivial. if you can do something in either way, I'd pick java over writing bytecode every day

bronsa23:02:45

there's a reason why people prefer higher level languages over lower level ones, they're easier to reason about and to write non-trivial code in

bronsa23:02:04

@qqq you can still use clojure and write the bits that you can't write in clojure in java

bronsa23:02:09

I'm not saying write everything in java

qqq23:02:42

Part of the problem may be infatuation with the idea of my clojure code can generate jvm bytecode on the fly and then I can execute that function perhaps after I shoot myself in the foot a few times, I'll come to appreciate your view πŸ™‚

bronsa23:02:53

all I'm doing is giving you pragmatic advice

bronsa23:02:56

I've been down that road

bronsa23:02:30

it's fun, but don't lie to yourself thinking that it's more productive than writing a boring java util class