Fork me on GitHub
#clojure
<
2018-05-02
>
theeternalpulse00:05:25

I'm making it a goal to stop going down the path of making generalized tools before I specifically need one

the2bears00:05:38

Rabbit holes all the way down, that's the story of my life πŸ™‚

πŸ‡ 20
WhoNeedszZz00:05:39

I've definitely been there

sophiago02:05:18

Is there a predicate for exceptions? e.g. (= (foo (/ 1 0)) false)

theeternalpulse02:05:02

slack needs a paren balancer πŸ™‚

πŸ˜‚ 8
theeternalpulse02:05:50

That looks like a job for macros

theeternalpulse02:05:59

not sure if there's an existing one though

sophiago02:05:48

I'm not even sure how I would implement a macro like this. My lack of Java knowledge is showing. It's essentially a pure version of try.

WhoNeedszZz02:05:07

Why not just use try?

theeternalpulse02:05:24

yeah, seems like you can even make foo as a macro that wraps try

sophiago02:05:52

I wasn't aware I could have try return a value rather than perform an effect

theeternalpulse02:05:28

the last statement in the body would return with a catch

πŸ‘† 4
noisesmith15:05:32

Gotcha here: the value in the finally clause is always run, whether you throw or not, and never returned

noisesmith15:05:39

Just use (try ~@body true (catch Exception _ false))

theeternalpulse02:05:21

(defmacro try-result [body]
  (try ~body
       (catch Exception e
         false)
       (finally
         true)))


(= (try-result (/ 1 0)) false)

metal 4
theeternalpulse02:05:29

might require some edge cases, but the base case works

WhoNeedszZz02:05:40

Does Slack not support Clojure highlighting?

theeternalpulse02:05:26

does that make sense @sophiago?

sophiago02:05:16

Yes, thank you. I was trying that outside a macro

theeternalpulse02:05:03

the problem is you want to delay evaluation, other than a macro, which doesn't require anything funky like eval and escaping list forms, there isn't a straightforward way to do that

theeternalpulse02:05:54

that is, delay evaluation of whatever may cause the exception

sophiago02:05:10

Yup, makes total sense now. I did actually try wrapping it in a delay before using try so that was at least the direction I was thinking

theeternalpulse02:05:00

I think delay you'd eventually have to run it through the try catch pattern

theeternalpulse02:05:56

(let [divide-by-zero (delay (/ 1 0))]
  (try (force divide-by-zero)
       (catch ArithmeticException e
         false)
       (finally true)))

theeternalpulse02:05:18

the problem with this is you won't be able to make a function that takes the expression as an argument

theeternalpulse02:05:28

(defn try-result-fn [expr-literal]
  (let [divide-by-zero (delay (eval expr-literal))]
    (try (force divide-by-zero)
         (catch ArithmeticException e
           false)
         (finally true))))

(try-result-fn `(/ 1 0))

theeternalpulse02:05:53

maybe that's not terrible, but not sure how evil eval is in the clojure community πŸ™‚

sophiago02:05:03

I have to run to a meeting, but your macro is always hitting the catch block in my use cases. I think due to the type of exception: (try? (apply concat [[1 2 3]])) => false

theeternalpulse03:05:59

my mistake, I misused finally as an "else" case. Rookie move 😞

eurythmia03:05:58

@theeternalpulse there are lots of java devs who do the same thing πŸ˜›

4
theeternalpulse03:05:16

(defmacro try-result [body]
  `(try
     ~body
     (catch Exception ~'e
       false)))

πŸŽ‰ 4
theeternalpulse03:05:00

this seems to work, I'm sadly lost when doing macros and using stuff like ~'

WhoNeedszZz03:05:20

I guess I'm not understanding what is ultimately trying to be accomplished that a standard try-catch-finally block can't accomplish

theeternalpulse04:05:27

I'd advise not do simple abstractions of something already simple like try-catch, it hides a lot if you make it too generic

πŸ‘† 8
emccue04:05:07

@sophiago The "pure" version of a try in other languages is a Result m***d

WhoNeedszZz04:05:13

Definitely goes against Clojure philosophy

emccue04:05:51

but a full one of that thing is usually overkill

emccue04:05:06

anyways, this is the macro i use

WhoNeedszZz04:05:42

I'm not seeing the point of that

WhoNeedszZz04:05:59

If it fails it will catch. If it didn't catch it didn't fail

emccue05:05:02

Im sure someone else can think of a better use, but my logic for parsing a message from kafka was basically just to convert it from json to a map

emccue05:05:27

but kafka doesnt like throwing an exception at that level

emccue05:05:34

so my function to parse the string from kafka was (fn [msg] (attempt (cheshire/parse-string msg)))

emccue05:05:46

It might be just self justification, but as long as something is a pure function and doesn't do something convoluted I think its pretty much in the safe zone for not violating the clojure philosophy.

emccue05:05:04

(feel free to yell at me; everyone

seancorfield05:05:42

@emccue I think it's reasonable to encapsulate exceptions sometimes, and that seems like a reasonable use case, but the unwrapping and rewrapping of the success/failure values can be more trouble than it's worth, if it pollutes upstream code. Definitely a case of evaluating the trade offs carefully.

sophiago06:05:12

Wow, my meeting went for over two hours so I'm glad other found the discussion I provoked useful πŸ˜›

sophiago06:05:07

For perpetuity: I never use try/catch, but I very frequently have a pattern where an argument to recur will fail right before the predicate triggers the base case and have never come up with a uniform way to handle that.

sophiago06:05:56

I usually can get by calling the predicate on next of whatever coll I'm processing, but that usually ends up with unnecessary let bindings to prevent code duplication (even though HotSpot will often inline them) and then occasionally I'm faced with situations like what prompted this where there's no analog to (empty? (next ..))

Bravi08:05:10

wrote this little article-like introduction to Clojure for my fellow FE team members at my workplace https://github.com/geganizharadzeatamido/clojure-wiki-for-amido/blob/master/README.md

athomasoriginal14:05:01

Thank you for sharing! I was in the same position as you - hardcore JS developer (front and back) building large scale applications. Why I switched is one of those things I am still exploring. This is why I thought this writeup was a good one. I do have a few comments which I hope are seen as a way to produce a more robust argument. RE: Ranting section Ecosystem + Tooling I don't feel this is a valid point. Both clj/s and js have challenges regarding ecosystem and tooling. However, at the least, if nothing else, JS has a greater amount of documentation which would let it win in this category. I do believe CLJ/S can have better tooling, but its not there yet. Not compared to JS. Side Note: this includes REPL and all the other nice things experienced developers talk about. RE: Core Library This is where CLJ/S shines the most in comparison to JS. The language is a better design and the core libaries...well, it takes 3 separate libraries in JS just to make something akin to spec and the these libraries are still not going to work together very well, if at all. So good point! RE: lein, boot and deps lein and boot are confusing. Sorry, I will take node scripts over these any day. Which is likely why I am moving toward clj and deps.edn. Perhaps mention these? RE: REPL + IDE's These are awesome, but difficult to grok for new developers. Especially given the JS feedback loop is so fast. This is NOT me saying one is better than the other, just that im not sure it would sway one way or another. RE: Clojure outside of web dev So can JS. Maybe provide an example of a place that JVM can go that JS cannot? Or maybe I misunderstood this point? RE: namespaces I would def talk about these more. Its another nice thing about clojure v. js that I really didn't notice until I was a few months in.

Bravi09:05:07

Hey, thanks for taking time for this reply! πŸ™‚ The most frustrating in JS tooling for me has always been getting to the point where I’m ready to work on my project. At my workplace, our basic starting kit is react + redux + jest + enzyme. And then we add things on top, depending on a project obviously. As project grows, there are so many tiny modules I need to add to make things work. I’ll give you an example. So some projects require us to use styled-components, some require sass or less. Now if I go down the styled-components route, then I can’t test my components anymore, unless I add jest-styled-components package. Yes, it’s just a one liner to install it and probably another example would be more valid where I’m also required to create a config file for it to work, but I think you’ll get my point. In Cljs I’ve never had this issue. Things just work, because there’s no magic being added behind the scenes. Maybe the projects I’ve worked on in Clojure haven’t been waaaay too complex, but I’ve built quite a few of them. And I felt that I was only ever adding things to my dependencies that I was actually using in my application, like a router, some middleware and etc. Now I’ve had situations in JS, where I upgraded a certain babel-blabla package, just because I needed some functionality from the newer version, and then suddenly another babel-blabla stopped working. Then I had to go through issues on Github and basically spend quite some time investigating this little detail that has nothing to do with my application. It’s just a part of a build process and it’s not something I’m importing in my application so to say. Whereas in ClojureScript, I’ve never had such problem using figwheel. I’m not sure about the REPL + IDE part you mentioned. I don’t quite understand what you meant there. But if it’s the speed you’re talking about, of showing changes in the browser, I find figwheel to be faster than webpack. The only complaint I would have in terms of IDEs is that I had to learn Spacemacs purely for Clojure. I know I could’ve chosen something else out there, but being a vim user seeing what Spacemacs had to offer, I couldn’t find a good alternative out there. I’m sure it would be easier for people using VS Code And finally, by Clojure outside of web dev, I actually wanted to say something like β€œDon’t think that you’re limited to just web dev, you can use it for anything”. It wasn’t a comparison to JS. The main reason why I wrote that was because this post is for my team members and previously I was showcasing Elm to them, which is purely for Front End. So I wanted to imply that there’s no limit like that in Clojure.

athomasoriginal13:05:12

No problem. Conversations like this are important! Your tooling system reflects my own. However, you gave an example of styled-components and needing another library to regain the ability to test. The counter I have to this is two fold. 1. while CLJS can make use of the JS ecosystem, it is non-trivial to get the same level of functionality working as fluidly in CLJS- ultimately, more than 1 line of code to get things working as expected and this is way beyond an early or beginner CLJS developer....because documentation is not as abundant as JS land. 2. The libraries are not always the most current Here is an example of what I mean. Only recently did I see a solid write up about using JSDOM as the test environment. But to do this, and really understand what this author was doing and why, you have to be no less than an experienced intermediate JS developer and it is much more than adding 1 line of code to get it working. Do you have to be intermediate to do it? No, but there are some nice extras which you will miss out on and will bite you if not configured up front. Could you use an existing solution? Point being, they both bring their challenges regarding tooling and library setup. This is seen when the CLJS application becomes large enough. You mention dependencies you are using, breaking your app. This is a problem in software in general. Rich Hickey actually just spoke about this in the new Joy Clark podcast. It happens in CLJ/S land too. It also not fair to compare figwheel to babel...they do different things and work for different goals. But I do appreciate that you may not have been comparing 1:1. The part about REPL is just to say that one thought against the need for REPL is when in a language like JS or GO you have rapid enough feedback cycles you really don't miss the REPL. Is this a valid point? Maybe not, but as someone who took advantage of this rapid feedback without using the REPL, and than moving to the REPL, its a task on its own to train yourself, because I had to do this, to use it. Figwheel being faster than webpack...they are both really fast πŸ™‚ Not sure what we gain from this point. Yeah, IDEs are tough. I struggled with it for a while also. True point about CLJ/S outside of web dev.

Bravi17:05:15

I think that REPL driven development or to be more specific, the fact that you can evaluate the code straight inside your editor, makes a huge impact on time you spend working on something. I only look at my code in browser when there are styling changes to be done. I tend to spend good y

Bravi17:05:28

* good few minutes in my editor before I check what I've written, in browser. I have even spent more than an hour and afterwards, when testing things in browser, it all just worked. Also, the fact that I can add / overwrite functionality or change state of my application from REPL, make it possible for me to do things like skipping a few pages of a multi page form that requires some data from the server. It feels like I can mock stuff LIVE as I'm testing things in my browser

Bravi17:05:30

I found this sort of development way more powerful than what devtools offer in JS ecosystem

athomasoriginal19:05:26

Agreed, but it takes some time and learning to switch over your workflow.

Bravi08:05:17

trying to convince them we should start using clojure in our projects πŸ˜„

pesterhazy09:05:23

@bravilogy the "Clojure for Java Programmers" videos could be useful: https://www.youtube.com/watch?v=P76Vbsk_3J0

tord09:05:38

@bravilogy: > Configuring and scaffolding your application can be a pain sometimes. A typical package.json file includes plugins that have nothing to do with the application I'm trying to build. There's usually at least 6 lines starting with babel- and another 6 - 7 lines of eslint-, not to mention enzyme- or jest-, in a typical package.json. And then there's a config file for jest, webpack, babel, polyfill, shim and etc. So I end up creating these random config files to actually get to the point where I can start working on my application. And I'm not even going to mention the version incompatibilities in some cases where one plugin doesn't like another. As much as I like Clojure(Script), the above paragraph perfectly describes how I feel about my project.clj files for any non-trivial Clojure project I write.

sophiago09:05:47

@bravilogy two pedantic notes: it's JVM bytecode and "code can be compiled at runtime" is an oxymoron

sophiago09:05:30

It's also just difficult to make statements like that about all Lisp dialects ever. Your intention behind that one isn't true for most.

sophiago09:05:02

The only reason I mention things like this are because I think the main turnoff to functional languages is the idea that they're purposefully obfuscated, when in reality misunderstandings just get passed on (not saying I haven't been guilty of that myself). You can draw an easy comparison with pointers: a shocking number of professional developers don't understand them, but few would claim they're some ivory tower academic concept. It's just a field that takes learning and the job market encourages shortcuts.

sophiago09:05:41

(by "field" I mean programming in general)

Bravi09:05:56

hmm I see what you mean @sophiago. should I remove those lines?

sophiago10:05:37

@bravilogy I do think those are very important things to mention. You can just substitute "JVM bytecode" for the first part. The second requires a bit more complicated explanation so I suppose I would rephrase it focusing on the benefits. Like basically describe what a REPL is and how compilation happens there too, but I would even find phrasing to ease them into the idea of working with a REPL.

sophiago10:05:24

I'm guessing you can draw on experience with something similar outside of Java. Python maybe? And then highlight the fact that code is actually compiled there and that should blow some minds.

Bravi10:05:52

yeah I actually think that I’m missing more about REPL

Bravi10:05:20

I just sort of mention it. And I think it’d be a good idea to perhaps show a few screenshots as well

Bravi10:05:15

I know for a fact that none of my team mates have any idea what REPL is

sophiago10:05:25

Yeah, I bet you can find a gif

Bravi10:05:45

easier to create it myself πŸ˜„

sophiago10:05:52

True πŸ™‚

Bravi10:05:56

thanks for suggestions

sophiago10:05:29

Np. Thanks for putting up with my pedantry πŸ™‚

πŸ˜„ 4
sobel10:05:06

I mostly came here for the pedantry

πŸ˜‚ 8
xtreak2911:05:05

Is it a good practice to include examples in docstrings for functions ? I see it's not used in Clojure core but I am not sure if there is a standard place to include example code for a function.

joelsanchez11:05:01

it's not a rare sight but bear in mind unit tests also serve as examples

xtreak2911:05:52

Yes, good point. But most of the users just use the jar and docstrings for docs and hence thought adding examples there will give them quick access.

abdullahibra12:05:35

is this very bad practice ?

johanmynhardt12:05:30

@abdullahibra please use preformat back-ticks. Your code is being bolded where you have an asterisk *

abdullahibra12:05:33

i'm in situation which need to make a recursion and save some state while make this recursion to the end.

abdullahibra12:05:06

so i thought i can use an atom for keeping this state while doing recursion, maybe i can use loop/recur

mpenet12:05:13

you likely want to use swap! instead, but you might want not to use an atom and use loop/recur

abdullahibra12:05:40

thanks πŸ™‚

abdullahibra12:05:51

so this is a bad practice right?

abdullahibra12:05:36

guys if i have a tree with multiple branches and want to find subtree of node A, how can i do this using loop recur

abdullahibra12:05:19

that's my current code

abdullahibra12:05:57

anybody there/

hlolli13:05:54

@abdullahibra not an expert on this, but I’d use zippers for traversing a tree like that instead of loop recur http://josf.info/blog/2014/03/21/getting-acquainted-with-clojure-zippers/

pesterhazy13:05:04

@abdullahibra first thing to try would be tree-seq

nathanmarz13:05:55

@abdullahibra that's easy with specter:

(def CHILDREN (path (srange-dynamic (fn [_] 1) count) ALL sequential?))

(def NODES
  (recursive-path [] p
    (continue-then-stay
      [CHILDREN p]
      )))

(select [NODES #(= 'A (first %))]
  '(R (H (h 1) (h2)) (M (N (A (a 11)) (B (b 5))))))
;; => [(A (a 11))]

nathanmarz13:05:33

can re-use the exact same code for transformation:

(setval [NODES #(= 'A (first %)) (before-index 1)]
  '(Z 0) '(R (H (h 1) (h2)) (M (N (A (a 11)) (B (b 5))))))
;; => (R (H (h 1) (h2)) (M (N (A (Z 0) (a 11)) (B (b 5)))))

nathanmarz13:05:31

and it will perform much faster than zippers

abdullahibra13:05:56

my recursive solution is ugly ?

nathanmarz13:05:36

the output has a bunch of spurious nils so it's not a solution

abdullahibra13:05:12

you are right πŸ™‚

abdullahibra13:05:28

specter is rescue here

Alex Miller (Clojure team)14:05:12

Recursive solutions are often easier to read/understand. The problem with recursive solutions is that you incur stack depth correlated to tree depth. Tree depth may be unbounded in your data, but stack depth is quite finite (which eventually means you break with a stackoverflow error).

michaellindon17:05:37

I have a recursive problem which actually seems very hard to do in a declarative/FP way. I'm wondering if there are any tricks for this kind of forward-backward recursion pattern. x_n is defined in terms of x_n+1 and y_n+1. y_n+1 is defined in terms of y_n. Every time I want to get x_n from x_n+1 I need to do the forward recursion from y_1 up to y_n+1. The imperative approach is to do the forward recursion and store it in a list, getting y_1 up to y_N. This list of y's then gets fed into the backward recursion which gets x_n from x_n+1 so get a list of x_1 up to x_N. Forming the initial list first seems very imperative. Is there a more functional declarative way?

gklijs17:05:09

Maybe you can use https://clojuredocs.org/clojure.core/declare it helped me getting a 'circular' dependency work.

noisesmith17:05:08

wait, how can you derive x_n from x_n+1 ? isn't that backwards?

lwhorton18:05:01

is there a cleaner way to handle this?

(every? (fn [k v] (empty? v)) (select-keys a [:name :type :value]))

lwhorton18:05:51

or put another way, is it possible to get the val of a key (not as a coll with vals) without destructuring or knowing the key ahaid of time?

noisesmith18:05:09

(every? empty? (map a [:name :type :value])

πŸŽ‰ 4
lwhorton18:05:26

i could do (comp empty? first vals) i guess ..

lwhorton18:05:30

oh yea that’s way nicer.

noisesmith18:05:14

alternatively (every? empty? ((juxt :name :type :value) a))

noisesmith18:05:28

but the map version reads nicer to me

andy.fingerhut18:05:32

@michaellindon Normal recursive function definition could be used here, and as long as they are pure functions, you can gain a lot of efficiency in some cases by memoizing the recursive function.

andy.fingerhut18:05:05

For example, this is a horribly inefficient way to write a function to calculate the n-th Fibonacci number, which takes exponential time if you do not memoize it: (defn fib [n] (if (< n 2) 1 (+ (fib (- n 1) (- n 2))))

andy.fingerhut18:05:26

But if you memoize it, it takes linear time in n.

andy.fingerhut18:05:33

The core.memoize library can help you in creating a memoized version of a function: https://github.com/clojure/core.memoize

andy.fingerhut18:05:53

Memoization only makes things faster if the function would be called multiple times with the same paramters, which is definitely the case for the slow-without-memoization version of fib I show above. In the function you want to calculate, it might be the case that x and y would only be called once for each value of n anyway, in which case memoization would not help speed things up.

ajs19:05:54

I have seen code in the wild that puts a function's return type hint before the name of the function (right after defn) and also right after the fn name, right before the arg list. Which is correct or are they the same? @alexmiller had suggested the latter a couple weeks ago, but the source for core functions often shows the former.

tbaldridge19:05:50

@ajs it's different depending on the situation. It's one way for deftype (I think on the arg list) and the other for defns

ajs19:05:04

We were discussing normal function definitions so perhaps it's confusing for everyone as he suggested the contrary

tbaldridge19:05:13

Hrm, I may have that backwards

tbaldridge19:05:36

But I know defn and deftype are opposite

ajs19:05:05

He referred to the difference between type hinting the symbol for the function's name, vs type hinting the return value. But I still see the former in core functions.

tbaldridge19:05:08

yes, from the deftype docs:

Method definitions take the form:
  (methodname [args*] body)
  The argument and return types can be hinted on the arg and
  methodname symbols. If not supplied, they will be inferred, so type
  hints should be reserved for disambiguation.

tbaldridge19:05:15

That's clojurescript, it might be different there

tbaldridge19:05:19

"let's just put the metadata everywhere, it can't hurt"

πŸ˜‚ 4
ajs19:05:16

I hope there is a reason for that beyond guesswork by its author

ajs19:05:52

Is it possible for a macro to include a type hint in its emitted syntax? It's hard to know since hints are stripped when things print to repl, like with macroexpand

Alex Miller (Clojure team)19:05:58

if you want to type-hint the return type, you should put before the arg vector, the end.

Alex Miller (Clojure team)19:05:35

yes, some core stuff does otherwise, that’s (mostly) historical

Alex Miller (Clojure team)19:05:30

@ajs yes, macros can return type hinted code, but doing so requires some care

Alex Miller (Clojure team)19:05:01

(source defmethod) is an example

eurythmia22:05:36

... I'm working with midje a lot lately, and one of the things that I find frustrating is that we can't assoc to Metaconstants. I was wondering if anyone could enlighten me about the rationale

tbaldridge22:05:02

@acripps Pretty simple, Midje is not Clojure, it's a testing DSL, and as such behaves differently than Clojure in many ways

tbaldridge23:05:00

But perhaps some examples would explain what's happening? "can't assoc into" is a bit vague, what's the error you're getting?

eurythmia23:05:39

@tbaldridge I got that πŸ˜› ... I've taken a look at the source and Metaconstant implements clojure.lang.Associative, but it explicitly throws on (assoc ...) ... I would personally find assoc on Metaconstants handy (more explicit than just passing in an empty map, or a pre-populated map for unit-testing functions). I am assuming that since the implementation explicitly prohibits this, there is a rationale, I just want to understand and find out if there's a more idiomatic way to unit test where a value that I want to pass in as a metaconstant would be assoc-d

tbaldridge23:05:04

yeah, looking at it now, it also throws on some cases of equality

tbaldridge23:05:15

It's one of the problems I have with Midje, the docs explain how to use Metaconstants, and I can read the code, but I don't know the connection between the two. Here's examples, and here's the implementation. Great, what's the rationale?

eurythmia23:05:33

exactly. I was thinking about fixing it myself, but when I looked at the code it looked to be done on purpose ... which is fine, but trying to track down the "why" is ... problematic.

tbaldridge23:05:40

From the code it looks like they are read-only. But I'm not sure why they didn't implement only read-only interfaces (ILookup vs Associative)

eurythmia23:05:25

that would have been even more explicit ...

eurythmia23:05:41

maybe that " (What would the name of the resulting metaconstant be?)" is the rationale ...

eurythmia23:05:06

I guess I'm going to have to sit and ponder this for a while ... try to figure out what the implications would be if there was a way to "extend" and/or "modify" a metaconstant

tbaldridge23:05:03

So they appear to the a form of nominal programming. ...foo... == ...foo... but ...foo... != ...bar... unless declared to be

eurythmia23:05:34

yeah ... what I really want here is to test that when I pass in a map (and I really don't care what it is, provided it contains key/value pairs that are required to operate on it), and check that the returned value contains the correct computed key/value pairs ... so "=contains=>" and (contains ...) are perfect, but the metaconstants themselves don't fully suit my needs here because I can't assoc onto them. I wonder if Mr. Marick would tell me to just use maps, or ... what?

tbaldridge23:05:21

Well, my advice would be to use spec and clojure.test

tbaldridge23:05:40

you can create a spec that says a map has to have certain keys, and what data types the values should be

tbaldridge23:05:48

and test.check will create data for you

eurythmia23:05:52

at this point we're already invested in using midje ... we evaluated spec, clojure.test, and midje, and to those of us uninitiated the latter seemed like the best choice at the time.

eurythmia23:05:34

... but I do appreciate the suggestion ... I'll take another look at spec next time I start on a personal project

qqq23:05:56

I'm a big fan of JCuda: 1. I write some *.cu files 2. I call launch-kernel from Clojure, passing it float-arrays / grid/block sizes 3. Hypothetically, even if the Cuda code creashes, the JVM is still fine. Is there something like JCuda, but minus the GPU. So it would be like this: 1. I write some *.c files 2. I call launch-c-kernel from Clojure, passing it float-arryas / ... 3. Said C code runs IN A DIFFERENT PROCESS 4. Even i said C code crashes, my Clojure/JVM is still fine. 5. Communication between Clojure/C is via copying int/float-arrays (willing to pay this overhead in exchange for C-code crashes not crashing JVM).

tbaldridge23:05:53

@qqq, that's fairly simple to do, just need to fire up a c compiler and mmap some memory

Alex Miller (Clojure team)23:05:36

You can use the JDK ProcessBuilder, or the JRuby guys (Charles Nutter in particular) have some ffi stuff for fork/exec, or there are probably some other options.

Alex Miller (Clojure team)23:05:00

with ProcessBuilder you can create stdin/out pipes

Alex Miller (Clojure team)23:05:31

if the amount of data you pass between is small, that might be ok

Alex Miller (Clojure team)23:05:40

prob not the newest take on that but the right ballpark

qqq23:05:20

I need to do this at 60fps (iClojure is passing some data to C, C caculates some new data, and Clojure uploads it back to GPU.)

qqq23:05:48

I should probably go for a long running C process with network i/o instead of forking a new C prog 60 times a second?

tbaldridge23:05:10

No, at 60fps you need mmap or a interprocess queue

qqq23:05:44

I'm familiar with mmap from C land. How does CLojure use mmap ?

tbaldridge23:05:43

but also look at https://github.com/real-logic/Aeron/wiki which is a high-speed 0 copy messaging system

Alex Miller (Clojure team)23:05:12

yeah that’s a whole different ball of wax so forget everything I said :)

tbaldridge23:05:25

but also ask, why? If you care that much about performance, you'll lose it in the communication unless you're super careful

qqq23:05:37

@alexmiller: sorry for dripping requirements bit by bit instead of stating everything upfront

tbaldridge23:05:38

and you haven't mentioned latency

the2bears23:05:47

Is 'C' using the data passed to it to calculate the new data? Then that's uploaded back to the GPU? Maybe the GPU could just manipulate the data itself, directly?

tbaldridge23:05:02

60fps with 100 frames of queuing is a lot different than real time πŸ˜„

tbaldridge23:05:36

Yeah cuda can mmap data from local memory (or the disk, which is mind blowing),

qqq23:05:58

Let me take a giant step back. I'm building a clj/cljs app that uses OpenGL/WebGL. My currentproblem with CLJS is that calculating the data for the vertex buffer objects is causing lag. On the CLJS side, I intend to have C code (running in Webassembly) to do the calculations. On the CLJ cide -- I could do the code in Cuda, but I'd prefer for both sides to share code. One solution around this mess is to do GPGPU, but vertex-transform-feedback requires WebGL2, and iPads only support webGL1, which means I can at most use "render to texture" if I want to do these computations on the GPU. Thus, I'm currently leaning towards doing these cals in C / webassembly.

qqq23:05:58

(as an aside: webcuda/webcl would solve my problems and make my life much nicer)

tbaldridge23:05:29

I'd look at the calculations code. This is a lot of work to get around something that shouldn't be a bottleneck. Remember, Minecraft was written in Java, runs on almost everything, and consistently renders about 512k cubes at 60fps.

tbaldridge23:05:58

And doesn't use GPU shaders

the2bears23:05:18

Yup, Minecraft was Java on top of LWJGL. Lightening in a bottle πŸ™‚

qqq23:05:25

I suspect Minecraft did not have code like this: (defn vec+ [lhs rhs] {:x (+ (:x lhs) (:x rhs)) :y (+ (:y lhs) (:y rhs))})

qqq23:05:47

Also, I believe most minecraft cubes are static. I'm not familiar with scenes where, say, 1000 cubes are having their locations being modified independently (one uniform for all 1000 doesn't count).

qqq23:05:43

Lastly, mine is a "text editor" of sorts, and people are much more tolerant of lag in fps than in wysiwyg editors.

tbaldridge23:05:29

that's probably your issue, to be honest, over use of GCs can kill performance. But rewriting that in JS/Java isn't a bad idea

tbaldridge23:05:20

Or even just switching to deftype vs maps helps a lot (I know that from personal experience)

tbaldridge23:05:48

We're getting a bit OT now, but handling 1000 objects each with their own transforms is exactly what GPUs are great for, can't you perform the transforms on the buffers during rendering, on the GPU instead of on the CPU?