Don't type into your REPL -- eval code from your editor 🙂

I used a plain Socket REPL for years with Atom/Chlorine and then VS Code/Clover and there was no REPL window to type into at all -- just an output panel. Now I start an nREPL server from the command-line -- again no input -- and connect to it from VS Code/Calva and I never open the REPL window at all. I use Portal for visualizing data (previously I used Reveal, before that Cognitect's REBL).

I agree that evaluating code from your editor is more convenient but in some cases I still use the console REPL. E.g. I sometimes work on multiple projects on the same day and want to switch to a project to test something quickly, I don't want another REPL windows in my editor since having many of those tends to get confusing. In such cases I start a plain old console REPL and use require + :reload while making changes in my editor. This even works with Notepad if you wanted to. This is also works for CLJS on Node.js for which I wouldn't even have an idea how to make it work with my editor without setting my hair on fire (I've tried years ago, but it was just too brittle in my experience). If typing into the console gets tedious you can just use load-file to load a snippet from a file. Console REPL isn't optimal but I'm happy it's there when I need it.

I use this when I need to: https://github.com/bhauman/rebel-readline

Despite my general position, I agree with both @U04V15CAJ and @U11EL3P9U that there are times when a quick'n'dirty console REPL is useful for quickly trying stuff out in the context of a library or a project, and I typically use Rebel Readline for that. My dot-clojure repo has a :rebel alias.

The main things I wish rebel had out-of-the-box (that I miss from lein's nrepl/reply setup) is interruptible evals, an init namespace, and pretty-printed results (see: https://github.com/greglook/whidbey). I've sort of cobbled together that feature set on top of it in one of my projects, so maybe I just need to standardize and release that. :thinking_face:

Fascinating, thanks! I use vim-fireplace for fast reloading namespaces, but not really evaluating anything interactively. I guess I’m pretty behind on IDEs in general. Maybe this is finally time to start coding in Emacs? Will check out the editors, thanks!

I think vim-fireplace and other vim plugins also provide the ability to evaluate forms via a REPL connection

I've heard good things about Conjure

I use Conjure. Tis the apoidea's modified hinge joint...

Follow-up: emacs + cider is actually generally pretty great. TODO: check how it works for CLJS.

If you’re using clj-http, you can use https://github.com/cjohansen/parrot

Just an attempt at it, without a careful consideration of potential corner cases:

(->> (tree-seq vector?
               (fn [[_tag maybe-attrs & children]]
                 (if (map? maybe-attrs)
                   children
                   (cons maybe-attrs children)))
               [:div {:class "x"} "y" [:span "z"] [:p 1]])
     (remove vector?)
     (map str))
=> ("y" "z" "1")

The most robust approach is probably to render Hiccup into an HTML string and then use it with something like jsoup to extract just the text from it.

echo '{}' > package.json
npm install nbb react-server html-to-text

$ npx nbb -e "(require '[reagent.core :as r] '[reagent.dom.server :as sr] '[\"html-to-text\" :refer [convert]]) (def html (convert (sr/render-to-static-markup [:p \"dude\" [:p \"hello\"]]))) html"

"dude\n\nhello\n\n"

Or if you like zippers:

#!/usr/bin/env bb

(require '[clojure.zip :as z])

(defn extract-strings
  [h]
  (let [result (atom [])]
    (loop [z (z/vector-zip h)]
      (if (z/end? z)
        (z/root z)
        (do
          (when (string? (z/node z))
            (swap! result conj (z/node z)))
          (recur (z/next z)))))
    @result))

(let [hiccup-vector [:html
                     [:head
                      [:title "Test"]]
                     [:body
                      [:h1 "Yay!"]
                      [:div {:class "foobar"}
                       [:p {:class "text-class"} "Imagine a long paragraph here..."]]]]]
  (println (extract-strings hiccup-vector)))

That's not entirely correct because AFAIK Hiccup also accepts other values and not just strings - exactly why my example has [:p 1] there.

that may end up in the wrong order perhaps, since z/next does depth first maybe?

Could be, could be. All valid concerns I admit 🙂

or it could be why it works correctly, haha

I think depth first is what provides the correct ordering, yes.

Maybe the test for (when (string? (z/node z)) …should be (when-not (map? (z/node z)) … instead, as I actually just intended to remove the attribute maps that way.

yes, I think you're right

> That’s not entirely correct because AFAIK Hiccup also accepts other values and not just strings - exactly why my example has [:p 1] there. Actually, the query was to extract the strings. So wouldn’t it be wrong to extract numbers 😛

> the query was to extract the strings "Text content". :)

In Hiccup 1 is text content in [:p 1].

map? is too broad, since it will also emit vectors and keywords

Ah, you’re right. 😂

Really?

Yep, I see.

probably you have to treat vector separately, and ignore the first keyword and optionally the second map? and then treat all the following non-collections as text nodes

which is pretty much what @U2FRKM4TW did

Would (when-not (or (coll? …) (keyword? …)) do it as well?

is [:p :foo :bar] valid hiccup?

maybe not

Hm… It could be. You know what, maybe the zippers just introduce more trouble than they are worth here 😄

But it was a fun diversion.

I don't konw how to close the nrepl client:sweat_smile:

In case you can't find any docs, you can try to backward engineer it with https://github.com/lambdaisland/nrepl-proxy. ::)

(quit)?

oh I thought docs about the nREPL protocol... sorry :)

Just ctrl-D or ctrl-C will work probably

Well, I’m not sure myself. 🙂

Here is the document for the nrepl protocol itself https://nrepl.org/nrepl/1.0/ops.html

@U0508956F thank you, I have been read this part, there have document about close a session, but I want to close the client. 😀

@U4ZDX466T I want to programmatically nrepl api, sorry, I forget say that. thanks you.

you can just send a close message and end the client?

@U04V15CAJ you mean the clojure.java.api source file document?

https://nrepl.org/nrepl/1.0/ops.html there is document about close a session, but have no document about close a client.

The messages described in https://nrepl.org/nrepl/1.0/ops.html are sent from the client to the server

so if a client is done, it should send a close message to the server, wait for the reply and then e.g. exit

I am talking to an nREPL endpoint programmatically, you mean I should send a close session message to the server? and how to exit?

that's it, just a close message.

got it, thank you very much~

Otherwise I’ll write a transform to convert between

(fn [[t theta thetadot]]
  [1.0 thetadot (* -9.8 (Math/sin theta))])

(fn [xs]
  #js [1.0 (aget xs 2) (* -9.8 (Math/sin (aget xs 1)))])

user=> ((fn [[a]] a) (doto (object-array 1) (aset 0 :a)))
:a
user=>

that is in clojure, dunno about clojurescript

❯ clj -A:cljs -M -m cljs.main -re node -r
ClojureScript 1.10.773
cljs.user=> ((fn [[a]] a) (doto (object-array 1) (aset 0 :a)))
:a
cljs.user=>

nice! is this slower perf-wise than aget?

i’ll test it out here too

it is nth

which is RT.nth, which does a check if it is an array and then does an aget

I suspect it will be slower and involve boxing and unboxing for arrays of primitives, but maybe pretty close to the same for arrays of objects

ah on jvm good call about boxing. maybe no issue in js though

@U0NCTKEV8 50x faster on the second one

(fn [[_t theta thetadot]]
  [1.0 thetadot (* -9.8 (Math/sin theta))])

(fn [^doubles arr]
  [1.0 (aget arr 2) (* -9.8 (Math/sin (aget arr 1)))])

Here is a related question: Why is this

(defn faster [^doubles arr]
  (let [t        (aget arr 0)
        theta    (aget arr 1)
        thetadot (aget arr 2)]
    [1.0 thetadot (* -9.8 (Math/sin theta))]))

(defn faster [^doubles arr]
  (let [t        (aget arr 0)
        theta    (aget arr 1)
        thetadot (aget arr 2)]
    (double-array [1.0 thetadot (* -9.8 (Math/sin theta))])))

aren’t you building a vector in both versions? The difference is that you take that vector and get a double-array from it after the vector is build?

@U11BV7MTK I had thought from… wait, yes, of course you’re right.

(defn faster [^doubles arr]
  (let [t        (aget arr 0)
        theta    (aget arr 1)
        thetadot (aget arr 2)
        result (double-array 3)]
    (aset result 0 1.0)
    (aset result 1 thetadot)
    (aset result 2 (* -9.8 (Math/sin theta)))))

use doto

;; 
(defn ->vec [^doubles arr]
  (let [t        (aget arr 0)
        theta    (aget arr 1)
        thetadot (aget arr 2)]
    [1.0 thetadot (* -9.8 (Math/sin theta))]))

(defn ->array [^doubles arr]
  (let [t        (aget arr 0)
        theta    (aget arr 1)
        thetadot (aget arr 2)]
    (doto (double-array 3)
      (aset 0 1.0)
      (aset 1 thetadot)
      (aset 2 (* -9.8 (Math/sin theta))))))

(let [xs (double-array [0 1 2])]
  (time (dotimes [_ 100000000]
          (->vec xs)))

  (time (dotimes [_ 100000000]
          (->array xs))))

still 2x faster to make a vector!

haha but then if I need to READ from the data later and include that in the benchmark (which I do for my animation case)

(defn vec-checker [arr]
  (let [result (->vec arr)]
    (= (result 1)
       (result 2))))

(defn array-checker [arr]
  (let [result ^doubles (->array arr)]
    (= (aget result 1)
       (aget result 2))))

(let [xs (double-array [0 1 2])]
  (time (dotimes [_ 100000000]
          (vec-checker xs)))

  (time (dotimes [_ 100000000]
          (array-checker xs))))

(each of these lines on the right is a physics simulation… these perf changes let 256 of these run at 60fps no sweat, vs my old lazy version)

double-array is overloaded, it can take a seqable of initial values or a size, and it has to do an instance check

and for the literal vector, I believe for a vector that small it is just allocating an array to hold the items

(doto ^doubles (make-array Double/TYPE  3)
      (aset 0 1.0)
      (aset 1 thetadot)
      (aset 2 (* -9.8 (Math/sin theta))))

hah

🙂

(was looking and forgot about make-array)

aset-double on the other hand is ultra slow; I guess because it’s coercing its argument, which I don’t need?

I dunno, I think that is very possible, the variable arity version of aset-double would also be slow

You can also set unchecked maths to shave a few ns more

Naive question: why send in an array and return an array? Can it be inlined so that you don't have to allocate a new array at all?

@UK0810AQ2 not naive, great question… in the JS version of the ODE solver, the interface just happens to want you to return an array. but I could send a patch to the library to supply a single output array and the function could just set the values. That would definitely be an improvement: https://github.com/littleredcomputer/odex-js#a-system-of-two-first-order-equations and then in the Java version, of COURSE now that I look more closely of course the code supplies an output array https://commons.apache.org/proper/commons-math/javadocs/api-3.6.1/org/apache/commons/math3/ode/FirstOrderDifferentialEquations.html

and I had been calling arraycopy https://github.com/sicmutils/sicmutils/blob/sritchie%2Ftidying/src/sicmutils/numerical/ode.cljc#L106-L115

well, looks like there are some monster performance wins on the table here. Thanks all for talking me through this!!

Also change out swap inc with an atomic integer, should be friendlier to the CPU

And flatten is evil

100%

The original code is written with these nested vector like objects, so my first pass a year ago did these dirty back and forth swaps

But now my compilation code is better, so I’ll add a new mode for the ODE-solver-only version

And then if the user specifically needs a callback with the immutable nested version, great

@U017QJZ9M7W - tmdjs had nice wrappers around the typed arrays. See the api docs for tech.v3.datatype in tmdjs

They include nth support , reduction, and fast sub-buffer and such.

Nice!!

A lot of your examples that use clojure.core/double-array -- ham-fisted.api/double-array is faster. The seq creation is unnecessary and a slower form of access than a reduction maintaining the index.

In general creating a ton of small arrays or small vectors is a I think bad design paradigm performance wise. You should be creating one 2D tensor and read/write to it's rows. Then you aren't creating a lot of garbage every frame.

OR just one or two double arrays and write to offsets within the array. You have a fixed number of particles...

Yes that is the change I’ll make here, and it should always work

Hmm - in my tests, reading from large double array and writing to another isn't a ton faster although in clojurescript it is. The JVM really is pretty amazing. Parallelizing the computation makes the no-gc pathway shine but serially it performs roughly the same so it depends on how loaded you intend to make the machine. Vectorizing it with jdk-17+ vecops would make a difference and that is where your compiler could really shine.

@UDRJMEFSN super weird, it looks like in cljs it is way faster to return #js [..] arrays in my tests than it is to recycle a single output array with aset

Did you try using typed arrays in js? I thought #js created generic object arrays.

https://github.com/cnuernber/tmdjs/blob/master/src/tech/v3/datatype/arrays.cljs

the answer is complicated

clojure does actually support shebang lines as comments at the top like that for scripts, but basically no one uses them, and there isn't really good support for them anywhere

I suspect the docs mean for you to do something like

${whatever you do to run clojure programs} -vvvp8080 foo --help --invalid-opt

since you might uses lein run or clj, or whatever

when you use clojure as a shebang script it doesn't call a main, it just loads the code given, so you need to have toplevel side effects if you want to ever do anything

you can also do some wild stuff to have a shell script that is also valid clojure

https://gist.github.com/hiredman/a8fb63ec64704ecb967f is a script I used to use to delete old database backups off s3

a shebang like that only works on osx, because different unix like systems handle arguments in shebangs differently

Alright, I guess this is what babashka is for

Thank you for the detailed answers and ideas

If you have a strong need for a native executable, you could compile with GraalVM. But if you are just wanting to use Clojure syntax for scripting, then Babashka is definitely the way to go.

to answer my own question, we can create a ./my-program file with this I guess:

#!/bin/sh
clojure -M -m my.program "$@"

I’m making liberal use of clojure libs and I’m not sure if it’ll work with babashka, but I haven’t used it much yet

I have another gist for you https://gist.github.com/hiredman/7b13eb5dcc4fe0ae03ef91d62f66ec2d

(thanks for the feedback that it was confusing as written!)

great, thanks @U04V70XH6!

https://gist.github.com/ericnormand/6bb4562c4bc578ef223182e3bb1e72c5

There's also this, which allows you to make a shebang script that does a clojure shell script (which like mentioned before must have top-level side effects) and which can include dependencies, pass command line arguments, etc.