It depends on what you want

Oz and Hanami are a little more than just wrappers. They do a few more things. If you are looking for something that's even lighter, then consider https://github.com/applied-science/darkstar

If I remember right, Oz actually uses it

I've used Oz more than Hanami. Simple and elegant.

Babashka with Vega/Lite is another option

Ah, nice. Acually might fit the bill. It's for a coding challenge, so bb might be even better suited

Thanks for the info @U8VE0UBBR

We use Apache Echarts, it is excellent: https://echarts.apache.org/en/index.html

We have single namespace wrapper that we'll oss as a mini project in the next couple of weeks. Doesn't do much other than make echarts a bit simpler to use in cljs.

depending on the use-case, https://github.com/nextjournal/clerk might be a good fit too - it lets you mix jvm/js clojure code, and supports a few visualization libs (plotly, vega lite) out of the box - the https://github.clerk.garden/nextjournal/book-of-clerk/commit/d74362039690a4505f15a61112cab7da0615e2b8/ has good coverage and is itself a clojure file rendered by clerk (https://github.com/nextjournal/clerk/blob/main/book.clj)

clj-kondo is able to output the dependencies between namespaces: https://github.com/clj-kondo/clj-kondo/blob/master/analysis/README.md Some tools built on this are vizns and morpheus (as seen here: https://github.com/clojure/tools.deps.alpha/wiki/Tools#deps-management) You could probably also use tools.namespace for this.

https://github.com/clojure/tools.deps.graph

Oh sorry, you are talking about ns deps, my bad

@U47G49KHQ yes, tools.deps.graph looks useful, but it's only for (external) maven deps - I'm looking for (internal) ns deps

@U04V15CAJ whoa that was easy!

All I needed was

clojure -M:clj-kondo-tools -m clj-kondo.tools.namespace-graph src

I'm so glad I asked here 🙂

Thanks :)

What about millions?

part of the point of go block concurrency is that thread count is not a directly limiting factor in concurrency. At millions you may see memory limitations as you store continuations or perhaps latency issues as the threads those millions of execution threads have to map to swap between them. If most of those threads are parked most of the time you may not see too much performance degradation, you just have to be careful with memory.

thanks!

@U4FFD43T4 But also... don't bother if you can use Java 19

Because virtual threads?

ye

I haven't seen much written on how they compare. Are virtual threads hands down better?

Notable is that virtual threads have identical restrictions/limitations as I said above for go threads. Virtual threads are basically less-limited go threads.

> Are virtual threads hands down better? Pretty much. go blocks do what virtual threads do just in "user space" - so your stack traces will start to suck

and you need to take special care around stuff that blocks

Virtual threads are hands-down better because of the type of concurrency they provide, the ability of the JVM to optimize them, and the fact that all IO that uses standard java features doesn't block the OS thread, and that stacktraces and debuggers can handle them normally.

idk if core.async's chan needs another look or if there is a better potential implementation now

but (one sec)

Quite possibly, but it does use standard JVM locking and monitor stuff, which should work fine with virtual threads and not pin carrier threads.

This is all great to know. I'll play with virtual threads. Thanks

(require '[clojure.core.async :refer [chan >!! <!!]])

(defmacro scram 
  [& body]
  `(let [chan# (chan)]
     (Thread/startVirtualThread 
       (fn []
         ~body))
     chan#))

https://git.sr.ht/~srasu/spindle

401

Oh, I need to make it public

err, one set

It's not battle-tested yet, but this is a monkeypatch that you can require before you require any libraries that use core.async and it should make them use virtual threads by default.

the link should work now

(defmacro scram 
  [& body]
  `(let [chan# (chan)]
     (Thread/startVirtualThread 
       (fn []
         (>!! chan# (do ~@body))))
     chan#))
=> #'user/scram
(scram 3)
=>
#object[clojure.core.async.impl.channels.ManyToManyChannel
        0x7dbe3ce6
        "clojure.core.async.impl.channels.ManyToManyChannel@7dbe3ce6"]
(<!! *1)
=> 3

here you go

you should be able to replace every "go" with "scram"

Also TIL sourcehut

Also https://git.sr.ht/~srasu/plet this is also not battle-tested but is about the structured concurrency incubator

hmm looking at what joshua wrote maybe also want this

sourcehut is really nice. I really like the email-based workflow because it allows you to manipulate history of proposed changes right up till they get merged.

nvm binding conveyor fn isn't public

plet here is for examples like:

(plet {val-a (alts [(http/get mirror-1) (http/get mirror-2)])
       val-b (http/get second-thing)}
  (do-stuff val-a val-b))

bit of twiddling to do to get dynamic vars over ( see this if your needs are simple enough that future works https://clojurians.slack.com/archives/C03S1KBA2/p1664289931495499)

actually

(defmacro scram 
  [& body]
  `(let [chan# (chan)]
     (future (>!! chan# (do ~@body)))
     chan#))
=> #'user/scram

use this + the monkey patch in the link

You also need to close the chan

I ...knew? that

otherwise this is a breaking api change compared to go

what he said^

but yeah, your scram is great except that you also need to patch parking put and take to be blocking instead otherwise regex-replace won't work. This is why I did the spindle.monkey-patch ns though, it just does all that for you.

only restriction is you have to require the patch before you require anything else that might use core.async

(require '[clojure.core.async :as async])

(defmacro scram 
  [& body]
  `(let [chan# (async/chan)]
     (future (async/>!! chan# (do ~@body))
             (async/close! chan#))
     chan#))

@U4FFD43T4 You see what we mean by better maybe? What used to take a whole complex DSL is now just a feature of the vm

core.async really is and was a marvel - just glad to not need marvels for this

go closes even on exceptions, you need to do a try/finally

(defmacro scram 
  [& body]
  `(let [chan# (async/chan)]
     (future (try
               (async/>!! chan# (do ~@body))
               (finally
                 (async/close! chan#))))
     chan#))

yep, that looks good to me

Missing a 3?

Probably need to upgrade:

Clojure 1.11.0
user=> (symbol #'inc)
clojure.core/inc

Ah, right on. 1.9

user=> `inc
clojure.core/inc
user=> `x
user/x

can be used resolve to Symbol.
`symbol`  throws exception if the var is not defined, 

Rather consistently, I'm getting about 1000 inserts a second. On the nodejs implementation, it's easily 10x this

Are you using insert! or insert-multi! or some other java.jdbc call?

For batch insert to work "as expected", many JDBC drivers expect you to provide additional hints or options on the connection URL (i.e., in the db-spec hash map or whatever). I know of options for PostgreSQL and MySQL (they're different, of course) but I'm not familiar enough with SQLite to offer specifics.

The next.jdbc docs have some details about this (for some DBs). java.jdbc doesn't have great docs.

(any reason you're still using java.jdbc instead of switching to next.jdbc?)

Thanks! No reason other than just referencing a prior project of mine that used java.jdbc - I'll give next a try 🙂 - I was using insert-multi!, now I'm trying insert! in a transaction with a pmap

insert! is always going to be slower than a batch insert -- because it makes a roundtrip to the DB for each row. Batch inserts can send a whole block of rows to the database in a single roundtrip so they should be faster. Although you may need some additional connection options to tell the driver to actually do that properly.

pmap is almost never the right solution to any problem BTW.

Oh, why is that? That was one of the things I liked - converting a sequential map into a pmap always felt like "free concurrency". I'm doing some cpu intensive binary file parsing (10k files) and after parsing, mapping them to an sqlite db - I'm fairly certain my sqlite still isn't doing the pragma, but some pmaps have changed the overall time from 30 seconds down to 15s - I've benched the parsing at 5s, so if sqlite would honor my pragma, I think the whole thing would be under 10s now

(in this case, I guess I should note, I need parallelization, not just concurrency)

pmap is unmanaged concurrency. Sometimes the overhead it introduces isn't worth it, sometimes you can end up with a lot of contention and/or overwhelming your system. You can't tune it so it's a bit of a sledgehammer. That said, we do use it in a couple of places at work -- just six places in 142K lines of Clojure -- but those are legacy code that we just haven't gotten around to rewriting yet on top of either executors or core.async 🙂

haha that makes sense, ty for the insight 🙂

@U04V70XH6, are these the executors you speak of? https://blog.raek.se/2011/01/24/executors-in-clojure/ :thinking_face:

Java Executors, yes.

You have a lot more control over threads, pools, queues etc. It also sets you up nicely to leverage virtual threads in Java 19.

FYI, threads and SQLite sometimes don't get along very well. Had some negative experience with the Xerial driver some years back. Just a warning. There be dragons.

(let [form [1 2]]
  (match form
         [] :a
         [k] :b
         [k & ks] :c))

can you show a complete test of the failure you are experiencing? (similar to my example)

If I use this function

(defn my-assoc-in [m ks v]
  (match ks
    [] m
    [k] (assoc m k v)
    [k & ks] 3
    :else 4))

(my-assoc-in {:a {:b 2}} [:a :b] 3) returns 4 instead of 3

when you are binding symbols, use something like ?name syntax

cus match cares about external symbols too

(defn my-assoc-in [m ks v]
  (match ks
         [?k & ?ks] 3
         ))

that's what i do, at least

oh you mean, if I use an external symbol it will apply equality?

match can use external symbols to match against, which is cool, but you ran into the main issue with that feature.

match was using ks to match against, so [k & ks] is impossible to match in that case

(and (vector? ks) (>= (count ks) 1)) (try
                                           (let 
                                             [ks_left__121914
                                              (subvec ks 0 1)]
                                             (cond
                                               (and
                                                 (vector?
                                                   ks_left__121914)
                                                 (==
                                                   (count
                                                     ks_left__121914)
                                                   1))
                                               (let 
                                                 [?k
                                                  (nth
                                                    ks_left__121914
                                                    0)
                                                  ks (subvec ks 1)]
                                                 3)
                                               :else
                                               (throw
                                                 clojure.core.match/backtrack)))
                                           (catch
                                             Exception
                                             e__5150__auto__
                                             (if
                                               (identical?
                                                 e__5150__auto__
                                                 clojure.core.match/backtrack)
                                               (do
                                                 (throw
                                                   clojure.core.match/backtrack))
                                               (throw
                                                 e__5150__auto__))))

so, using your test case, the match expands to something like [k & [:a :b]]

definitely, I ignored the use of external symbols. I was expecting symbols to be only pattern specific (as in most ML based languages). Good to know, you saved my evening, thank you!

(defn my-assoc-in [m ks v]
  (let [form [1 ks]]
    (match form
           [?k ks] 3
           ))
  )

(my-assoc-in
  {:a {:b 2}}
  [:a :b]
  3)

i think meander actually doesn't do crazy things like match, but meander is also a lot more verbose in most cases. the macro expands for match are not too hard to read, or step threw, though. macro expanding is a good tool for figuring out what they are doing

yes, now I know about this equality, the error message is clear!