Fork me on GitHub

What would be the best way to suppress exceptions in clojure unit tests? I am trying to write unit tests for a scenario where something gets retried if it sees some exception, so, I am mocking a function to throw an exception but when I run the tests, it throws the exception which has a long stacktrace, so while running the tests it looks like there’s something wrong with the tests, when there isn’t.


you mean you see it printed in the test results?


any way we can fix that?


i suspect you are catching and logging the exceptions. so its your code that is doing this.

👍 1

yes, that’s a possibility, I’ll check my code, maybe it’s just my logs


Is there a somewhat authoritative list of Clojure implementations? Besides Clj, CljCLR, and Cljs there seems to be Babashka, Lumo, Planck (maybe not really implementations?), Joker, Clojerl, ClojureC, ..?


Nbb is another one, like babashka based on SCI but runs in Node.js. #nbb


Ah, I suppose SCI is an implementation and Babashka/Lumo/Planck/nbb are.. ? What's a good word? Feels like a sort of wrapper?



👍 1
Kris C10:03:23

Will the use of transient make sort-by faster?


Probably not because internally it uses a mutable Java array anyway.

✔️ 1
Kris C11:03:29

Can I use a transient collection to speed up clojure.walk/postwalk?


Which part exactly would you apply transient to?

Kris C11:03:53

I would apply transient to a vector that I use postwalk on

Kris C11:03:53

Code I wish to optimize:

(defn flatten-db-ident [v]
    #(if (and (map? %) (contains? % :db/ident)) (:db/ident %) %)


Is that code a bottleneck? Even if making the outer vector a transient collection - what about the nested data structures? Do you know how to get the source code of a Clojure function?

Kris C11:03:29

(source fn)?

Kris C11:03:24

Yes, this part and the sorting are bottlenecks...


Yep. The source code of postwalk (and walk which it uses) is ~10 lines. You should be able to see that internally it already uses transients because it uses into.


> Yes, this part and the sorting are bottlenecks... Really?.. It seems that you're accessing a DB - and you're saying that DB access is faster than or of comparable performance to your usage of postwalk?


How did you determine exactly that sort and postwalk are bottlenecks?

Kris C11:03:26

they are after I get the data from DB. The processing of the data (vector of maps) is where I want to optimize stuff.


If your DB takes 100 ms to extract the data, and then your processing takes 1 ms to churn through that data, that 1 ms doesn't matter. The DB is still the bottleneck - not that postwalk or sort.


BTW consider sorting on the DB end - it might be dramatically faster there just because your DB has access to all the indices. Depends on your DB schema and the sorting query, of course.

Kris C11:03:25

I know, but I am using datomic, no sorting support

Kris C11:03:17

and the result set is large (because I have to realize most data in order to sort)


Alright, but my statement about bottlenecks is still applicable. Unless your data processing takes comparable time to IO, don't think about optimizing it - it would be time spent in vain.


And if your data is truly large, then sort simply can't be made faster - it's O(n * log(n)), it will drown your postwalk which is O(n).

👍 1
Kris C11:03:08

I am on the datomic peer, no IO (all in memory)...

Kris C11:03:02

ok, thanks for your insights, will try a different route (since I cannot get a "cheap" sort-by and/or postwalk optimization)

Kris C11:03:10

Thanks a lot for your help

👍 1

We actually ran into the same bottleneck with a similar postwalk function over our datomic results, and replaced them with an xform -fn (which is :db/ident) in the pull expression. This requires the devs to add the xform option explicitly with idents but has the upside of being pretty expressive and very performant.


Right, forgot to mention - if postwalk is indeed a bottleneck, apart from xform another good library is specter.

👍 1
Kris C12:03:37

@UHJH8MG6S thanks, this is actually the path I am taking now

Kris C12:03:14

@U2FRKM4TW thanks, I considered this as well

Joshua Suskalo16:03:55

@U013100GJ14 datomic has sorting support

Joshua Suskalo16:03:16

(although I'm now asking a few questions over in #datomic because it appears that these aggregators may not do what I think they do)

Kris C16:03:03

No sorting in Datomic, @U5NCUG8NR

Joshua Suskalo16:03:50

Yeah, sorry, I misunderstood some stuff. That just seemed really surprising to me

Al Z. Heymer12:03:40

Is there a library for data exploration or similar? I use REBL for the task, but I would love to have something like map x: {:key1 [250 int], :key2 [142 String]}


Not sure what you mean. Isn't that exactly what REPL shows you? Also, there are Reveal and Portal - maybe they have additional data views that would be useful to you.

Al Z. Heymer12:03:32

I see.. I was unclear. I didn't mean data exploration, but more like a library that would basically give me a schema of what I am exploring, so I don't have to scroll through tons of data, but to see the shape of a data structure on the first look. So [250 int] would be just a quick look on how many integers this list has, and that it actually only consists of integers.


Maybe you're looking for something like spec-provider:


is clojure and clj the same thing?

Alex Miller (Clojure team)13:03:48

Almost - clj is a wrapper that adds terminal facilities for repl editing so is better for that


rlwrap clojure is almost clj


clj is clojure wrapped with rlwrap