Fork me on GitHub
#clojure
<
2022-03-07
>
Pratik06:03:57

What would be the best way to suppress exceptions in clojure unit tests? I am trying to write unit tests for a scenario where something gets retried if it sees some exception, so, I am mocking a function to throw an exception but when I run the tests, it throws the exception which has a long stacktrace, so while running the tests it looks like there’s something wrong with the tests, when there isn’t.

dpsutton07:03:28

you mean you see it printed in the test results?

Pratik16:03:47

any way we can fix that?

dpsutton16:03:55

i suspect you are catching and logging the exceptions. so its your code that is doing this.

👍 1
Pratik16:03:33

yes, that’s a possibility, I’ll check my code, maybe it’s just my logs

clj.max08:03:40

Is there a somewhat authoritative list of Clojure implementations? Besides Clj, CljCLR, and Cljs there seems to be Babashka, Lumo, Planck (maybe not really implementations?), Joker, Clojerl, ClojureC, ..?

borkdude08:03:07

Nbb is another one, like babashka based on SCI but runs in Node.js. #nbb

clj.max08:03:42

Ah, I suppose SCI is an implementation and Babashka/Lumo/Planck/nbb are.. ? What's a good word? Feels like a sort of wrapper?

borkdude08:03:34

environments?

👍 1
Kris C10:03:23

Will the use of transient make sort-by faster?

p-himik10:03:36

Probably not because internally it uses a mutable Java array anyway.

✔️ 1
Kris C11:03:29

Can I use a transient collection to speed up clojure.walk/postwalk?

p-himik11:03:49

Which part exactly would you apply transient to?

Kris C11:03:53

I would apply transient to a vector that I use postwalk on

Kris C11:03:53

Code I wish to optimize:

(defn flatten-db-ident [v]
  (clojure.walk/postwalk
    #(if (and (map? %) (contains? % :db/ident)) (:db/ident %) %)
    v))

p-himik11:03:02

Is that code a bottleneck? Even if making the outer vector a transient collection - what about the nested data structures? Do you know how to get the source code of a Clojure function?

Kris C11:03:29

(source fn)?

Kris C11:03:24

Yes, this part and the sorting are bottlenecks...

p-himik11:03:45

Yep. The source code of postwalk (and walk which it uses) is ~10 lines. You should be able to see that internally it already uses transients because it uses into.

p-himik11:03:35

> Yes, this part and the sorting are bottlenecks... Really?.. It seems that you're accessing a DB - and you're saying that DB access is faster than or of comparable performance to your usage of postwalk?

p-himik11:03:54

How did you determine exactly that sort and postwalk are bottlenecks?

Kris C11:03:26

they are after I get the data from DB. The processing of the data (vector of maps) is where I want to optimize stuff.

p-himik11:03:34

If your DB takes 100 ms to extract the data, and then your processing takes 1 ms to churn through that data, that 1 ms doesn't matter. The DB is still the bottleneck - not that postwalk or sort.

p-himik11:03:53

BTW consider sorting on the DB end - it might be dramatically faster there just because your DB has access to all the indices. Depends on your DB schema and the sorting query, of course.

Kris C11:03:25

I know, but I am using datomic, no sorting support

Kris C11:03:17

and the result set is large (because I have to realize most data in order to sort)

p-himik11:03:29

Alright, but my statement about bottlenecks is still applicable. Unless your data processing takes comparable time to IO, don't think about optimizing it - it would be time spent in vain.

p-himik11:03:07

And if your data is truly large, then sort simply can't be made faster - it's O(n * log(n)), it will drown your postwalk which is O(n).

👍 1
Kris C11:03:08

I am on the datomic peer, no IO (all in memory)...

Kris C11:03:02

ok, thanks for your insights, will try a different route (since I cannot get a "cheap" sort-by and/or postwalk optimization)

Kris C11:03:10

Thanks a lot for your help

👍 1
thumbnail12:03:19

We actually ran into the same bottleneck with a similar postwalk function over our datomic results, and replaced them with an xform -fn (which is :db/ident) in the pull expression. This requires the devs to add the xform option explicitly with idents but has the upside of being pretty expressive and very performant.

p-himik12:03:49

Right, forgot to mention - if postwalk is indeed a bottleneck, apart from xform another good library is specter.

1
👍 1
Kris C12:03:37

@UHJH8MG6S thanks, this is actually the path I am taking now

1
Kris C12:03:14

@U2FRKM4TW thanks, I considered this as well

Joshua Suskalo16:03:55

@U013100GJ14 datomic has sorting support

Joshua Suskalo16:03:16

(although I'm now asking a few questions over in #datomic because it appears that these aggregators may not do what I think they do)

Kris C16:03:03

No sorting in Datomic, @U5NCUG8NR

Joshua Suskalo16:03:50

Yeah, sorry, I misunderstood some stuff. That just seemed really surprising to me

Al Z. Heymer12:03:40

Is there a library for data exploration or similar? I use REBL for the task, but I would love to have something like map x: {:key1 [250 int], :key2 [142 String]}

p-himik12:03:49

Not sure what you mean. Isn't that exactly what REPL shows you? Also, there are Reveal and Portal - maybe they have additional data views that would be useful to you.

Al Z. Heymer12:03:32

I see.. I was unclear. I didn't mean data exploration, but more like a library that would basically give me a schema of what I am exploring, so I don't have to scroll through tons of data, but to see the shape of a data structure on the first look. So [250 int] would be just a quick look on how many integers this list has, and that it actually only consists of integers.

pavlosmelissinos13:03:13

Maybe you're looking for something like spec-provider: https://github.com/stathissideris/spec-provider

jumar13:03:48

is clojure and clj the same thing?

Alex Miller (Clojure team)13:03:48

Almost - clj is a wrapper that adds terminal facilities for repl editing so is better for that

1
Apple13:03:54

rlwrap clojure is almost clj

delaguardo13:03:07

clj is clojure wrapped with rlwrap