Fork me on GitHub
#clojure
<
2022-10-13
>
seancorfield01:10:40

Just found an interesting difference between JDK 18 and JDK 19 that affected Clojure/Java interop for us: (Thread/sleep (computed value)) may now be a reflective call and needs a type hint -- in JDK 19, Thread/sleep now has an additional overload that accepts Duration.

🙏 5
jjttjj01:10:13

Oh yeah, that's kind of cool that it accepts a duration now

seancorfield01:10:05

"Luckily", we have (set! *warn-on-reflection* true) in every (non-test) namespace so we caught it in CI. We had 30 such calls across our whole codebase. Calls like (Thread/sleep (* 1000 60 60)) are still fine.

lread02:10:53

Interesting! Sean, I’m sure you know this, but I’m finding https://github.com/jonase/eastwood is helpful in reporting on reflections (among other things).

pppaul02:10:36

what type hint does this need?

jrychter07:10:21

But that would only be a performance problem, right?

borkdude09:10:48

Yep, I've also encountered this one

Joshua Suskalo14:10:42

It's only a performance problem, but if you have short-duration sleeps that could make a huge difference.

Joshua Suskalo14:10:47

The other issue with this that I think most people don't pay much attention to is if you're calling sleep on an OS thread and not a virtual thread, very small sleep times are not precise. Schedulers can make a thread sleep for 5 ms minimum when the thread is marked high priority up to 20 ms on windows, and 15 ms on linux.

Joshua Suskalo14:10:10

(I need to double-check those numbers, I researched this a while back while writing a game loop, but those numbers should be in the right ballpark)

Joshua Suskalo14:10:45

If you need relatively high precision on short sleep times you need to spin pretty much.

seancorfield15:10:21

Not just performance. Without the type hint there were some calls Clojure could no longer resolve at all - the code wouldn't compile.

Joshua Suskalo15:10:07

ah, that's good to know, although surprising. Were there previously casts that were happening implicitly as invoke was called?

seancorfield15:10:45

That was what alerted me, when CI failed on 19. So I ran a full reflection checker to make sure all such calls got fixed. But it was a surprising change since sleep has been around and unchanged for so long - and I was expecting more obvious differences to pop up (but that's been the only one we've found so far).

seancorfield15:10:42

Presumably, yes, Clojure was inserting casts but not needing reflection.

seancorfield15:10:24

Or at least adding unboxing calls.

Casey19:10:28

Could someone please explain what "reflective call" means in this context? (I'm familiar with reflection in java, but not in clojure)

Joshua Suskalo19:10:15

if you understand it in java you're 90% of the way there. In Clojure if it can't tell with 100% certainty the exact method you're going to be calling at compile time (maybe because it doesn't know what class it's on, or the method has multiple overloads), then it generates code that looks at the object with java reflection to then try to call the method.

Joshua Suskalo19:10:58

You can help Clojure be sure about what method is being called by adding type hints.

Joshua Suskalo19:10:51

You can tell if Clojure is generating reflection if you put

(set! *warn-on-reflection* true)
in your file after the ns form.

Gerome09:10:45

Hello! I've grown a little reagent/re-frame app that has no backend over the last couple of months and I'm at a point where I'm considering moving into backend territory. In my day job I'm a UI dev and I have literally no experiences with databases. Is there a good tutorial or book that I can use to get started with both backend and database development?

elken09:10:45

Ha, jinx

🙌 1
valerauko09:10:46

high five

🙌 1
p-himik09:10:41

You might also find this interesting: https://github.com/dharrigan/startrek

👍 2
rmxm11:10:52

are there any tools for showing dependencies in the source code on a graph view? like static analysis drawing a graph of functions calling other functions - to visualize stuff like:

(defn x [a] a)
(defn y [a] (x a))
(defn a [a] (+ (x a) (y a)))

=> {x nil y [x] a [x y]}

borkdude11:10:56

Not all of the options @U2FRKM4TW mentioned do function <-> function dependencies though (but most of them do?)

p-himik11:10:47

Yeah, I was too lazy to double check. :)

rmxm11:10:30

why so few stars, seems like super handy tools for large codebases? (I usually have a baseline that super useful libs in clj get ~500-1k)

p-himik11:10:09

Personally, I have never had a use-case for those. But maybe I do have use-cases and I just prefer to use stuff that's built into my IDE and I "simulate" those libraries basically in my brain using the IDE tools.

borkdude11:10:27

Some tooling libraries are used only every so often, this is why they are less popular in terms of stars

marrs12:10:51

Does anyone know of a Babashka-like tool that compiles to native machine code rather than running inside a vm?

borkdude12:10:04

GraalVM native-image?

borkdude12:10:18

Babashka itself is a native binary compiled with GraalVM native-image

marrs12:10:20

Maybe I don't know enough about GraalVM. Doesn't it implement the JVM?

borkdude12:10:02

GraalVM native-image compiles JVM bytecode to machine code so it can run as a standalone binary

marrs12:10:14

Thanks Borkdude, I thought GraalVM binaries were quite resource intensive as well, but I'll check out native image. Part of my motivation is to keep the executable size small and resource usage low. I don't mind running against an interpreter as long as the interpreter is lightweight and fast. Think Dash over Node.JS or JVM. Thanks for the ClojureDart suggestion, Evan. I'll check that as well

borkdude12:10:11

@UVDD67FFX GraalVM binaries are not resource intensive. E.g. bb only requires 5mb of memory to run

👍 1
borkdude12:10:29

clj-kondo and #CM5HRADAA are a graalvm binary as well

borkdude12:10:46

but if you don't mind running in an interpreter, you can use babashka as well, if your code runs with that

borkdude12:10:19

I don't know what Dash is

marrs12:10:30

Dash is a lightweight Bourne Shell implementation. It's a popular alternative to Bash for shell scripts that don't use Bash-isms

borkdude12:10:59

If Node.js is an option, you can also use #C029PTWD3HR which is for scripting on Node.js using Clojure

p-himik12:10:34

> bb only requires 5mb of memory to run I know that those things don't really correspond to each other, but it's still somewhat surprising, pleasantly so, given that the binary itself is >80 MB.

borkdude12:10:13

It's < 80mb on macOS ;)

borkdude12:10:22

The Node.js binary is around the same size and starts up fast. But bb starts faster ;)

p-himik12:10:29

Huh! I definitely expected the exact opposite, with me being on Linux. My intuition is heavily skewed by Docker on MacOS that at least used to run in a VM, I think. And by some MacOS tools that are vastly limited compared to their (probably differently licensed) Linux counterparts.

borkdude12:10:31

Binary size is a weird thing but give or take 10-15 mb they're roughly the same size

borkdude12:10:43

The minimal size for a graalvm binary is around 5-10 mb

marrs12:10:23

@U2FRKM4TW Linux is a mixed bag. I find the popular Desktop distros to be pretty resource intensive. I don't think they're any better than macOS in terms of performance.

p-himik12:10:42

Oh, I'm not arguing about that. My point was that the basic functionality in Linux tends to be better.

☝️ 1
1
🐧 1
Ed13:10:20

There's also #C03SRH97FDK which I think it's pretty incomplete, but is a native Clojure I think.

👍 1
lispyclouds13:10:14

> the basic functionality in Linux tends to be better. 100% support this after 7+ years on mac and the last 3 on linux. 7 years was a lie 😛

bmo 1
mpenet14:10:32

I go back & forth every 10 years or so

mpenet14:10:41

back to osx in a few days

mpenet15:10:19

after a bad streak of unexplained kernel panics on my xps (possibly caused by hw issue) and just tired of dealing with stability issues

mpenet15:10:44

maybe in a week I ll be regretting that, we ll see

marrs15:10:11

osx...oldskool 😉

borkdude15:10:39

snow leopard for life

marrs15:10:49

I think Mavericks was the pinnacle for me. After that they just added stuff I didn't care about

mpenet15:10:25

last time I used one is after they switched to intel 😛

mpenet15:10:47

there's a pattern, I wait for shiny new arch to go back

Ed15:10:33

The last Windows I used as a daily desktop was Windows 2000. I've been working on Linux since then, until a few years ago when I had to use a Mac for work. It was a very frustrating experience. The window manager on a Mac is terrible.

👆 1
marrs15:10:36

Yeah, I dropped them after they switched to USB/TB3

lispyclouds15:10:25

My (literal) breaking point for Macs was when i started having an infra-dev title in my job description as well, started noticing overall dev on linux is just simply better. still have to deal with a mac at work though and would give M1 speed the claps it deserves.

lispyclouds15:10:41

Maybe can try asahi linux sometime

mpenet15:10:53

it's a bit too early for asahi imho

lispyclouds15:10:14

yeah wouldnt wanna brick the work machine 😛

lispyclouds15:10:50

@U050SC7SV citing your stability issues, could recommend Fedora. running it since i switched. almost as quick updates as arch but hasnt crashed once

mpenet15:10:54

I still have another xps for personal stuff, but the mac would be a work thing

mpenet15:10:35

I think I have a corrupted mem stick or something

mpenet15:10:54

it's very random, always at boot, and the message mentions something like that

lispyclouds15:10:12

probably h/w issues i suppose

mpenet15:10:19

+ I reached the 3yo on that machine so I could get it replaced, so it was just good timing

mpenet15:10:47

my other xps is flawless I have to say

lispyclouds15:10:12

gentle reminder of the original thread getting totally derailed. hope thats okay

mpenet15:10:47

oh sorry my bad

lispyclouds15:10:57

i started it!

mpenet15:10:01

to go back to the original subject: +1 for cljdart

marrs15:10:55

I got all I needed from the thread. Thanks to everyone who answered. I'm happy for the discussion to move onto more important things 😉

didibus16:10:08

GraalVM can do both JIT and AOT compilation. So if you use it's native image feature it does AOT compilation to native, similar to Go.

didibus16:10:25

If you're more looking for a packaged executable, as opposed to a compiled binary, you can check this out: https://github.com/babashka/nbb/tree/main/doc/caxa It packages an nbb script as a single executable.

souenzzo17:10:07

Why call str on lazy sequences triggers the evaluation of the sequence?

user=> (str (map prn [1 2 3]))
1
2
3
"clojure.lang.LazySeq@745f"

hiredman17:10:01

wild to see that marked as a bug after all this time, but that doesn't actually answer the question

souenzzo17:10:57

and the fix will be a breaking change 👀

hiredman17:10:48

yeah, the whole thing is surprising

souenzzo17:10:38

• it evaluates everything, not just the first chunk • from clojure 1.0.0 up to 1.11.1, it behaves exacly the same, with same hash code in the str etc.

sam18:10:20

According to the Jira bug, the lazy seq is realized in order to compute the hash used in the string representation. (Just saying this because it wasn’t obvious to me based on the existing messages in this thread).

p-himik18:10:37

Yeah, my own assumption in my previous message was wrong, sorry for the confusion.

skylize18:10:55

Pure coincidence that Alex was apparently just looking at this? from https://clojure.atlassian.net/browse/CLJ-2647 > Alex Miller updated the Summary yesterday > Revisit printing of lazy seqs → Lazy seqs do not print readably

Asko Nōmm17:10:08

Has anyone ever tried to serve webfonts using io/resource ? When I slurp on it, it seems to give me an corrupted file (perhaps encoding issue?). I'm running a web app and serving resources via io/resource and while CSS files, for example, work fine, webfonts (ttf, woff, woff2, eot) do not. By "do not work" I mean I get the file contents just fine, but when served with the corresponding mime type and all, doesn't work when visiting with the browser. Reason I think its corrupted is because when I download a TTF via the browser, I can't open it because it says it's faulty - and yet I can open the file in resources/ just fine. Hence, I'm guessing encoding, but I'm not sure.

hiredman17:10:38

don't use slurp for binary data

Asko Nōmm17:10:35

What should I use instead?

hiredman17:10:50

depends what you are doing

hiredman17:10:13

there are ring middlewares for serving resources if that is all you want to do

hiredman17:10:58

if you want say the contents of a file as a byte array, then you can use a ByteArrayOutputStream together with http://clojure.java.io/copy

Asko Nōmm17:10:52

I have my own baked solution - no ring - that already works fine with static assets, except webfonts. So that's why I'm, in particular, interesting in figuring out what's the issue there. Why slurp doesn't work, and what would. I'm not looking to replace my entire system.

hiredman17:10:57

slurp builds a string

hiredman17:10:57

creating a string involves taking a bunch of bytes(8 bits), and interpreting as characters (actually 16 bit code points? some characters are two code points depending) using an encoding

hiredman17:10:28

just don't use slurp for binary data, use a baos

hiredman17:10:36

even better for large things, don't pull them entirely into memory at all, but how to do that will depend on whatever webserver api you are working with

Asko Nōmm18:10:25

Yeah large things will prove to be a headache most likely, thankfully I just need webfonts which are very small. I got it working with

(io/input-stream (io/resource path))
Thank you!

jpmonettas20:10:40

Any ideas why could it be that doing

clj -Sforce -Sdeps '{:deps {mount/mount {:mvn/version "0.1.16"}}}' -Stree
Downloading: mount/mount/0.1.16/mount-0.1.16.pom from clojars
Downloading: mount/mount/0.1.16/mount-0.1.16.jar from clojars
org.clojure/clojure 1.11.1
  . org.clojure/spec.alpha 0.3.218
  . org.clojure/core.specs.alpha 0.2.62
mount/mount 0.1.16
and then doing
cat ~/.m2/repository/mount/mount/0.1.16/mount-0.1.16.pom | grep artifactId

<artifactId>mount</artifactId>
<artifactId>clojure</artifactId>
<artifactId>clojurescript</artifactId>
<artifactId>datascript</artifactId>
<artifactId>compojure</artifactId>
<artifactId>ring-jetty-adapter</artifactId>
<artifactId>cheshire</artifactId>
<artifactId>hiccups</artifactId>
<artifactId>clojurescript</artifactId>
<artifactId>cljs-time</artifactId>
<artifactId>logback-classic</artifactId>
<artifactId>tools.logging</artifactId>
<artifactId>hooke</artifactId>
<artifactId>tools.namespace</artifactId>
<artifactId>tools.nrepl</artifactId>
<artifactId>datomic-free</artifactId>
....
  
So I don't understand why -Stree doesn't show them, and also no idea why mount packaged with all those deps, since they are dev dependencies

p-himik20:10:56

Have you checked the <scope> of those dependencies? It's not "packaged" with them, it just depends on them.

jpmonettas20:10:45

oh ok, they are provided, I see

p-himik20:10:26

Yeah, and some are for tests only.

jpmonettas20:10:21

thanks again @U2FRKM4TW!

👍 1
Alex Miller (Clojure team)20:10:13

deps only considers compile and runtime dependencies of Maven artifacts

p-himik20:10:07

Maybe a nonsensical question, but does it make sense to exclude compile-time dependencies when building an uberjar? Or does a compile-time dependency conceptually also constitutes a runtime one?

Alex Miller (Clojure team)20:10:12

conceptually, maybe, but in practice Clojure libs don't use compile scope dependencies, and even Java libs pretty rarely use something only at compile time and not as a runtime dep

p-himik20:10:54

Ah, so deps considers them just because not doing that would potentially break some libs?

Alex Miller (Clojure team)20:10:28

better to include and let you exclude than the opposite imo

p-himik20:10:42

Right, I see. Thanks!

Alex Miller (Clojure team)21:10:12

(but again, it's very rare to encounter this at all)