Fork me on GitHub
#beginners
<
2023-08-23
>
Ronny Li11:08:31

Hi everyone, I was just wondering if it's normal for it to take almost 10 (actually 4) minutes to start up a cider repl?

delaguardo12:08:10

Depends. Is it a fresh start and it fetches dependencies? Also there are might be several places in clojure project that gets invoked during repl start: https://github.com/technomancy/leiningen/blob/github/sample.project.clj#L378 option in project.clj or user.clj namespace somewhere in project's sources

daveliepmann12:08:57

Normal? Absolutely not. Possible? Yes, in some cursed situation.

souenzzo12:08:07

old clojure (before 1.10.1) may have pretty slow startup with new JVM's https://insideclojure.org/2019/03/23/journal/ that is the only case that I can imagine a 10min startup (or a pretty slow machine)

Ronny Li12:08:53

Hah okay good to know! Yeah I just see "clojure is slow to start " but didn't have any specifics on what that meant. I'll try to figure it out and will ask more questions here! (On clojure 1.11 btw)

souenzzo12:08:00

btw: it is not due "clojure initialization" it is probabliy: • lein plugins loading • project loading thing into classpath during the REPL initialization i personally consider both a bad practice.

🙏 2
Ronny Li12:08:11

Any tips for timing and debugging the startup process?

jpmonettas12:08:16

as a reference, the project I'm working with gets me from cider-jack-in to when I can type something at the repl in under 6 seconds in an old laptop from 2013. And it is loading some extra nrepl middleware

🙏 2
jpmonettas12:08:14

> Any tips for timing and debugging the startup process? do you have a user.clj file?

Ronny Li12:08:29

Thank you for the benchmark! I'm on a much newer Mac so yeah something's wrong

Ronny Li12:08:01

I have a user profile in profiles.clj but no user.clj

souenzzo12:08:06

see how much time time lein classpath takes to run

👀 2
Ronny Li12:08:07

Will do! Not at a computer right now so I'll update later in the day

jpmonettas12:08:33

oh, if you are on lein I think it will also load whatever ns you have on :main if iirc. My times were using clj and not loading anything

Ronny Li12:08:38

User name space on main

jpmonettas12:08:31

oh, then it shouldn't load anything at startup, which is more weird

delaguardo12:08:30

:repl-options { :init might contain expression that gets executed during repl start up. For example the project I'm maintaining at the moment takes ~4 minutes to run a repl just because during :init it does some heavy lifting ensuring development database state

👀 2
Ronny Li12:08:26

For context if I do anything with lein at all (like lein show-profiles) it takes several minutes

jpmonettas12:08:00

another thing you can try is :

JVM_OPTS=-verbose:class lein repl | grep your_project_top_ns

🙏 2
jpmonettas12:08:39

passing -verbose:class to the jvm will make it log every class it is loading, so if you see your ns there, they are being loaded just by doing lein repl

👍 2
jpmonettas12:08:41

you need underscores in your_project_top_ns if it is your-project-top-ns

jpmonettas12:08:07

after you figure out what of your ns are being loaded, I would search in them for top level expressions that are evaluated and can take a long time, like : (def thing (download-big-file))

👍 2
Ed16:08:19

If you're on an arm mac, are you sure that you've installed the correct architecture jvm? you might be emulating x86 by accident?

☝️ 2
flowthing17:08:37

I wrote a blog post on this topic a while ago: https://blog.flowthing.me/clojure-repl-start/ One thing that article only touches on, though, is that Clojure starts noticeably faster on newer Java versions (17+ or so).

🙏 2
Ronny Li04:08:00

Thanks everyone, here's where I'm at. Btw it turns out my original assertion of 10 minutes to start the repl was an exaggeration. In reality it's around 3-4 minutes. • time lein classpath 71.16s user 62.80s system 96% cpu 2:19.04 total • using JVM_OPTS=verbose showed another 40 seconds of loading internal libraries. There were some things that looked like they took ~5 seconds • Cleared out everything in my user.clj and the repl still took 3-4 minutes to start. • @U0P0TMEFJ I have an M1 so it's possible there's an issue here. How can I check? • I probably won't mess with the java version just yet since I'll be out of sync with our production systems

delaguardo08:08:49

> I have an M1 so it's possible there's an issue here. How can I check? exec this command with running repl: ps aux | grep -f <(fuser /usr/libexec/rosetta/runtime 2>/dev/null | xargs -n1) you will see a list of processes started under rosetta

flowthing08:08:26

I think file $(which java) might also work?

λ file $(which java)
/usr/bin/java: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e]
/usr/bin/java (for architecture x86_64):        Mach-O 64-bit executable x86_64
/usr/bin/java (for architecture arm64e):        Mach-O 64-bit executable arm64e

Ronny Li16:08:18

Perfect, thank you! (here's mine)

file $(which java)
/usr/bin/java: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e]
/usr/bin/java (for architecture x86_64):	Mach-O 64-bit executable x86_64
/usr/bin/java (for architecture arm64e):	Mach-O 64-bit executable arm64e

Joseph Graham13:08:28

So many ways to serialise stuff. I can use Transit, Fressian, Nippy. I just want to serialise some data before putting it into a RabbitMQ using langohr (my map includes a bytearray). I tried Nippy and Transit and both worked fine. I couldn't get Fressian working because it gives me HeapByteBuffer which there seems to be no strait-forward way to convert that to a bytearray which is what langohr needs. What is the different between these libraries other than the fact that they all provide different outputs: • nippy gives me a bytearray • Transit gives me a stream • Fressian gives a HeapByteBuffer :face_with_raised_eyebrow:

delaguardo14:08:56

The most important factor to me is if there might be breaking changes in the serialisation process after updating a library. I remember some painful migrations when I used nippy. Nothing like that should happen with transit because of the way it is designed. And I can't say anything about fressian because I never used it.

hiredman15:08:37

You can also just use pr-str

🎯 2
delaguardo15:08:59

pr-str doesn't work well with bytearray, does it?

delaguardo15:08:23

unless with custom print-method

hiredman15:08:09

The question isn't asking about serializing byte arrays, but wants to serialize to a byte array

delaguardo15:08:19

> my map includes a bytearray I'm believing that there is bytearray to serialise

hiredman15:08:53

Sure then yes you'd need a custom print-method

hiredman15:08:15

Heapbuffers have a .array method

hiredman15:08:49

Transit write to a java.io.bytearrsyoutputstream

hiredman15:08:09

Not a fan of nippy

hiredman15:08:51

Transit is actually a family of formats, the most common one is encoded as json, which also has issues with binary data

Joseph Graham16:08:14

Thanks both. I have already tested nippy and transit+msgpack. Both are working fine. But I wanted to try them all to find which is most convenient. Honestly nippy was easiest to implement because it gives me a byte array which langohr accepts. Fressian gave me a HeapByteBuffer which I have no idea what to do with that.

Joseph Graham16:08:21

> Transit write to a java.io.bytearrsyoutputstream Yes but I can convert this to a byte array by simply calling .toByteArray on it

hiredman16:08:51

HeapByteBuffer.array

Joseph Graham18:08:13

yes Fressian works too when calling .array on the HeapByteBuffer. Thanks!