Fork me on GitHub
#clojure
<
2023-07-20
>
dpsutton19:07:46

question if this is a bug or intentional design. If you AOT all files, should empty namespaces be compiled? A file (ns segment.foo) in segment/foo, on the classpath, compiled, etc. It did not create a segment/foo__init.class file. So it seems it is not compiled at all

hifumi12320:07:34

I would consider this a bug only if the empty namespace is required elsewhere and it didnt get AOT compiled

dpsutton20:07:13

(find-ns 'segment-foo) would behave differently when run from a jar than from source.

dpsutton20:07:23

it seems weird to me. just wanting to know if it was intentional

dpsutton20:07:45

we used it (try (require 'segment.foo) true (catch E e false)) to figure out if additions were present

seancorfield20:07:58

I can't reproduce this:

Thu Jul 20 13:19:38
(~/clojure)-(!2006)-> mkdir src/segment

Thu Jul 20 13:19:54
(~/clojure)-(!2007)-> echo '(ns segment.foo)' > src/segment/foo.clj

Thu Jul 20 13:20:08
(~/clojure)-(!2008)-> rm -rf classes/ && mkdir classes

Thu Jul 20 13:20:17
(~/clojure)-(!2009)-> clj
Clojure 1.12.0-alpha4
user=> (compile 'segment.foo)
segment.foo
user=>

Thu Jul 20 13:21:28
(~/clojure)-(!2010)-> tree classes/|tail
│           ├── alpha$fn__98.class
│           ├── alpha$loading__6803__auto____3.class
│           ├── alpha$quotable.class
│           └── alpha__init.class
└── segment
    ├── foo$fn__128.class
    ├── foo$loading__6803__auto____126.class
    └── foo__init.class

4 directories, 53 files

Thu Jul 20 13:21:38
(~/clojure)-(!2011)->

dpsutton20:07:25

interesting. thank you sean. i’ll investigate more

seancorfield20:07:11

How are you doing the AOT? Is it possible the (empty) ns is being filtered out by something and not actually compiled?

dpsutton20:07:31

mishmash of tools build and depstar (!) since we haven’t touched it in a while

seancorfield20:07:23

depstar? Hahaha... there's your problem right away! :rolling_on_the_floor_laughing:

dpsutton20:07:02

yeah. it does have one unique benefit though in that it doesn’t explode onto disk and you don’t run into casing issues with license files and LICENSE directories. those can coexist in a jar and on linux but clash in osx

seancorfield20:07:18

Yeah, and that one-pass approach also makes it faster -- when I first took on depstar, it did explode onto disk, just like tools.build.

dpsutton20:07:45

ok. so it seems this namespace was not in the dependency graph because it’s a dynamic require. and therefore was not compiled. and removing sources meant it could not be located at runtime

Joshua Suskalo21:07:46

sounds like you just need to declare it as a second aot root

seancorfield21:07:02

(or make sure that source file is copied into uberjar)

dpsutton21:07:43

the source file has always been there. we’re removing source files and this cropped up. it’s now added as another aot root

2
seancorfield21:07:06

I think the only reason we don't run into this issue at work is that everything is a :local/root dep and uber copies those as source into target/classes -- because we only AOT the main ns and for at least one of our projects, that uses requiring-resolve to load everything else 🙂

Joshua Suskalo21:07:35

What kind of performance difference is there between AOTing nothing, AOTing the main ns and doing everything with requiring-resolve, and AOTing everything? Have you observed much?

Joshua Suskalo21:07:10

I guess I should qualify "performance" as primarily startup time, I'd imagine that besides direct linking it wouldn't affect runtime performance much either way.

dpsutton21:07:54

AOT vs non AOT is a huge penalty. we’re still investigating direct linking

seancorfield21:07:21

For several of our apps, the startup time difference is seconds (AOT) vs minutes (not), but otherwise it doesn't matter. For the processes where we dynamically load things, startup time doesn't matter to us.

👆 2
seancorfield21:07:02

The only reason we AOT at all is to reduce startup time to improve rolling deployments across the cluster.

dpsutton21:07:01

same here. startup decreases by minutes with aot

👍 2