This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
- # aleph (16)
- # announcements (7)
- # asami (4)
- # aws (26)
- # babashka (26)
- # babashka-sci-dev (50)
- # beginners (118)
- # biff (7)
- # calva (15)
- # cider (6)
- # clj-kondo (8)
- # cljs-experience (3)
- # clojure (30)
- # clojure-austin (26)
- # clojure-europe (20)
- # clojure-france (2)
- # clojure-ireland (1)
- # clojure-nl (3)
- # clojure-norway (2)
- # clojure-spec (7)
- # clojure-uk (6)
- # clojurescript (12)
- # community-development (5)
- # conjure (1)
- # copenhagen-clojurians (3)
- # core-typed (71)
- # cursive (3)
- # datomic (1)
- # emacs (4)
- # fulcro (2)
- # helix (2)
- # introduce-yourself (3)
- # jobs (1)
- # london-clojurians (6)
- # lsp (122)
- # malli (2)
- # missionary (5)
- # overtone (14)
- # pathom (4)
- # polylith (1)
- # reagent (4)
- # reitit (1)
- # releases (1)
- # shadow-cljs (80)
- # testing (10)
- # tools-deps (6)
- # vim (3)
- # xtdb (19)
anyone know if it's possible to use Aleph TCP in a graal native image? it seems to pull in all too much of netty when requiring the
Aleph is built on Netty, so I would expect it to pull in a lot of it. In particular, it requires
which results in the following dep tree:
(def netty-modules '[transport transport-native-epoll codec codec-http handler handler-proxy resolver resolver-dns])
[io.netty/netty-codec-http "4.1.65.Final"] [io.netty/netty-codec "4.1.65.Final"] [io.netty/netty-handler-proxy "4.1.65.Final"] [io.netty/netty-codec-socks "4.1.65.Final"] [io.netty/netty-handler "4.1.65.Final"] [io.netty/netty-resolver-dns "4.1.65.Final"] [io.netty/netty-codec-dns "4.1.65.Final"] [io.netty/netty-resolver "4.1.65.Final"] [io.netty/netty-transport-native-epoll "4.1.65.Final"] [io.netty/netty-transport-native-unix-common "4.1.65.Final"] [io.netty/netty-transport "4.1.65.Final"] [io.netty/netty-buffer "4.1.65.Final"] [io.netty/netty-common "4.1.65.Final"]
You may have some issues building Aleph with Graal. A few people have asked over the last couple years, but Aleph was never built with Graal’s requirements in mind. There’s issues with top-level single namespaces in deps, statically-initialized threads, and probably some others. If you want to try it and catalog the issues you find, we could use the assistance.
@U10EC98F5 thanks for the reply, I guess I'm not sure why the
aleph.tcp namespace causes netty classes like
io.netty.channel.epoll.Native to be initialized, while
aleph.http does not and can be built fine here https://github.com/clj-easy/graalvm-clojure/tree/master/aleph
I'm starting to dig into the
aleph.netty namespace and I think it might be due to some macro in there that tcp uses
I’m not familiar with clj-easy, although I know @UM4NVAD62 has been working to get Aleph working with it. It’s another layer on top of Graal, and has its own issues (namely, it can’t handle single-segment namespaces, which some of Zach’s libs use since they were written at the dawn of time). At a low level, the use of epoll shouldn’t have anything to do with your protocol of interest (e.g., http vs tcp). It’s supposed to be an optimization for Linux. That being said, I haven’t looked into that stuff at all since becoming maintainer. WHy do you want/need to not load io.netty.channel.epoll.Native?
@U10EC98F5 so that specific clj-easy repo graalvm-clojure has just small example projects showing basic graalvm native image compatibility for various clojure libs. you can see that it needs to tell graal to initialize various netty classes at runtime in order to work https://github.com/clj-easy/graalvm-clojure/blob/master/aleph/project.clj#L32 I believe you're thinking of this repo https://github.com/clj-easy/graal-build-time#graal-build-time which does indeed have the single-segment ns issue you mentioned I see people claim that they have success building at least parts of netty with graal native https://github.com/netty/netty/issues/10616#issuecomment-997282655 so that makes me think it might be possible to get a netty tcp client in a native image. but something in either Aleph or the specific netty classes that it uses is causing these classes to be initialized at build time, which breaks the native-image build for me
making some progress, got an image to build, and am now running into reflection issues in gloss. I see someone else did some work on that https://clojurians.slack.com/archives/C0G922PCH/p1641253399033500 might try starting there and adding more type hints that I need
@U10EC98F5 fixed my issue with Gloss, needed to replace the
match-loop fn with a version that doesn't use
The original gloss is archived, but a few people have had problems, and it's available in clj-commons now, which we can edit. Maybe make an issue for it?
what are your plans for the libraries with single segment namespaces like
@UM4NVAD62 was doing some work on it, but I haven’t heard back from him in a while. Without him, I’ll get around to it when I get to it. But for both byte-streams and primitive-math, the plan is to copy the code to a multi-segment ns, and leave the old code in place. For byte-streams, I’ll update manifold/aleph to use the multi-segment ns, which should alleviate those clj-easy issues.
that makes sense, thanks! if I get impatient enough I might send a PR to add the multi-segment version of byte-streams, since it causes a pain in graal
Here’s what I sent Piotr about his changes (in https://github.com/piotr-yuxuan/byte-streams) for primitive-math and byte-streams. The tl;dr is we want to copy code to a ns like clj-commons.byte-streams, and deprecate the single-segment ns for byte-streams (but not primitive-math, I think that would cause chaos, it’s too popular).
I’m catching up on the primitive-math issue for clj-easy. Your fork of the
master branch looks good, want to make a PR for it against the clj-commons repo?
My only request is to remove
@Deprecated() from the namespace/packages. The deprecation comments should remain, though, and maybe add a note at the bottom of the README explaining that the nested ns is available if needed.
The reason for this is, with over 4 million downloads, we don’t want lots of people upgrading, and immediately thinking they need to change their namespaces to fix something, or asking questions, when the change is only necessary for a handful of people. I can see this causing a lot of unnecessary work by accident.
We can do the same with byte-streams when primitive-math is ready, but we can leave the deprecation metadata in, because unlike primitive-math, byte-streams will probably get updates.
cool ty, I'll take a look when I get a chance, probably this coming weekend.
I think I understand what you're saying, that
primitive-math should get the cloned namespaces first, and merged and released, and then
byte-streams can since it depends on it
@U10EC98F5 found some time to look at
let me know if that PR is in the right direction, happy to make other changes if you want