Fork me on GitHub
#clojure
<
2022-04-14
>
MistyBayou07:04:57

GM. I'm trying to build an Uberjar for my project and it's coming out to 1GB. It's a pretty small project so I figure this size is coming from somewhere in the dependencies. How would I go about figuring out where its coming from?

Tuomas-Matti Soikkeli07:04:42

I guess it’s a jar file in the end so you can unzip it and take a look whats inside.

MistyBayou07:04:14

Oh thanks, is a JAR a zip under the hood?

p-himik07:04:42

What tool are you using to specify your dependencies?

phronmophobic07:04:06

yes, jars are just zips. If you rename it from foo.jar to foo.zip, you can just unzip it to see what's inside

Tuomas-Matti Soikkeli07:04:09

Yes, JAR file is just an ordinary zip file.

Tuomas-Matti Soikkeli07:04:41

Not 100% sure about uberjar thought, haven’t tested it.

phronmophobic07:04:06

If you're trying to find out which dependencies are contributing the most to your uberjar, I wrote small lib to help called https://github.com/phronmophobic/snowball

flowthing07:04:35

Or you can just run jar xf my-uberjar.jar.

MistyBayou07:04:49

@U4ZDX466T I saw that command yesterday but I don't have jar installed and can't find it in my OS package manager.

Tuomas-Matti Soikkeli07:04:24

I guess it comes with JDK installation.

flowthing07:04:25

Yeah, it's included in the Java Development Kit.

phronmophobic07:04:39

oh, snowball doesn't work with lein projects directly

p-himik07:04:57

Try using lein deps :tree then. I don't think it will report the size of the dependencies, but it'll at least show why things are in the uberjar.

MistyBayou07:04:01

Ah OK, I'll reinstall openjdk, mine is probably ancient.

p-himik07:04:25

There's also lein deps :why some/coord where some/coord is group/artifact-id of a specific jar - it'll tell you why specifically that jar was included. And if you're still discovering things, I would recommend checking out tools.deps as well - i.e. Clojure CLI along with a deps.edn file instead of project.clj. It's overall simpler and more flexible, and for this particular task, you'll be able to use https://github.com/phronmophobic/snowball and https://github.com/clojure/tools.deps.graph with its size option.

MistyBayou07:04:36

Thanks, I've been meaning to check out tools.deps as everyone seems to be moving to it these days.

eskos07:04:38

@U03BA00LJ9L To be the naysayer, I haven’t noticed that 🙂 Of course here on Clojurians and the libraries closer to the core ecosystem are moving to those tools, but a few steps further away everyone still seems to prefer Leiningen, mostly because as a tool it is “a solved problem”. Optics, perspective, positive reinforcement, statistics…things like this can get complicated quite quickly.

Ben Sless12:04:56

What version of lein?

MistyBayou07:04:30

That's what I thought!

MistyBayou07:04:44

My suspect is clojure2d because I found something else mentioning it making huge JARs.

oly07:04:40

any recommendation for a s3 library thats works for aws gcp and do kind of want something for just storage instead of pulling in an entire suite of libs like the aws library

oly07:04:26

well I am aware of that but would i not be bringing in api's for all the aws services instead of just s3

oly08:04:19

although

{:deps {com.cognitect.aws/api       {:mvn/version "0.8.539"}
        com.cognitect.aws/endpoints {:mvn/version "1.1.12.192"}
        com.cognitect.aws/s3        {:mvn/version "822.2.1109.0"}}}
make's me think perhaps not I wonder how well it works with GCP and do

p-himik08:04:11

Yep, it's modular and rather small. But no clue about your specific needs - personally, I use it only for S3.

oly08:04:13

my need is basically to use storage but work for any platform instead of tying the library to a specific cloud provider

p-himik08:04:01

If conceptually S3 serves you, I'd definitely go with it - quite a few other providers have picked it up.

oly08:04:04

yeah it may be fine I will have to test, looking around not finding any uptodate alternatives anyway

Oliver Marks12:04:39

seems this does not work overly well outside amazon ecosystem https://github.com/cognitect-labs/aws-api/issues/150

flowthing11:04:10

I guess there's really no "official" documentation for user.clj? The only page I can find is this https://clojure.org/guides/dev_startup_time, which is useful, but only touches on user.clj rather briefly.

Alex Miller (Clojure team)12:04:31

There's a deps and cli guide in the same place

Alex Miller (Clojure team)12:04:01

I don't know if there is any user.clj “guide”

Alex Miller (Clojure team)12:04:06

It's kind of a weird historical artifact. The Clojure runtime loads user early in startup, if it exists

flowthing12:04:27

Yes, I'm familiar with how it works. I'm just looking for something to link to in a blog post that (also) discusses user.clj, really. 🙂

flowthing12:04:03

A rationale sorta thing, I guess.

Alex Miller (Clojure team)12:04:23

Don't think I have anything you can point to

flowthing12:04:49

All right, thanks.

Alex Miller (Clojure team)12:04:54

it is literally unmentioned in the reference section, I'll add a doc issue for that

🙏 1
flowthing12:04:39

Great, thanks! :thumbsup:

pinkfrog14:04:37

What’s the proper way of converting a namespaced keyword to its qualified name string

:a/b => a/b

Ben Sless14:04:15

string or symbol?

Ben Sless14:04:39

(str (symbol kw))

thanks3 1
pinkfrog08:04:46

I just happened to know there is also a

(.-fqn :foo/bar)

Ben Sless08:04:08

Maybe on Clojurescript, the other is a more portable solution

pinkfrog14:04:14

Given a function foo that returns a new val. If the val is not nil, return the val, other wise return default.

(let [val (foo)]
  (if (some? val)
    val
    default-value)
Possible to avoid the temporary val variable?

Andrea Imparato14:04:39

maybe with or? (or (foo) default-value) ?

p-himik14:04:31

And if you need to handle false separately from nil, use if-some.

pinkfrog14:04:27

thanks. if-some is the one I am looking for.

Aaron Decker18:04:37

Is there a good way to create a Clojure sequence that implements the java.lang.Iterable interface? Using clojure.reflect/reflect on things like [], ’() shows that many of these sequences have an iterator member but they don’t have other things like a forEach method required by the java.lang.Iterable interface. The context here is that I’m writing mocking code for a java library that returns an Iterable from one of its methods.

Alex Miller (Clojure team)18:04:09

forEach is a default method provided by the interface so you don't need that

Aaron Decker18:04:56

Oh, I see. So then is it safe to say that things like clojure.lang.PersistentVector implement Iterable since they have the iterator member?

Alex Miller (Clojure team)18:04:08

yes, that's the only requirement

👍 1
Alex Miller (Clojure team)18:04:28

you just have an Iterable from some other source, not a type you're making yourself?

Aaron Decker18:04:14

I’m writing a wrapper around some google Firestore code that returns a Iterable of documents in one of its functions. That part is straightforward; I’m also writing a mocked-up version that uses an atom for unit-testing code that interacts with the database.

Alex Miller (Clojure team)18:04:03

iterator-seq is a tool for making a seq that wraps an iterator, but I think you want the reverse?

Aaron Decker18:04:12

Exactly. I need to grab a node out of the mock db-atom and make it into something that implements Iterable. But if clojure.lang.PersistentList implements Iterable (like it sounds like it does) then I can just return (get-in @db-atom path) .

Alex Miller (Clojure team)18:04:11

most seqs are Iterable so you're probably set in most cases just returning one

👍 1
rayat21:04:53

No clue which slack channel would be better for the following apart from this one: I'd love to get the new-ish github feature where you can directly click on symbols to get jump to Definition/References when viewing code. I know it involves the github libraries Tree-Sitter and CodeStacks (or something like that, not at computer rn). Would anyone be interested in exploring contributing clojure to the set of supported languages? No idea how herculean it might be tho tbh

Alex Miller (Clojure team)21:04:31

I feel like it's been mentioned and someone is working on it? Maybe hit the search here

1
Cora (she/her)22:04:27

I imagine you'd need to get clojure as an official language in tree-sitter first?

Cora (she/her)22:04:45

afaik this is the most popular tree-sitter clojure attempt https://github.com/sogaiu/tree-sitter-clojure

Cora (she/her)22:04:13

apparently the scope is a problem due to supporting "higher level constructs" https://github.com/sogaiu/tree-sitter-clojure/blob/master/doc/scope.md

Cora (she/her)22:04:37

and of course Michiel has experimented with it https://github.com/borkdude/analyze-reify

😆 1
❤️ 1
rayat00:04:58

Not surprised that he has hahah! I'll take a look around here and in those repos. Cant wait to spread myself thin with how much opportunity there is in the clojure DX effort !

💜 1
😅 1
rayat00:04:09

oh wait there's #tree-sitter! It's not exactly pertaining to using it for GH DX, but prob a first order requirement to get to that anyways

didibus02:04:31

I feel the issues they mention might not matter that much for just jumping around inside GitHub.

1
Cora (she/her)03:04:08

I think if you're talking about defs it probably will? I mean jumping around is usually "jump to definition" and "jump to references" and so supporting defs is required, I'd think?

didibus06:04:47

Ya, I mean false positive around supporting it which they talk about.

chancerussell23:04:06

Dumb, jokey, trivial question—does anyone know why java.lang.AutoCloseable doesn’t seem to be resolvable without a namespace (unlike the other java.lang interfaces and classes)?

chancerussell23:04:10

$ clojure -M -e "(println Cloneable) (println Iterable) (println Runnable) (println java.lang.AutoCloseable) (println AutoCloseable)"
java.lang.Cloneable
java.lang.Iterable
java.lang.Runnable
java.lang.AutoCloseable
Syntax error compiling at (REPL:1:93).
Unable to resolve symbol: AutoCloseable in this context

chancerussell23:04:19

Oh, it may just be anything newer than 1.6

chancerussell23:04:12

Forgot these were brought in with that big map in clojure.lang.RT

Alex Miller (Clojure team)23:04:47

Yeah, it's just missing. We have a patch for this, keep forgetting to get it in a release

jmckitrick23:04:45

What is the best way to unit test a ring response that returns a stream?

emccue23:04:44

slurp the stream?

jmckitrick02:04:04

Ah, nice! I see that in the notes now. I’ll give that a shot!

jmckitrick12:04:42

Hmm, no good. The slurp just returns a text-based rendering of the stream object: #object[java.io.PipedInputStream 0x37ca9a1c “.PipedInputStream@37ca9a1c”

emccue13:04:35

...huh what

emccue13:04:41

which definitely supports an InputStream

jmckitrick13:04:00

I agree, it makes sense on paper. But I’m getting empty string content with no reflection warnings or exceptions, etc.

emccue13:04:14

oh so with a PipedInputStream there is another thread dumping contents - maybe its a race condition?

jmckitrick13:04:35

Hmm. Not sure how to address that one. It’s a unit test, so even if the app spins off a thread, the response should still be on the original thread. At least that makes sense to me. If it’s not, I’m not sure what to try next.

jmckitrick15:04:21

So I found out I cannot slurp the stream from the response body returned during the unit test, but I can slurp from the stream itself if I break the ring-io/piped-input-stream call out into a separate function and return that in the unit test.