Fork me on GitHub
#clojure
<
2023-09-05
>
roklenarcic08:09:29

If I have this: (map :id seq-of-huge-maps) and if the whole sequence is realized, will the huge data be GCed? In other words, does the source of the lazy sequence go away after it has been realized?

p-himik08:09:15

It will be, if nothing else holds on to the huge seq.

jeeger09:09:44

"Don't hold on to your head" ☺️

pavlosmelissinos11:09:07

Nice question! What does "hold on" mean though? Like if the seq is a value of a map and you pass around the map without dissoc'ing the seq even though you no longer need it, will the seq still never be GCed because the map is used? Or does the code have to actually use the seq?

p-himik11:09:02

The seq doesn't have to be used in the sense of operations running on it directly. But it has to be used in the sense that GC sees references to it.

👍 2
jeeger09:09:32

I'm working on a smallish clojure project with some Java interop, and I'm trying to build a native image with GraalVM and Nix using https://jlesquembre.github.io/clj-nix/. Two questions: • What's the standard way to enable *warn-on-reflecion* ? It seems lein check gives me some warnings that running with *warn-on-reflection* doesn't give me. Currently, I'm just doing (set! *warn-on-reflection* true) in my main namespace (not inside the main function). • GraalVM seems to be substantially more demanding than *warn-on-reflection*. Is there any way of checking the required type hints without trying to recompile with graal, which takes some time.

jeeger09:09:31

To be more precise, I'm having issues with builtins like File/exists and SocketChannelImpl/close.

p-himik09:09:07

*warn-on-reflecion* is per namespace. Setting it in just one place will only enable it for that place. If you're using project.clj then it seems that you can add :global-vars {*warn-on-reflection* true} there.

👍 2
jeeger09:09:47

Ah, I'm using deps.edn.

p-himik09:09:40

There's also the clojure.compile.warn-on-reflection property that you can set to true but I haven't used it myself.

pithyless09:09:08

Should work if you add to user.clj

(alter-var-root #'*warn-on-reflection* (constantly true))

👍 2
p-himik09:09:08

IIRC, that will enable the warning for any loaded namespace, including the libraries.

jeeger09:09:23

Ah, using an extra main-arg in the run profile works as well.

jeeger09:09:34

Got it running with GraalVM, that's pretty nice.

vemv11:09:50

https://github.com/clj-easy has misc projects that can further help you understanding/preventing issues with graalvm

lread13:09:54

The clj-easy repo I'd start with is https://github.com/clj-easy/graal-docs. There is also a #graalvm channel.

2
Martynas Maciulevičius10:09:52

I don't understand what I'm doing wrong:

{:joined-at #inst "2020"}
=>
{:joined-at Wed}
Wait, what? There is a reader function #inst but it's not reading what it's supposed to :thinking_face: How could this happen? The Wed is a symbol... If the reader function wouldn't exist I'd be getting an error. But now not only it exists but it returns a very weird value. 😨 The thing is that my clojure version is pinned to this: org.clojure/clojure {:mvn/version "1.11.1"} I use babashka tools.deps and launch it from babashka bb.edn file. If I open a leiningen project (that one has [org.clojure/clojure "1.10.1"]) then it prints this:
{:joined-at #inst "2020"}
->
{:joined-at #inst "2020-01-01T00:00:00.000-00:00"}
I don't understand why the babashka version could do this 🤷 How do I fix this? I don't want to migrate to Leiningen.

p-himik10:09:13

Seems like it belongs in #CLX41ASCS.

Martynas Maciulevičius10:09:45

But the problem is that this is JVM REPL

p-himik10:09:41

You're saying in the OP: "I use babashka for this". Babashka is not a JVM REPL.

Martynas Maciulevičius10:09:17

I use clojure tools deps. Sorry. This is what I meant. I'll fix it.

p-himik10:09:57

How exactly do you run your REPL? What is the minimal deps.edn that still exhibits this behavior?

Martynas Maciulevičius10:09:22

This started happening when I updated my system today. It printed a message that ~/.deps.clj doesn't exist (then I thought "ok, I haven't seen this directory before") and then it proceeded to download a couple of JARs. It happens on an older project too that worked before. So I don't think it's a project config. I tested on two tools.deps projects. I'll try to come up with a minimal example.

Martynas Maciulevičius10:09:58

When I copy deps.edn file into a new project without any changes (with everything intact) then it doesn't happen in a new project. On the old project it still fails if I use the clojure -M command like this (but not on fresh project). So it's surely not a babashka problem:

clojure -M:project/dev:project/debug:nrepl/interactive
I'll try to remove ~/.clojure and ~/.deps.clj directories. Maybe that could solve it.

Martynas Maciulevičius10:09:40

Removing of .cpcache didn't help. Moving ~/.clojure and ~/.deps.clj directories didn't do anything. Moving ~/.m2 directory helped.... I nuked my .m2 directory twice during last 30 days... what's going on... Maybe it's my SSD

p-himik10:09:48

No idea. I haven't done anything to that directory in at least the last 5 years. Do you ever use mvn directly, specifically mvn install...? Do you depend on any snapshots, perhaps transitively? Next time something like this happens, instead of nuking random stuff I'd suggest trying a stepping debugger in order to understand where that bogus inst reader came from.

Martynas Maciulevičius10:09:29

> I'd suggest trying a stepping debugger in order to understand where that bogus inst reader came from. I don't really have a real debugger in my nvim. But I still have .m2 directory. I just renamed it.

p-himik10:09:00

> I don't really have a real debugger in my nvim. Then it sounds like the first step should be getting such a debugger. :) Venturing outside of your main IDE for specific pieces of functionality is totally fine.

👍 2
p-himik10:09:30

Also, while a stepping debugger is a better tool for some tasks, you can still do a lot in a plain REPL. In this particular case, there are *data-readers*, default-data-readers, and *default-data-reader-fn*. They are checked in this order. When you encounter a problem like this, you can always check where the inst tag reader is specified and inspect the value. The corresponding function, if there is one, will have some info about it (even if it's just its class name, which under normal circumstances includes the namespace).

👍 2
vemv11:09:22

> clojure -M:project/dev:project/debug:nrepl/interactive Maybe that's adding cider-nrepl? I hadn't heard of it formatting insts oddly, but some things are certainly printed in a custom way

vemv11:09:05

(->> (-> (Thread/currentThread)
         (.getContextClassLoader)
         (.getResources "data_readers.clj"))
     (enumeration-seq)
     (distinct)
     (mapv str))
This will give you all the data readers in your classpath

❤️ 2
Martynas Maciulevičius11:09:48

> Maybe that's adding cider-nrepl? It was. I got rid of all the directories as I want to do other things. But that's a good snippet. But then I didn't have any of the middlewares set up in my tools.deps global configuration files. And it actually downloaded some JARs into a new directory.

vemv11:09:14

data readers are loaded simply by virtue of being in the classpath

vemv11:09:38

happy you got it. TIL too (as I never thought of using this snippet for data_readers)

Martynas Maciulevičius11:09:43

> data readers are loaded simply by virtue of being in the classpath I was playing with them long time ago and they also have to be in a special location, I think. Or those namespaces have to be imported to execute the top-level code.

p-himik11:09:21

Note that data_readers.cljc are also loaded.

👍 2
Martynas Maciulevičius11:09:21

> Note that data_readers.cljc are also loaded. Exactly. I think it loads this file if it's in the project first. And then proceeds to load everything else. This way the codebase won't start misbehaving randomly. It's not a very good idea to make your own. If they'll sometimes fail to load you won't be able to read your EDN.

👍 2
p-himik12:09:25

The loading order doesn't matter and isn't forced since conflicting tags are reported as an exception.

adi18:09:52

Is there a cli deps command to enumerate all the alias names defined across deps.edn config files? I can locate the deps.edn files being read, via clj -Sverbose but can't for the life of me figure out the cli invocation to print out the names of :aliases from those files. I have scoured the https://clojure.org/reference/deps_and_cli, and my search-fu has come to no avail. I must be missing something blindingly obvious.

ander19:09:02

This was a little surprising to me as well. This part of the documentation helped: https://clojure.org/reference/deps_and_cli#_other_programs I suppose the invocations listed under 'Other programs' are sort of special cases?

adi19:09:52

Oh man, I can't believe I missed that. It was there all this time, after all... time to walk away from the computer and go look at the grass or sky or something.

adi19:09:35

Incidentally I noticed the section for "Aliases"[1] trails off at "These Clojure CLI subprocesses use data which can be read from aliases:" I suppose "subprocess" refers to one of tool / resolve-deps / make-classpath-map etc [2]. [1] https://clojure.org/reference/deps_and_cli#_aliases [2] https://github.com/clojure/tools.deps/blob/be30a1ae275eabfd1eba571080e039451c122c69/src/main/clojure/clojure/tools/deps.clj#L807

adi19:09:20

If yes, I'm happy to submit a patch per the "small change" contrib instruction.

practicalli-johnny19:09:37

I only recently discovered clojure -X:deps aliases after looking at the api and spotting the aliases function. I was planning to submit an issue or pr to add it to the clojure --help output which would make it easier to discover

Alex Miller (Clojure team)19:09:41

if it's in tools.deps, or brew-install, probably easiest to file a question at https://ask.clojure.org

👍 4
adi21:09:06

I took the liberty to send a "small changes" patch to update the deps and cli "Aliases" reference at clojure-site https://github.com/clojure/clojure-site/pull/669/files