Depending on what behaviour you would like the readers of the globals to be, you could use a different construct. Clojure provides a promise that you can deliver. Maybe something like

(def init-state (promise))

(defn init! []
  (deliver init-state :value))

(defn do-work []
  (str "Tada! " @init-state))

(comment

  (init!)
  (do-work)

  )

Another option might be to just use a delay to initialise the state. This would run the initialisation code once on first deref and mean you don't have to call (init!)

(def init-state (delay (do
                         (prn '>> 'init-state)
                         {:something (+ 1 2 3)})))

(defn do-work []
  (str "Tada! " @init-state))

(comment

  (do-work)

  )

Blocking would not be a good experience for the users and specifying the timeout every time I use the value internally would not be ergonomic. Though, I could set a default value to something that derefs to a thrown exception.

delay would not work in my case because the native library passes a pointer that cannot be determined beforehand

you could also memoize the function?

Isn't that the same idea as with delay?

but that would turn it into a lookup, essentially. Meaning users would need to have a reference to the pointer.

... I didn't explain my idea very well. I meant write a function that turns p-get-proc-address into the data structure you need, and memoize that

the blocking would only happen first time on the promise. I guess that you could write a wrapper function that did the deref + timeout + exception throwing for them?

I guess it very much depends on what you mean by "ergonomic" 😉

My criteria: - the users shouldn't be worried about that value's existence - it isn't cumbersome to manage in code - is testable I think the promise idea is the best idea at the moment, though, instead of using promises, I could set a function that throws exception on deref.

I think the "best" thing to do is often to not have global state, and make each function declare it's dependencies. I think that means rejecting your first criteria. The users do care about the value that they get from derefing the promise, or however they get that value. The semantics of what they do if that value isn't available, will help you choose a construct.

But why globals? it is in general not the best idea to ship a library that requires user to "initialize" some global state

it might be easiest to separate out all the pieces into different functions and then orchestrate them at the end.

> it is in general not the best idea to ship a library I'm not sure that the question is about shipping a library. It could be just be an application.

I understand that global variables are to be avoided but I believe there is no way around it. I am making godot-clojure, a library that lets users call Godot functions via Clojure and that interop has to go through C and that requires me to have a global handle that I can use to call the corresponding C functions.

can your library initialize globals itself? without a request from the user

I suggest creating a "context" object (with a constructor function) that you pass to each of the functions that require c interrop. Then it's up to the consumer of the lib to manage that state in whatever way makes most sense to them.

> can your library initialize globals itself? without a request from the user no because I need a function pointer from Godot which is only known when Godot starts running

got you

so the user starts godot process and after that can call api functions, right?

yes, exactly. The idea is that the user defines a hook which is the first thing that Godot calls and the low-level details are handled behind the scenes

> the users shouldn't be worried about that value's existence this sounds very much in conflict with > the user starts godot process and after that can call api functions Could the process be: 1. user stats godot process 2. user constructs context obj 3. user calls lib functions using context obj as param Then the state is explicitly handled by whatever mechanism the user wants?

and if it falls over or fails, they're in charge of how it restarts and so-on?

instead of exposing top-level functions I would expose a protocol with an implementation that wraps p-get-proc-address as far as I can tell, your lib exposes call! and unsafe-call! public functions. They can become a part of that protocol.

Passing around that state could be a solution although that would mean that the user has to bring it everywhere because almost all useful functions would require the state. If the context changes then that would mean Godot would have restarted and spawned a new Clojure process. Protocol and passing around the context might be the best option, thanks for all the help!

(also call! and unsafe-call! are low-level functions and the "real" api would be higher-level but the idea stays the same)

it means you need to restart your repl

I tried, same result.

tech.v3.datatype.native_buffer.NativeBuffer cannot be cast to class tech.v3.datatype.native_buffer.NativeBuffer is saying a classes with the same name are not actually the same class, which happens because you loaded some code that defined that class, made an instance of the class, then reloaded the code that defines the class got a new class with the same name, then tried to use the old instance with code compiled to expect the newly defined class

what does "tried" mean?

what are you doing that results in that error?

do you have a user.clj? are you using and kind of auto code reloading? are you calling clojure.core/compile some where?

could maybe also be a stale aot class file issue, where the class is being imported somewhere, but the clojure code that defines the class is not being loaded first, so that import gets an old version of the class from disk, then the clojure code is loaded and it defines the class again

> what does "tried" mean? I did what you suggested and it didn't fix the issue. > what are you doing that results in that error? It happens when I call the open-db fn here:

(ns com.example.rad.components.database
  (:require
    ; [com.fulcrologic.rad.database-adapters.datomic-cloud :as datomic]
    [mount.core :refer [defstate]]
    [com.example.rad.model-rad.attributes :refer [all-attributes]]
    [com.example.rad.components.config :refer [config]]
    [tmducken.duckdb :as duckdb]))

(comment
  ;; FIXME: move paths n stuff to config
  (duckdb/initialize! {:duckdb-home "/tmp/tmducken/binaries/"}) ; true
  (def db (duckdb/open-db "/tmp/duck.db"))
  (def conn (duckdb/connect db)))

(ns user
  (:require
    [clojure.pprint :refer [pprint]]
    [clojure.spec.alpha :as s]
    [clojure.tools.namespace.repl
     :as tools-ns
     :refer [disable-reload! refresh clear set-refresh-dirs]]
    [expound.alpha :as expound]
    [portal.api :as portal]
    [taoensso.timbre :as log]))

(alter-var-root #'s/*explain-out* (constantly expound/printer))

(def p (portal/open {:app true
                     :theme :portal.colors/material-ui}))
(add-tap portal/submit)

I notice I also get these warnings when starting the repl:

; Assumed session: Toyger (Unknown )
; (err) Reflection warning, tech/v3/dataset/io.clj:60:3 - reference to field close can't be resolved.
; (err) Reflection warning, tech/v3/dataset/io.clj:60:3 - reference to field close can't be resolved.
; (err) Reflection warning, tech/v3/dataset/io.clj:60:3 - reference to field close can't be resolved.
nil

I dunno that I have the patience to try and untangle the techascent libraries to figure out where NativeBuffer is defined, but the issue is ultimately that the code that defines it is being run twice, resulting in to incompatible definitions

there are many ways this can happen, the two big ones are some kind of auto-reload tooling or aot compilation

(by incompatible definitions I just mean you end up with two classes with the same name, which is possible on the jvm because class identity is name+classloader)

ah, it comes from https://github.com/cnuernber/dtype-next not under the techascent github

What is the significance of that finding, @U0NCTKEV8?

if you don't know where the code is coming from how can you read it?

if you add a require of tech.v3.datatype.native-buffer it might make the error go away

(as early as you can before any other code that might use it is loaded)

Nope, same deal. I'll make a minimal repro to isolate it.

@U0NCTKEV8 requiring tech.v3.dataset before tmducken.duckdb seems to have fixed it! The example in https://github.com/techascent/tmducken does it in that order but it wasn't obvious to me that it's required.

it shouldn't be required, it is a symptom of something being very broken, and a hack to get around it

something in the dependency tree may be aot compiled (a problem for libraries) which generates a class file for NativeBuffer, and then something else is importing it without requiring the clojure namespace that defines it

Weird, I could not reproduce in a minimal repro; I was able to just require tmducken.duckdb without tech.v3.dataset. Could this be a case of when blowing away my .m2 might just magically fix it?

unlikely

how are your deps specified? project.clj, deps.edn?

if you are using deps.edn and on linux something like clj -Spath | sed 's/:/\n/g'|grep jar|xargs -n1 jar -tf|grep NativeBuffer will tell you if some dependency has a NativeBuffer.class somewhere

@U0NCTKEV8

$ clj -Spath | sed 's/:/\n/g'|grep jar|xargs -n1 jar -tf|grep NativeBuffer

tech/v3/datatype/NativeBufferData.class

you can add something like (alter-var-root #'clojure.core/*loading-verbosely* (constantly true)) before your ns form, which will cause it to print out all the namespaces as they are loaded, and you might see tech.v3.datatype.native-buffer loaded twice

yeah, looks like something is causing tech.v3.* namespaces to get loaded over and over

that'll break stuff

Looks like native-buffer appears ~900 times in that output in these three variants:

; (out) (clojure.core/alias 'native-buffer 'tech.v3.datatype.native-buffer)
; (out) (clojure.core/alias 'nbuf 'tech.v3.datatype.native-buffer)
; (out) (clojure.core/in-ns 'tech.v3.datatype.native-buffer)

the aliases are what matters, the lines that matter are like ; (out) (clojure.core/load "/tech/v3/datatype/native_buffer")

which happens maybe 8 times

Is this somehow my fault or is it something that should be reported to the tmducken repo?

dunno

seems unlikely to be your fault, but it could be some tooling you are using, you should try and replicate without nrepl and cider and without the user.clj

since it seems to be effecting text.v3.* namespaces only, it makes me wonder if they have some stray :reload-all directives floating around

$ clj
Clojure 1.10.3
user=> (clojure.core/require 'com.example.rad.components.database)
Reflection warning, tech/v3/dataset/io.clj:60:3 - reference to field close can't be resolved.
Reflection warning, tech/v3/dataset/io.clj:60:3 - reference to field close can't be resolved.
Reflection warning, tech/v3/dataset/io.clj:60:3 - reference to field close can't be resolved.
2024-06-07T18:33:32.001Z xps INFO [tmducken.duckdb:326] - Attempting to load duckdb from "/tmp/tmducken/binaries/libduckdb.so"
Execution error (ClassCastException) at tech.v3.datatype.ffi/make-ptr (ffi.clj:587).
class tech.v3.datatype.native_buffer.NativeBuffer cannot be cast to class tech.v3.datatype.native_buffer.NativeBuffer (tech.v3.datatype.native_buffer.NativeBuffer is in unnamed module of loader clojure.lang.DynamicClassLoader @598464b2; tech.v3.datatype.native_buffer.NativeBuffer is in unnamed module of loader clojure.lang.DynamicClassLoader @5fc44b4b)
user=> 

is that without the user.clj aswell?

Yes, it is. I was able to reproduce in a new project as well, but it turns out that upgrading from clojure 1.10.3 -> 1.11.1 made the problem go away 😕

oooo

I just started commenting out deps in deps.edn and when I got to org.clojure/clojure {:mvn/version "1.10.3"} so it defaulted to my local 1.11.1, I saw the problem was gone.

🤷

Thanks very much for your patience and help, @U0NCTKEV8. I noted your suggestions to try *loading-verbosely* and -Spath parsing to review in more detail later.

dynamic vars have a global root value and a per-thread override

set! changes this thread's binding

alter-var-root changes the global root

Ah, I see. Thanks 👍:skin-tone-2:

the other non-obvious thing is that set! only works if binding has been used in the current thread context to set a per-thread value. The repl itself sets a bunch of core dynamic vars before control is handed to you, which is why you can (set! *warn-on-reflection* true) but may not be able to set! other dynamic vars

(see https://github.com/clojure/clojure/blob/master/src/clj/clojure/main.clj#L76-L100 for that code and the bindings set)

If binding hasn't happened, then would alter-var-root be the only way to replace the value of a dynamic var with something else? Of course assuming it's correct for that value to apply across all threads.

you can use binding of course

vars have dynamic per-thread scope - using binding will define that dynamic scope as "inside this block and anything it calls"

so you can (binding [*warn-on-reflection* true] (do-some-stuff)) and reflection will be checked in everything called by do-some-stuff

also, some multi-threaded contexts (like future) convey the current thread bindings to the executing thread

Most likely you tried to eval (add 1 2) without eval'ing (defn add ...)

😮 ohh thanks; it worked, I thought the calva repl actually runs from the top also, how do you get that idea that I didn’t evaluated the (defn add …) from the stacktrace? How can I read that in this scenario?

The error Unable to resolve symbol: add in this context indicates the running REPL doesn't know about add so it seemed likely that you hadn't eval'd the defn form.

My recommendation is to eval each top-level form as you write it or change it -- alt+enter (Windows/Linux) opt+enter (macOS) -- and just get into the habit of doing that whenever you make any changes.

is there a way by which I can eval every statement in the file? Run all kind of thing?

ctrl+alt+c enter (`ctrl+opt+c enter` on macOS) will (re)eval the whole file. You could also use ctrl+shift+alt+enter (`ctrl+shift+opt+enter` on macOS) to eval from start-of-file to cursor -- be aware that will close brackets at the cursor if you're not at the end of a top-level form.

I often work for extended periods without even saving files, and just press alt+enter any time I make any change, even across multiple files.

Thanks, this helped a lot 🙏

I’ll follow the same

There’s a command in Calva that starts a small REPL tutorial, @U076UKX1ACQ. Calva: Fire up the Getting Started REPL. Just FYI. I think I will add something about understanding that message Unable to resolve symbol: foo in this context. It has gotta be something a lot of beginners face.

ns-publics does this for public vars (which would include functions too), but only in a single namespace

There's a bunch of ways to do this. You can use something like codox to create a fancy webpage. You could use clj-kondo's static analysis to get the data for programmatic usage. You could use https://github.com/borkdude/grasp. Grep works. As @UFTRLDZEW suggested, you could use the various tools available at runtime.

please use a thread instead of separate messages for one question... As long as the arg to swap! is a function, it should be doable to use a threading macro. It would probably help to boil down a self-contained example and share some code so it's clearer what you mean by 'use x with y'

I would be extremely skeptical of multiple swap! calls in a row on a single atom

Very likely introducing a possible race condition

You'll want to combine all the changes into a single function. (swap atm #(-> % <changes>))

(let [atm (atom {:foo 42})]
  (swap! atm
         #(-> %
              (assoc :bar "bar"
                     :baz :baz)
              (update :foo inc))))
;; {:foo 43, :bar "bar", :baz :baz}

Ah ok , I have multiple swaps 🙂 . Let me try...

One of the fun parts of learning clojure is that you realize that you can write the same programs with dramatically less state. One trick is combining multiple changes that should happen at the "same time". Another is that you often find that you didn't need the atom in the first place.

Sure , but the hard part for me is that is just give you a compilation error and that's it .

but thank you , I need to rewrite my functions and see if it helps 🙂

That's fair. I think something that really helps is having a good setup that let's you write and run your program in small pieces (ie. repl driven development). It's possible you're already setup, but it can be a big shift depending on what development setup you're used to.

e.g,. Calva has an option to eval the current form to the cursor position, adding closing parens as needed. This lets you take a threading form and put the cursor after each successive subform in the threading pipeline and eval to that position. That makes it a lot easier to debug things like this.

Ha !

It works

(-> (swap! my-atom ...)| (swap! my-atom ...) (swap! my-atom ...)) where | is the cursor and ctrl+alt+enter will eval (-> (swap! my-atom ...)) - then move the cursor to after the next swap! call and ctrl+alt+enter will give you that failure. But you'll see that the value of the first eval is a hash map and not an atom, which is the hint.

I am getting better with the brackets . This problem was syntax related I just used the threading macro and called two functions . Each function had a swap statement

Now I removed the functions and just call assoc-in two times and it works 🙂 , but I haven't seen this syntax

Since you didn't post your original code, we're just guessing based on the error you posted 🙂

And both of us are suggesting techniques to help you debug this sort of error on your own -- eval'ing small pieces of code into the REPL so you can see how a larger expression is working (or not).

Thats is what I also try to do . Check if the smallest peace of your function works and then go to the next layer .