Fork me on GitHub
Jakub Holý (HolyJak)08:10:48

Hello! I struggle finding a good way how to perform parametrized initialization only once in the face of concurrent calls. What I have now is:

(defn get-or-init-adoc [atom params]
  (compare-and-set! atom nil (delay (some-expensive-init-reading-files params)))
Is there a better way (then delay inside an atom)? What I essentially want is (atomic-if (initialized? x) x (init+return x, params)) where x cannot be changed between I check whether it is initialized and initializing it. Then, inside a frequently called function I do (defn render [params file] (.convert (get-or-init-adoc adoc params) file))


what's wrong with just having a (def x (delay (some-expensive-init params))) ?


where are the params coming from?


there's also promise which you can deliver once and will block all defref s until it's delivered ... but you need to be careful not to create deadlocks where the read calls block all the running threads before any thread can be started to deliver the promise


or maybe (defn get-or-init-adoc [atom params] (swap! atom #(when (nil? %) (some-expenisive-init params)) ?


I guess you're trying to make sure that the init only gets run once, and have the function lazily initialise some value?


so maybe you could memoize the function?


note that memoize is not suited for non-referentially transparent functions


and it doesn't guarantee that the underlying function will be executed once


you could use locking to lock some sentinel, which will probably be the least headachy


I don't really see the benefit of putting an atom outside the delay here, the delay already locks and will only execute once

Jakub Holý (HolyJak)18:10:26

The problem is I need to do the initialization inside a function that gets 'params' as it's argument. And I don't want to unnecessarily create a new delay upon each its call.

Jakub Holý (HolyJak)18:10:16

swap! doesn't work because it can execute the function multiple times.


agents never retry, fwiw


so sending an fnil to an agent should work

👍 6
Jakub Holý (HolyJak)20:10:48

Thanks! I guess careful use of locking is the simplest solution here...

Eugene Koontz14:11:38

I had the same problem (doing something slow and expensive that should only be done once per process) and came up with the following solution:

Eugene Koontz14:11:58

and then every usage of the model calls (load-model) , although the actual loading is only done once : subsequent calls to load-model simply deferences the ref


@U017Y2J9F0W delay might be a simpler way to get that result (depending on whether you want to re-execute)

Eugene Koontz19:11:02

ahh, I do want to re-execute, so I can reload the models, so a delay won’t work, as you said. So my original code would be better.


also, there are dedicated libraries for stateful initialization, which take things like cross-ns dependencies and re-initialization into account - eg stuartsierra/component and integrant

👍 3
Eugene Koontz19:11:09

i feel like I have to work through the primitives (what the core clojure libraries provide) to even appreciate the dedicated extra libraries


Do you know what's a free service to quickly deploy a Clojure Deps.edn application?


I would normally use Heroku but they do not support Clj tools out of the box and I need to do to many changes to my project


Make an uberjar and deploy that one


Yeah I guess that's the easiest way. Any chance you have a link about producing an uberjar with @zilti


2020-10-29T14:47:15.148369+00:00 app[web.1]: Error: Invalid or corrupt jarfile app.jar


Interestingly enough, the deployed jar on Heroku seem to be broken or something :thinking_face:


@vincenz.chianese Does it work locally though?


Yeah, I think I made a mistake in the jar execution, let me try again 🙂


I was specifying a wrong namespace


Ok that works now, but it seems like I'm having issues running datomic dev-local on heroku


the name doesn't suggest you should run it in production maybe? just guessing


No, I think it's because I'm using a wrong directory name, let me triple check…


Ok I got it working. It takes ages to startup but eventually it works


Hi does anyone have an idea of how I would do this? I’m trying to conditionally define functions using a global lein profile depending on whether I have certain libraries loaded, ex. something like this where I’m developing several libraries at once, some of which are childrens of the others. Say if the dependencies looked something like

[my.lib5 [my.lib1 my.lib2 my.lib4]
 my.lib4 [my.lib1 my.lib2]
 my.lib3 [my.lib2]]
and basically the workflow would be something like while i’m in a repl for my.lib3, to have all the custom function definitions defined using variables defined in my.lib3 and my.lib2 and so on. So I have something like this being loaded via {:repl {:repl-options {:init (load-file "~/.lein/startup.cljc")
;; load libs if they exist
(let [repl-deps ['[my.lib1 :as l1]
                 '[my.lib2 :as l2]
                 '[my.lib3 :as l3]
                 '[my.lib4 :as l4]
                 '[my.lib5 :as l5]
                 '[clj-async-profiler :as prof]
                 '[criterium.core :as crit]]]
  (doseq [dep repl-deps]
      (require dep)
      (catch Exception e nil))))

;; Create custom functions depending on what's loaded
(let [namespaces (->> (all-ns) (map ns-name) (map name) set)
      loaded? (fn [& args] (every? namespaces args))]
  (when (loaded? "my.lib1")
    (defn custom-fn-1 (l1/foo) )
  (when (loaded? "my.lib2")
    (defn custom-fn-2 (l2/bar)))
  (when (loaded? "my.lib2" "my.lib4")
    (l4/init l2/options)
    (defn custom-fn-3 (l4/baz)))
But it doesn’t work because I get a compile error No such namespace: l4 when in a repl for my.lib1, my.lib2 and my.lib3 . So I’m wondering, is there anyway to solve this without too much mucking about? Or is the only way to solve this to have a compiler option *allow-unresolved-namespace* in the compiler which would be used in resolveIn in

borkdude20:10:47 This is called the Gilardi scenario:


I could have sworn he was on Slack here as well as @sgilardi

Darin Douglass22:10:35

i actually asked him (since i was pairing with him at the time) if "Gilardi scenario" was a coincidence. totally wasn't 😛


why would it be a co-incidence?

Darin Douglass22:10:28

I figured given the community it’d be pretty unlikely it WASNT him. But still wanted to ask


nice little bit of nostalgia courtesy of @technomancy and you two. :)


I'm honoured to finally meet the person behind the name :)


🙂 thank you! I’m honored and happy to meet you! Thanks so much for your work and initiative in producing such useful and clever tools for us.


OMG. Mr Gilardi Scenario himself!


Gilardi issue is secondary symptom - the root issue is you are throwing away your exceptions . Who knows what that is suppressing


But a good hypothesis is that lib4 is not loading initially, being skipped silently, causing a subsequent compilation error later (failure to resolve symbol l4/init )


but also good to understand the gilardi scenario


Umm forgive me if I’m misunderstanding but I don’t think the Gilardi scenario is related, I’m working on a project that has about 20 different individual projects, and basically, sometimes I will be able to do a require on that my.lib4 but sometimes I won’t (because it’s not a dependency)


and I want specific blocks using variables from my.lib4 to not be run


you can't compile a form containing the symbol l4/init even if its behind a when form if the l4 alias isn't present


but you are attempting to


I’d like to, that way I can have it all in one startup.clj file


and then say if I have multiple projects that use criterium, I can have those all load


only when its avaialble


if that makes sense


right. and that seems to be exactly the Gilardi scenario


hmm okay, maybe i’ll take another look at it, thanks everyone


i think you could easily get around this with requiring-resolve

👍 3

^ I was about to suggest that.


Bear in mind that it will throw an exception if the namespace can't be required.


Right right, because it would be a runtime thing and okay, after a second reading it totally makes sense now thanks again everyone


(when-let [foo (resolve 'l1/foo)]
  (defn custom-fn-1 (foo)))
I think that would work

👍 3

That way, it will only resolve if l1 is loaded. Well, if it's loaded and l1/foo resolves 🙂


with the understanding that it's not the best practice and for good reason, is there a way to fetch the latest commit of a branch via deps.edn, rather than a specific sha?


@joshkh I have an example in my dot-clojure repo, if you've seen that?


nope! searching now


exactly what i was looking for. just to test my understanding, extra-deps brings in tda:add-lib3, which then handles the :add-libs key of deps.edn? nvm i misread the indentation


(relies on the add-lib3 branch of t.d.a.)