integrant

lispyclouds 2022-10-04T11:50:48.505409Z

Hello! Is it possible to ignore some top level keys when loading an edn file? for example im using a mix of integrant and aero and would like integrant to ignore the :common key:

{:common                #include "bob/common.edn"
 :bob/storage           #ref [:common :bob/storage-conn]
 :entities/queue-config {:database #ig/ref :bob/storage}
 :bob/queue             {:url      #ref [:common :bob/queue-conn :url]
                         :user     #ref [:common :bob/queue-conn :user]
                         :password #ref [:common :bob/queue-conn :password]
                         :conf     #ig/ref :entities/queue-config}}
currently i have this to ignore it
(defmethod ig/init-key
  :common
  [_ _])

wevrem 2022-10-04T14:57:14.139019Z

Does that work? Do you not need to return the value (even if unmodified)?

(defmethod ig/init-key :common [_ val] val)

lispyclouds 2022-10-04T14:58:56.656259Z

Since I dont use it anywhere with a #ig/ref it works 😅 otherwise yeah it shouldn't i think

lispyclouds 2022-10-04T15:00:19.488889Z

since im using it with aero to reuse conf hence its a bit clunky. the :common is purely a map which is used only by aero

wevrem 2022-10-04T15:01:22.321559Z

I’m doing something similar, but never realized that I was never using ig/ref to call into :secrets (that is what mine is called that is doing the same role as your :common)

wevrem 2022-10-04T15:03:10.268139Z

I guess after you read it in with aero, but before passing it to integrant, you could remove :common and any other top-level keys that integrant doesn’t need.

lispyclouds 2022-10-04T15:03:55.956909Z

yeah i was thinking of that too, was thinking if theres some other/declarative way. 🙂

lispyclouds 2022-10-04T15:05:06.588899Z

will go with that for now, thanks!

wevrem 2022-10-04T15:08:01.653199Z

If you wanted it to be more ‘declarative’ (and maybe you’ve already thought of this, too) you could put some metadata on the value for :common (and any other keys you want to discard), read it in with aero, then scan the map for values flagged with the metadata and drop them, and then pass to integrant. That puts the info of ‘I don’t want this key in integrant’ in your config.edn file, and not in code. I don’t know if aero preserves metadata (hopefully it does?).

lispyclouds 2022-10-04T15:12:05.127139Z

Right, makes sense. Yeah didn't think that much 😛 Putting a dissoc with a TODO: make more declarative now and will try out your idea! Thanks again!

wevrem 2022-10-04T15:14:06.262109Z

Belay that. It might not be possible to include metadata in edn files. (I just tried a quick experiment.) But it might be possible with a custom aero reader tag that can be responsible to add the metadata as the object is created, and then do the same: look for the metadata and drop those values from the map before passing to integrant.

lispyclouds 2022-10-04T15:23:57.702449Z

right

joakimen 2024-01-05T16:04:45.799019Z

Just stumbled upon the same problem. Considering a) integrant and aero both seem like commonly used tools, and b) I can't find docs or examples explaining how to make integrant deal with aero "meta-keys" such as reading secrets, I get the feeling that I'm not supposed to solve the problem this way. Did you find a decent approach to this?

joakimen 2024-01-05T16:06:59.805499Z

(and yeah I know the thread is a year old, thought it'd still be worth asking)

wevrem 2024-01-05T16:57:13.099199Z

My current approach (probably based off the ideas that came about from this very thread, but I can’t remember for sure) is to organize my config.edn file into groups based on namespaced keywords. That is, my config map has keys like :config/secrets, :config/datomic-client, :config/postgres-spec, etc. And it has other keys like :services/datomic, :services/adapter, etc. Then I have these functions to load it:

(ns acme.config
  (:require [aero.core :as aero]
            [integrant.core :as ig]))

(defmethod aero/reader 'ig/ref [_ _ value]
  (ig/ref value))

(defn get-profile []
  (or (keyword (System/getProperty "profile")) :dev))

(defn config
  "Read `config.edn` from classpath using :profile from JVM sys prop."
  [profile]
  (aero/read-config (io/resource "config.edn") {:profile profile}))

(defn services []
  (into {} (remove #(= "config" (namespace (key %)))) (config (get-profile))))

;;; ------

(ns acme.main
  (:require [acme.config :as config]
            [integrant.core :as ig]))

(defn -main [& _]
  (ig/init (config/services)))

;;; ------

(ns user
  (:require [integrant.repl :refer [go halt reset]]
            [acme.config :as config]))

(integrant.repl/set-prep! config/services)

(comment
  (go)
  (halt)
  (reset)   ;; NOTE: mapped in VSCode/Calva to ctrl_alt_space r
)
Basically, before passing the config map to ig/init or set-prep! I strip out any entries where the key has namespace ‘config’. Then I don’t have to provide init-key and halt-key! methods for all of them. It’s also nice because my services are passed around all over in my app, but the original secrets are gone and can’t be accidentally leaked.

joakimen 2024-01-07T12:17:22.444949Z

Yeah precisely, the fact that we have to manually "clean up" the configuration prior to handing it to integrant feels like a hack to me

âž• 1
joakimen 2024-01-06T18:25:52.489659Z

I see, thanks for the thorough reply! I ended up dissocing the undesired keys prior to passing it to integrant as well, like in this example and the thread. It gives me somewhat of an itch though, that you are able to combine the integrant configuration and aero configuration in a single file by introducing the ig/ref reader, but that integrant stumbles when encountering some of the basic syntax of aero, like #include. I'll run with this for a while, but might revert to separating the integrant and aero configs. Thanks again!

joakimen 2024-01-06T18:27:17.341009Z

On a completely unrelated note, would you mind sharing how you assign keyboard shortcuts to forms/commands in Calva? I've looked for that, but haven't found it yet Might have found it, ignore me 🙂

wevrem 2024-01-06T21:14:12.312729Z

What do you mean by integrant stumbling? I process with aero first, so by the time integrant sees things, all the aero-specific stuff is handled/expanded/resolved and gone.