Fork me on GitHub
#beginners
<
2020-11-01
>
Louis Kottmann17:11:27

I am trying to list all files in a configs resources directory, read them as yaml, and merge it all in a config map during dev with CIDER, this works fine:

(def config (->> (io/resource "configs")
                 (io/file)
                 (.listFiles)
                 (map yaml/from-file)
                 (reduce merge)))
However at runtime, from the jar, it fails due to:
Caused by: java.lang.IllegalArgumentException: Not a file: jar:file:/path/to/app.jar!/configs
Indeed, it is not a file, it should be a directory. I checked and the files are present in the classpath (at the root, in the configs directory as I would expect) My google-fu turned out lots of complicated answers, but I can't believe this is not more straighforward, how should I achieve this?

practicalli-johnny17:11:37

My guess is because io/resources and io/file look in different places. io/file is from the root of the project or an absolute path. io/resources is relative to the project/resources directory.

noisesmith16:11:59

also, you can't use io/file for things inside jars - io/resource is relative to all classpath entries (which can include 0 or more actual directories on disk)

👍 3
Louis Kottmann17:11:18

ok I solved it using:

(def project-path (.getParent
                   (io/file (.getCanonicalPath
                             (io/file "project.clj")))))

(def config (->> (str u/project-path "/resources/configs")
                 (io/file)
                 (.listFiles)
                 (map yaml/from-file)
                 (reduce merge)))
but I'm not sure if it works because the files are present next to the jar files in the docker container, or if it actually fetches the files from the jar

teodorlu20:11:37

I believe what you've written will fetch files "next to the jar". To load from the jar, try io/resource

Louis Kottmann23:11:46

I did, but io/resource does not work with directories when running with an uberjar, I mean I did not find out how to use it properly

teodorlu10:11:18

Yeah, I can imagine that might give you problems. Java resources don't really have directories, in the same sense that normal file systems do. The classpath specifies how different directory trees should be merged, so what you're loading from is the "merged classpath from your project, and all of your dependencies". Looking at a stackoverflow answer to a Java question[1], I suspect you'll want to do something along the lines of

Thread.currentThread().getContextClassLoader().getResourceAsStream(yourPath)
, but I suspect other Clojurians might have encountered this problem before! [1]: https://stackoverflow.com/questions/3923129/get-a-list-of-resources-from-classpath-directory

Louis Kottmann10:11:27

yeah I've seen the jarFile.entries() before but that outputs a shitton of files (every file from every dependency + the resources of the project), and I'm looking for just one "folder"

teodorlu14:11:27

I guess that explains why projects "namespace their resource folders"; putting config files in resources/com/teodorheggelund/myproject rather than the top level. I don't see an easy way out for you, though, unless you can choose where to store the files.

Louis Kottmann15:11:21

I just copy the original files and load them as normal files, bypassing the resources shenanigans

Louis Kottmann15:11:08

and now consider resources to be a java idea gone haywire until further notice ^^

bpenguin19:11:08

I've got a question on something I saw recently in one of the explanations of https://findka.com/blog/essays-implementation/

(defn send** [api-key opts]
  (http/post (str "")
    {:basic-auth ["api" api-key]
     :form-params opts}))

(defn send* [{:keys [mailgun/api-key template data] :as opts}]
  (if (some? template)
    (let [template-fn (get templates template)
          mailgun-opts (template-fn data)]
      (send** api-key mailgun-opts))
    (send** api-key (select-keys opts [:to :subject :text :html]))))

(defn send [{:keys [params template recaptcha/secret-key] :as sys}]
  (if (= template :biff.auth/signup)
    (let [{:keys [success score]}
          (:body
            (http/post ""
              {:form-params {:secret secret-key
                             :response (:g-recaptcha-response params)}
               :as :json}))]
      (when (and success (<= 0.5 score))
        (send* sys)))
    (send* sys)))

bpenguin19:11:27

Is there anything special going on with the * in those function names?

phronmophobic19:11:19

nothing built into clojure. I've also never seen that convention before. not sure if that's just a convention by the author or if there some's tooling used by the author that relies on it.

seancorfield19:11:10

Feels like it would be a good addition to this section of the FAQ https://clojure.org/guides/faq#qmark_bang

seancorfield19:11:47

The convention is typically that foo* is usually some sort of implementation piece of foo itself.

seancorfield19:11:43

For example, you might have a function fetch-data that caches results for a short time, and it might call fetch-data* which would fetch the data but not cache it.

bpenguin19:11:03

Yes! Thanks a bunch. I read it as breaking out peices of send which are sent explicitly but the rubyist in me saw them as splat operators and I was curious to know if there was any magic in them

seancorfield19:11:23

You'll also see *foo* which is a convention for dynamic Vars that can be used with binding.

andy.fingerhut20:11:53

And a little bit more than merely a convention, since the Clojure compiler warns if you name a Var with "earmuffs", but do not also declare it as ^:dynamic

seancorfield20:11:00

Does it? I thought it used to, but you don't get a warning here:

user=> (def ^:dynamic foo 42)
#'user/foo
user=> (binding [foo 13] (println foo))
13
nil
user=> foo
42
user=>

Michael W20:11:19

user=> (def *foo* 42)
Warning: *foo* not declared dynamic and thus is not dynamically rebindable, but its name suggests otherwise. Please either indicate ^:dynamic *foo* or change the name.

seancorfield20:11:38

Ah, yes, it's the other way around from what I was thinking.

seancorfield20:11:12

So you can declare anything ^:dynamic but if you use earmuffs, you should use ^:dynamic.