This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-10-28
Channels
- # announcements (8)
- # architecture (10)
- # babashka (24)
- # beginners (93)
- # calva (2)
- # chlorine-clover (6)
- # cider (2)
- # cljfx (3)
- # cljsrn (5)
- # clojure (59)
- # clojure-australia (2)
- # clojure-canada (9)
- # clojure-europe (46)
- # clojure-nl (3)
- # clojure-spain (1)
- # clojure-spec (2)
- # clojure-uk (6)
- # clojurescript (98)
- # cryogen (20)
- # datomic (22)
- # depstar (14)
- # emacs (4)
- # etaoin (2)
- # figwheel-main (1)
- # fulcro (5)
- # graalvm (1)
- # java (69)
- # malli (33)
- # off-topic (5)
- # pathom (1)
- # re-frame (14)
- # reagent (10)
- # shadow-cljs (24)
- # sql (22)
- # vim (2)
Built an uberjar with clj -A:depstar -m hf.depstar.uberjar prhyme.jar
Library has the following line: (nippy/thaw-from-file (io/resource "dark-corpus-2.bin"))
File confirmed in jar with jar ft prhyme.jar | grep dark-corpus-2.bin
jar depended on by another project with {:local/root "/home/user/code/prhyme/prhyme.jar"}
Error thrown when starting a repl:
Caused by: java.lang.IllegalArgumentException: Not a file: jar:file:/home/user/code/prhyme/prhyme.jar!/dark-corpus-2.bin
at $fn__11362.invokeStatic(io.clj:61)
Any thoughts on what would cause this? Is that !
in jar:file:/home/.../prhyme.jar!/dark-corpus-2.edn
relevant? Or is it some artifact of an internal of java loading resources?
I see a sorta similar issue at https://github.com/seancorfield/depstar/issues/27 but that person was getting an error about a file being skipped. Not the case for me. resources
is on the path in deps.edn.I may have just answered my own question. I see there is a difference between an io/resource
and an io/file
.
(= (io/file (io/resource "dark-corpus-2.bin"))
;; => #object[java.io.File 0x4f44ce9c "/home/user/code/prhyme/resources/dark-corpus-2.bin"]
(io/resource "dark-corpus-2.bin")
;; => #object[java.net.URL 0x31865f0d "file:/home/user/code/prhyme/resources/dark-corpus-2.bin"]
)
Or I was close...
https://groups.google.com/g/clojure/c/8scFidro4ow
> Just delete that line. The io/resource
function returns a URL which all the Clojure IO functions can handle just fine-as is. When running in development the URL happens to be a file://
URL, and thus something io/file
can handle. Once the resource is in a JAR that is no longer the case, and hence exceptions. Just don't require a file when any URL will do and you'll be fine.
But nippy's thaw-from-file
sends it back to an io/file
.
(defn thaw-from-file
"Convenience util: like `thaw`, but reads from `( <file>)`.
To thaw from a resource on classpath (e.g in Leiningen `resources` dir):
(thaw-from-file ( \"my-resource-name.npy\"))
See also `freeze-to-file`."
([file ] (thaw-from-file file nil))
([file thaw-opts]
(let [file (jio/file file),
ba (byte-array (.length file))]
(with-open [in (DataInputStream. (jio/input-stream file))]
(.readFully in ba))
(thaw ba thaw-opts))))
@ericihli Have you confirmed that file is actually in the JAR? jar tvf prhyme.jar | fgrep dark-corpus
Yeah, it's definitely in there. I'm pretty sure now this issue has nothing to do with depstar.
https://groups.google.com/g/clojure/c/8scFidro4ow
I'm gathering you can't use io/file
to access a packaged resource.
Nippy is using io/file
to get the length of the file to create a byte-array.
(defn thaw-from-file
"Convenience util: like `thaw`, but reads from `( <file>)`.
To thaw from a resource on classpath (e.g in Leiningen `resources` dir):
(thaw-from-file ( \"my-resource-name.npy\"))
See also `freeze-to-file`."
([file ] (thaw-from-file file nil))
([file thaw-opts]
(let [file (jio/file file),
ba (byte-array (.length file))]
(with-open [in (DataInputStream. (jio/input-stream file))]
(.readFully in ba))
(thaw ba thaw-opts))))
I'm trying an alternative.
(defn thaw-from-file
([file ] (thaw-from-file file nil))
([file thaw-opts]
(let [xin (io/input-stream file)
xout (ByteArrayOutputStream.)]
(io/copy xin xout)
(nippy/thaw (.toByteArray xout) thaw-opts))))
That does appear to fix it.Interesting. Which line exactly throws the exception? Is it the .length
call?
Works fine until you package the code up as a jar and another package depends on it.
Ah, so the docstring is misleading then:
To thaw from a resource on classpath (e.g in Leiningen `resources` dir):
(thaw-from-file ( \"my-resource-name.npy\"))
Well, glad you have it working now!
We used to use Nippy at work but it's a strange beast so we decided to switch to something else. In the end, I think we stopped using all the taoensso libraries.
Oh man that's interesting to hear. I've found myself migrating towards all of them.
Also, if you build the jar like this, you'll see it tell you every file it puts into the JAR: clj -A:depstar -m hf.depstar.uberjar prhyme.jar -v