This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-11-18
Channels
- # 100-days-of-code (2)
- # beginners (10)
- # boot (1)
- # calva (8)
- # cider (13)
- # cljdoc (7)
- # clojure (32)
- # clojure-uk (15)
- # clojurescript (9)
- # code-reviews (1)
- # cursive (8)
- # datomic (1)
- # emacs (1)
- # fulcro (9)
- # hoplon (4)
- # nrepl (1)
- # off-topic (30)
- # onyx (2)
- # re-frame (11)
- # reitit (3)
- # shadow-cljs (33)
- # tools-deps (3)
- # vim (5)
Hi all, I do a lot of 3D image processing in python and I was wondering what tools are out there in the Clojure ecosystem. One nice thing about python is that so many libraries agree on using numpy ndarrays, but idk if there's such a standard in Clojure (core.matrix?). I eventually want to shift to Clojure to make parallel image processing easier, maybe even distributed with something like Onyx.
http://thi.ng geom may be similar to what you are looking for depending on what type of work you are looking at doing https://github.com/thi-ng/geom
@jswaney absolutely no clue, but I do know that deeplearning4j has an NDArray class that might be useful
Seems like a good fit for handling the image data. My next problem is that my 3D images are larger than memory, so I need to chunk them for processing. I sometimes use HDF5 and other chunk compressed formats for this, but it's definitely not ideal
I would much rather have nodes accessing the image data from a parallel file system and use Onyx to process the image chunks in a distributed manner. I have no idea if something exists for breaking up huge arrays for some Onyx number-crunching
okay so, full disclaimer, totally new field to me. Not out of college yet and the real world looks scary. That being said, i know nothing about Onyx, so my first guess would be something something mapreduce
Yeah I had looked into using hdfs since it's available on the cluster that I use. idk why but the initial barrier to entry for me was so large that I ended up just using memmapped arrays
and, for my benefit, what exactly are the benefits of HDFS (if you get over the barrier to entry)