is dtype-next suitable for machine learning despite not having GPU support? @chris441 i’d be interested in knowing why you chose to go via optimized JVM route and what use-cases that wouldn’t be suitable for, as I’m learning ML and exploring Clojure as an alternative to Python
> I’m learning ML Depends on how much performance you need. For non neural networks (e.g. decision trees) - these are mostly implemented in CPU - you may even be able to just using normal clojure types (vectors, maps etc). For small nerual neworks (e.g. less than a 100,000 nodes) you can do these again using base clojure types or something more optimised like dtpe-next. For bigger neural networks you will likely want GPU support.
check "noj" for ML (non deep-learning) https://scicloj.github.io/noj/
Use neanderthal - then moving to the GPU is easy.
Or use a preexisting python library via libpython-clj. dtype-next in this situation is the interop language and provides some processing capabilities but not really efficient dense numeric operations.
Looking at a job posting for a data engineering technical lead for a Clojure team, and it includes this line:
Note on Clojure: While Clojure has been available since 2007, its mainstream growth began around 2019, making it a relatively recent addition to the data engineering ecosystem.
Nice to see someone calling it mainstream in the data ecosystem