data-science

Aleed 2024-12-16T14:08:04.771859Z

is dtype-next suitable for machine learning despite not having GPU support? @chris441 i’d be interested in knowing why you chose to go via optimized JVM route and what use-cases that wouldn’t be suitable for, as I’m learning ML and exploring Clojure as an alternative to Python

Rupert (Sevva/All Street) 2024-12-16T14:13:58.753009Z

> I’m learning ML Depends on how much performance you need. For non neural networks (e.g. decision trees) - these are mostly implemented in CPU - you may even be able to just using normal clojure types (vectors, maps etc). For small nerual neworks (e.g. less than a 100,000 nodes) you can do these again using base clojure types or something more optimised like dtpe-next. For bigger neural networks you will likely want GPU support.

👍 1
2024-12-16T15:02:04.866429Z

check "noj" for ML (non deep-learning) https://scicloj.github.io/noj/

chrisn 2024-12-16T17:04:25.752329Z

Use neanderthal - then moving to the GPU is easy.

chrisn 2024-12-16T17:05:13.181439Z

Or use a preexisting python library via libpython-clj. dtype-next in this situation is the interop language and provides some processing capabilities but not really efficient dense numeric operations.

👍 1
Nick McAvoy 2024-12-16T23:15:08.952229Z

Looking at a job posting for a data engineering technical lead for a Clojure team, and it includes this line:

Note on Clojure: While Clojure has been available since 2007, its mainstream growth began around 2019, making it a relatively recent addition to the data engineering ecosystem.
Nice to see someone calling it mainstream in the data ecosystem

👍 1
🚀 1
🎉 8