Clojurians Log v2

Do you have links to the wrappers you're thinking of?

Probably this is not the one Andriy is talking about, but https://djl.ai/ seems to be https://djl.ai/engines/tensorflow/, among other things, and it has https://github.com/scicloj/clj-djl. There used to be more direct wrappers a few years ago.

@levitskyyandriy I'm curious about what you have in mind for a Jax-style wrapper, and whether you've considered wrapping Jax itself.

Just googling "tensorflow clojure github" there are like 5 different projects, all of which were developed 4-7 years ago and then discontinued. Top 3 with around 100 stars each are: https://github.com/kieranbrowne/clojure-tensorflow https://github.com/bpiel/guildsman https://github.com/enragedginger/tensorflow-clj

@daslu IMO running models on client-side is still a hard problem, and Clojure may offer a good solution. I was considering wrapping Jax itself, but I think it is unbearable amount of effort for me as even Google (that owns both Jax and TensorFlow) suggests converting to TensorFlow first https://blog.tensorflow.org/2022/08/jax-on-web-with-tensorflowjs.html The reason why I mentioned TensorFlow, because it is the only mature framework that runs on Javascript and it also has both Java and Javascript API. I mentioned Jax, because of it is functional design which may resonate more with Clojure. Potentially, there could be a problem of mismatch between a functional API of Jax and non-functional core of TensorFlow

What kinds of models would you like to run?

llama.cpp (and by extension, llama.clj) supports many of the LLM models.

I don`t use LLM models for now, but over career worked with time-series, tabular, image and text data, including models such as ResNet, UNet, WaveNet and BERT. I used PyTorch exclusively for all those, and currently just reading on what Clojure has to offer

llama.clj supports BERT

How do you work with the other *Net models? Do you train them yourself? Do you use a predefined architecture provided by a pytorch library?

Image Classification: torchvision usually has good pre-defined architectures with downloadable pre-trained weights usually on ImageNet dataset Other Image/Video related tasks: there are frameworks on top of PyTorch such as https://github.com/open-mmlab/mmdetection that provide training/inference API Time-Series models: usually just build from scratch given layers in PyTorch NLP: https://github.com/huggingface/transformers huggingface transformers library on top of PyTorch would be my go-to place, they also provide their own training/inference API

I'm still pretty new to to this, but it seems like almost all of the models are mostly just a bunch of matrix multiplications. However, I haven't been able to figure out if there's a better way of figuring out how to implement these models other than reading a bunch of dense c++, python, or academic papers.

I've been playing around with a clojure wrapper of https://github.com/ggerganov/ggml?tab=readme-ov-file#updates, which I know can run several real-world models (like llama, yolo, and stable diffusion). I haven't been able to figure out if there's some straightforward way to port models to ggml that aren't yet supported.

I am looking at ggml repo, and I only see inference examples, I will need to dig deeper later to see how it can be used for training, how good is the auto-differentiation capabilities etc

Cool, it would be cool to hear what you think.

I think the devil is usually in optimizations of calculations, i.e. implementing a transformer is relatively easy, but implementing a transformer that can handle big data fast requires a lot of man-hours to build

@smith.adriane https://petewarden.com/2023/10/15/the-unstoppable-rise-of-disposable-ml-frameworks/ this article may be helpful about the differences between Pytorch/TenorFlow and llama.cpp/ggml