Fork me on GitHub
#data-science
<
2020-01-10
>
blueberry13:01:00

@mkvlr I've just checked and GPU is not advertised as being included in the free plan. Is it just available on a case-by-case basis until it becomes too popular to be unviable, or you plan to offer it publicly? In any case, it is an awesome feature, especially for testing and demonstrations. I would be interested in porting a few tutorials that demonstrate how to do the GPU stuff in Clojure (not immediately due to work overload, but in a month or two). Would it be viable, since it only makes sense if a large(ish) number of people are able to try it, but I'm afraid that it would be a drag for your resources?

fabrao16:01:18

Do you think libpython-clj is production ready?

gigasquid16:01:39

I don't see why not - the link above from the facial rec shows you how to put it together with a conda/docker container

gigasquid16:01:07

The surface area of the interop library itself is pretty small

gigasquid16:01:31

after that - it's a matter of do you think python libs are prod ready 🙂

8
☝️ 4
gigasquid16:01:48

But - you would be on the bleeding edge - which implies the chance of blood

gigasquid16:01:02

Depends on your risk tolerance

fabrao16:01:31

I understand, I think python envs is a realy mess comparing with clojure

fabrao16:01:56

mainly about dependencies

gigasquid16:01:11

Yes but it is a well known mess

fabrao16:01:13

I couldn´t use CUDA libs with python yet because conda/python3.6/python3.7 ... make me very confuse

fabrao16:01:47

beside using it in Windows SO

gigasquid16:01:34

I'm not arguing with you 🙂 - It's another option in the tradeoff matrix that you have to make on when deciding how to build things

chrisn20:01:16

@fabrao - I think using GPU from windows is adding another major layer of complexity. The pathway through GPU docker with ubuntu is at least well known and tested. Then when you deploy things you are at least not changing operating systems. In my opinion, the major simplification to all of this is docker. Without that you are really fighting a lot of environment issues from cuda driver versions to python versions to a lot of other things and these things will make it really hard to move forward. So docker is your friend here but GPU docker and windows...not a good mix from my experience. GPU docker and Linux - doable but also has some caveats. Once that is working then layering gpu-enabled conda, mxnet, jvm, potentially neanderthal and clojure should also be within reach. Hmmm. Maybe that should be a github project right there. So - if you want GPU then you are going to have environment battles whether you use python or clojure. Windows makes these things much harder. Docker is designed specifically to solve the environment problem. Docker with GPU is a bit more involved that docker without GPU but it does work fine. Do you intend to test/deploy on Azure?

fabrao20:01:47

I will intend to deploy to desktop based running.

fabrao20:01:31

How do you work with CV with Cuda for local processing?

fabrao20:01:31

Will docker see gpu in this enviroment?

David Pham20:01:47

Is there any docs on how to use GPU docker containers with Clojure/lib-python-clj or Nehandertal? I always get into trouble just to install CUDA DevKit on Ubuntu.

gigasquid21:01:29

I just got the Huggingface GPT2 text generation working 🎉 https://gist.github.com/gigasquid/f276693bf3519a98afd4ed722edf55ec

4
😮 4
gigasquid21:01:02

Just putting it out here for now - will put together a blog post to go over it more in detail

gigasquid21:01:22

(generate-text "Clojure is a dynamic, general purpose programming language, combining the approachability and interactive" 20)
;=> "Clojure is a dynamic, general purpose programming language, combining the approachability and interactive. It is a language that is easy to learn and use, and is easy to use for anyone"