Fork me on GitHub
#data-science
<
2016-03-06
>
joelkuiper18:03:20

Hey just sharing a bit of architecture here. We do some heavy natural language processing at our company, support vector machines, deep learning and the likes. While Clojure is a great language it’s just too slow for these type of things, it can’t compete against MKL optimized native BLAS libraries that Python has (with numpy etc). So we have a “microservices” system

joelkuiper18:03:10

essentially a simple dockerized JSON-RPC system that does the ML in Python, and we consider those bits idempotent and non-destructive (e.g they should only add or mutate existing fields, not change the datastructure radically)

joelkuiper18:03:16

and so far this has been working great

joelkuiper18:03:38

take a look at http://robot-reviewer.vortext.systems for a system in Clojure with a python and NodeJS RPC call for the ML stuff