Fork me on GitHub
#data-science
<
2016-02-19
>
nkraft19:02:51

Incanter is a nice library, though I've found it tends to choke on really, really big datasets. I keep trying to get away from R, but it doesn't seem that Incanter is quite there yet.

aaelony20:02:19

@nkraft, are you saying that your experience is to use R for larger datasets than Incanter can handle? For big data, what is your experience with transducers? with core.matrix?

nkraft21:02:12

Yes, a lot of the work I do is against hadoop and ElasticSearch datastore map/reduce output, often TB at a time. Incanter just can't work with anything that large, but oddly enough, old quirky R can. I'm not writing apps for this purpose, this is mostly REPL-driven one-off data visualization tasks.

aaelony23:02:17

the quality of R packages is quite good, but it's not really R that is doing the work here. You can somewhat easily replicate what those R packages are doing in Clojure (in a more scalable, functional manner).