Fork me on GitHub
#data-science
<
2020-08-16
>
metasoarous00:08:48

Hi @smith.adriane! I'd highly recommend Oz, but as the author, am highly biased šŸ™‚

šŸ™‚ 3
metasoarous00:08:18

Quil is really cool, and you have a lot of fine grained control with it if you want to do something wild.

metasoarous00:08:49

And manually creating SVGs can also be a good way to go if you want something really custom, though may not scale if you have a lot of data.

metasoarous00:08:36

Oz uses Vega-Lite & Vega for visualization, and there are a number of other tools in the ecosystem that are going that way now (Saite, Hanami, Gorilla, Pink-Gorilla, darkstar, etc). So you can't go wrong with VL & V specifically.

metasoarous00:08:42

VL & V have very "data-driven" declarative approaches to specifying a data visualization as a mapping from data to visual aesthetics (like ggplot, Grammar of Graphics, etc), which I think fits really well with the Clojure model of programming (philosophically).

phronmophobic00:08:47

how much does oz know about the vega data viz model? how crazy of an idea would it be to implement a graphics backend for the vega descriptions in clojure?

metasoarous00:08:08

I think some folks have looked at that with clj-plot

metasoarous00:08:21

Which is javafx based IIRC

metasoarous00:08:38

It's a little crazy, but depends on how much of the spec you want to cover

metasoarous00:08:52

I've been wanting to build a simple ascii render for scatterplots and such

metasoarous00:08:39

The other thing you get from Vega / Vega-Lite is a grammar of interaction which allows you to build interactivity into your graphs in a very simple but powerful way

metasoarous00:08:20

As far as Oz itself, it goes beyond just data visualization and is really a complete scientific document processing toolkit

metasoarous00:08:41

So if you want to render markdown or hiccup with embedded vega to html or (soon) pdf, you can do that too

metasoarous00:08:49

Has live-view mode that will watch a file for changes and update as you save

metasoarous00:08:27

And doing that with clj files even tries to be smart about how it evaluates forms, so you can use it as a notebook-like environment from the comfort of your vim editor

metasoarous00:08:36

Also reagent components if you want to build a full-on app using VL/V

metasoarous00:08:11

Stepping away now, but feel free to ask more questions, and I'll circle back around if other folks haven't answered already

phronmophobic00:08:27

my other side project is a platform agnostic graphics and ui library (linux, mac, and webgl support and experimental support for html, terminal, java2d). ideally, I'd like a library that helps with the data analytics side and if there's a model for visualizations that's already well designed, then I can just start implementing the parts of the data viz model that I care about.

metasoarous01:08:43

I'll not that there is a webgl vega renderer already: https://github.com/vega/vega-webgl-renderer

šŸ‘ 6
metasoarous01:08:53

Would be cool to integrate that in oz

jsa-aerial20:08:31

@smith.adriane I think what you are looking for is @tsulejā€™s CljPlot: https://github.com/generateme/cljplot

šŸ‘€ 3
šŸ‘ 3
phronmophobic20:08:25

neat! will check it out!

genmeblog09:08:17

I'm happy to answer your questions

phronmophobic14:08:46

@tsulej , are the plots themselves reified?

genmeblog15:08:42

@smith.adriane can you elaborate a little bit what do you mean by reified plots?

genmeblog15:08:19

Generally plots are represented by nested maps consisting data, scales, aesthetics, configuration + some functions (this means that these are not pure data). Based on above you render chart on canvas as a separate step.

phronmophobic16:08:28

basically just wondering if there is a data model for the charts/graphs/etc.

genmeblog19:08:53

kind of, but rather generated and not simple to write by hand. I'm going to change it though.

phronmophobic19:08:28

what I would be looking for is some sort of data model that does a good job describing different charts. it doesn't really matter if the descriptions are meant to be produced by hand.

phronmophobic19:08:54

eg. ā€¢ here's all the type charts ā€¢ here's the data they can expect ā€¢ here's all the different ways to customize them ā€¢ maybe some specs for the chart descriptions and the acceptable input data

genmeblog20:08:14

vega/vega-lite does it perfectly well

phronmophobic20:08:21

It matters less to me if there are actually implementations of drawing all of the charts. I've been working on this graphics library for a while, https://github.com/phronmophobic/membrane and would love to just implement the graphics for a given chart spec library

genmeblog20:08:54

There was discussion to introduce vega/vega-lite spec to cljplot, but first I need to make rendering backend closer to this (if it's possible, I don't know)

phronmophobic20:08:29

vega* requires a whole ecosystem of non jvm stuff and it's unclear how extensible it is from clojure land

genmeblog20:08:00

but vega contains well defined specification which can be adopted to other backends

phronmophobic20:08:41

how many backends do they have?

genmeblog20:08:01

I focused mostly on rendering at the beginning and stopped there since I hit the dead end with data manipulation and data dimensionality representation (called aesthetics in ggplot2 or enconding in vega*)

genmeblog20:08:12

they have one backend šŸ™‚

genmeblog20:08:13

Vega advertises itself as a grammar or language, the rendering engine comes at the end of the desctiption

phronmophobic20:08:22

i actually have a copy of the grammar of graphics that I haven't read yet

phronmophobic20:08:34

yea, it definitely seems like a good starting point, but I am wary of large specifications that only have one implementation. it makes it very easy for the spec and the implementation to start to bleed into one another

genmeblog20:08:17

and it's huge

genmeblog20:08:23

personally I prefer ggplot2 approach but nothing stops one to build another, grammar level on the top of rendering library to store/exchange/reuse chart description

genmeblog20:08:40

it is, but it's highly declarative: map data and add this, add that etc.

phronmophobic20:08:09

declarative sounds great. in clojure, that's just data

jsa-aerial20:08:55

The key thing for Vega and Vega-Lite is these things are interactive - they are not simple static things lke ggplot2. If all you want is ggplot2 like stuff, then a good way now may be to just use ggplot2 via Clojisr

phronmophobic20:08:42

what's the support like for adding interactivity to a vega-lite plot via clojure?

jsa-aerial20:08:56

It comes for free

phronmophobic20:08:27

like if I wanted to add a custom interaction?

jsa-aerial20:08:15

Depends on what it is. There is a lot of support for that in Vega-Lite

genmeblog20:08:42

(and event streams)

jsa-aerial20:08:45

And in Vega as well (since Vega-Lite compiles down to Vega)

jsa-aerial20:08:06

It doesn't cover all aspects - for example, various external controls that depend on recomputing the input data are not typically covered

phronmophobic20:08:52

I will have to take some time to digest all this

genmeblog20:08:29

if you want to see reimplementation of vega-lite or lattice (R) examples in cljplot- take a look here https://github.com/generateme/cljplot/tree/master/sketches - I made it to test my concepts (even with kind of interactivity through java2d)

šŸ‘ 3
phronmophobic20:08:56

yea, I've been poking around with that

genmeblog20:08:03

but as I said cljplot shouts for deep refactoring

jsa-aerial20:08:12

That sort of 'interactive' control requires some form of reactive components coupled to the data streams TBH, it is really difficult to 'compete' with what the IDL folks are doing. They have a lot of resources and a boatload of talent

genmeblog20:08:29

this is deep true

phronmophobic20:08:51

that's true, but I feel like I would also be giving up a lot of control. Like I don't think it would have been difficult (not impossible) to make this project using vega-lite, https://blog.phronemophobic.com/treemap/treemaps-are-awesome.html

jsa-aerial20:08:55

Any time you opt for some form of 'declarative' coding, you give up control. Vega sits on top of D3 as its primary backend, It gives up the total control you have in D3, but you get back a great deal in the form of consistency simplicity (I know - Vega does not seem simple, but compared to D3? yeah)

jsa-aerial20:08:12

Vega-Lite also give up some control as it is layered on top of Vega and targets simplifying a lot of the nuts and bolts of generating a simple chart (kind of like ggplot2 in that regard)

phronmophobic20:08:26

I think you can do what looks like declarative coding in clojure that is really just manipulating data. my experience is that you get a lot of the benefits of declarative coding, but you still retain the flexibility

jsa-aerial20:08:38

I don't think tree maps are available in Vega-Lite (yet), but they are definitely in Vega: https://vega.github.io/vega/examples/treemap/

jsa-aerial20:08:49

Only up to a point. Otherwise you wouldn't 'need' things like core.logic

phronmophobic20:08:32

the treemap project wasn't about drawing a traditional treemap, it was about augmenting the treemaps in different ways, including interactivity

jsa-aerial20:08:39

Functional stuff often looks declarative but it really isn't. There is still an explicit flow

jsa-aerial20:08:02

well, you have interactivity for free in vega

phronmophobic20:08:09

we might be talking about different things with respective to writing declarative code in clojure. what I'm referring to is when you're working with just data. data has no explicit flow and hopefully, the what is decoupled from how

jsa-aerial20:08:16

With respect to "platform agnostic graphics and ui" you may also want to take a look at Saite https://github.com/jsa-aerial/saite which is a "turnkey" data exploration and visualization system. Just need Java8 and it works on linux, mac, and windows. But it is Vega/Vega-Lite based. A new release featuring Shiny like dash construction (with R if you like as well) is coming. I am giving a demo of this new stuff at the upcoming SclCloj virtual meeting: https://twitter.com/scicloj/status/1291845872884625408

jsa-aerial20:08:17

One last thing to know about would be hanami https://github.com/jsa-aerial/hanami (and bare bones example: https://github.com/jsa-aerial/bbhanami) which is 'just' a library - does not provide complete document like capabilities that Saite and Oz do. It is agnostic on that and only provides a the 'kit' components and capabilities for you to build your own vis apps.

phronmophobic20:08:45

really great options!