Fork me on GitHub
#membrane
<
2022-07-13
>
Darrick Wiebe19:07:41

Starting with number 2, an edge label is an edge type label. A given node can have edges of multiple types. Another way of thinking about that (and how it's implemented in fermor) is that an edge label is a pointer to which graph the edge is present in. A node can be present in multiple different graphs simultaneously.

Darrick Wiebe19:07:02

so I may have a graph (dw) -friends-> (jh) , etc, and also (dw)-uses->(clojure)

Darrick Wiebe19:07:32

In one view, dw has 1 friends edge and one uses edge. In another view, there is a friends graph and a uses graph, and dw is present in both of them.

Darrick Wiebe19:07:33

The first question: You can not enumerate edges directly but you can enumerate edges on given vertices. To get all edges of a given type, do this: (->> g vertices (out-e :friends))

phronmophobic21:07:08

Is the main entry point fermor.force-atlas/force-atlas ? Is there an example I can check? I tried using a normal graph which which gave an error. It seems like I'm supposed to use a graph created with fermor.force-atlas.graph/make-graph , but I'm not sure what the values for the triples are supposed be.

Darrick Wiebe03:07:36

Sorry, I was afk for a while. Here are a couple of examples of make-graph: This one makes just a simple string. That sounds kind of boring but if you try to lay it out with a force-directed graph algo you'll see that it's actually one of the harder (hardest?) cases.

(def g (make-graph (for [^long i (range 100)]
                     [i (* 10.0 ^double (rand)) (inc i)])))

Darrick Wiebe03:07:07

The algo is tuned right now for about 800 cycles, so run it like this:

(last (take 800 (iterate force-atlas g)))

phronmophobic03:07:39

what are the 3 values of the triplet?

phronmophobic03:07:02

is it [id, x, y]?

Darrick Wiebe03:07:24

[from-vertex-id, edge-weight, to-vertex-id]

phronmophobic03:07:59

ah. i was way off. Does edge-weight affect the layout?

Darrick Wiebe04:07:29

Yeah it does. Here's an example:

metal 1
Darrick Wiebe04:07:33

You can see that it sticks crossings preferentially on low-weight edges

phronmophobic04:07:14

and is the result inspectable via the functions in fermor.core? does it provide coordinates for each node somehow?

Darrick Wiebe04:07:21

I just noticed that it severely misbehaves right now with some other shapes of nets. I'll have to fix it... 🙂

Darrick Wiebe04:07:25

Yeah, the concept in fermor is that each vertex or edge can have a "document" object attached to it. The document can be any arbitrary value. In this case (and the reason it's currently also broken for arbitrary fermor graphs) I have vertex and edge document deftypes in the fermor.force-atlas.graph namespace.

Darrick Wiebe04:07:15

So for any vertex, its document looks like this:

(deftype VDoc [id
               ^:unsynchronized-mutable _position
               ^:unsynchronized-mutable _velocity
               ^:unsynchronized-mutable _prev_velocity
               ^double size ^double mass ^long degree ^long squares]
  IVDoc
  (position [d] _position)
  (velocity [d] _velocity)
  (prev-velocity [d] _prev_velocity)
  (set-position [d v] (set! _position v))
  (set-velocity [d v] (set! _velocity v))
  (set-prev-velocity [d v] (set! _prev_velocity v))
  (swap-position [d f v] (set! _position (f _position v)))
  (swap-velocity [d f v] (set! _velocity (f _velocity v)))
  (swap-prev-velocity [d f v] (set! _prev_velocity (f _prev_velocity v))))

Darrick Wiebe04:07:43

To get a the position of the vertex with id 42:

(-> g
  (fermor.core/get-vertex 42)
  fermor.core/get-document
  fermor.force-atlas.graph/position)

phronmophobic04:07:17

that seems like enough to get up and running

Darrick Wiebe04:07:13

Or all positions:

(->> g g/vertices g/documents (map fermor.force-atlas.graph/position))

phronmophobic06:07:41

ok, I think I got it draw your example graph.

Darrick Wiebe06:07:38

By the way I just pushed a fix. It was drawing certain types of graphs really poorly. A bit better now. some samples:

Darrick Wiebe06:07:54

via

(def g (let [n 500]
           (make-graph (for [^long i (range n)]
                         (let [a (mod i 20)
                               b (if (= a i) (inc i) i)]
                           [a (rand) b])))))

Darrick Wiebe07:07:43

A graph with a bit more structure:

phronmophobic16:07:24

not as pretty as your color versions

phronmophobic17:07:57

Interesting to see how the layout progresses

phronmophobic17:07:40

there's some magic at around step 250

Darrick Wiebe20:07:27

Nice idea with the slider! HSV colors FTW.

Darrick Wiebe20:07:52

There are a few magic time points where the algo changes. Gravity starts at 0, gets enabled and then disabled again. The force algo also changes from global to local. It actually shouldn't really have a bunch of points sitting on top of each other at step 249 I think...

phronmophobic21:07:20

What property do you use for coloring?

Darrick Wiebe21:07:15

vertex id

👍 1
chromalchemy17:07:34

These visualizations look wonderful!

genekim21:02:38

Hi, @smith.adriane — have you published source code for this wonderful app? Trying to build an app with a slider bar, and wanted to try out membrane again. Thank you!

phronmophobic22:02:09

The slider bar is the slider bar that's included in membrane.basic-components/number-slider. It's a little goofy if you don't release the mouse drag within the number-slider bounds. I know what the fix is for that, but I haven't gotten around to it. However, if you need it, I can work on adding the fix soonish.

genekim22:02:57

Holy cow. Amazing! I will try writing an app later today — I’ll write more about what I’m trying to do, as soon as I get some urgent stuff done.

👍 2
Darrick Wiebe05:02:29

Hey, thanks for pushing that ui example! I had kind of messed up my force-atlas branch in git a while ago. It had been on my todo list to fix that up so this gave me the excuse I needed. The force atlas code is now resurrected in a new branch called force-atlas2 based on the latest (and much improved) update of fermor. I've pulled your fermor.ui namespace example into that branch while I was at it. https://github.com/pangloss/fermor/tree/force-atlas2#force-directed-graph-layout Cheers!

phronmophobic18:02:39

@U01D37REZHP, very cool. I recently tried to use fermor to layout the graph of namespaces usages of all the clojure projects on github, https://github.com/phronmophobic/dewey/releases/tag/clojure-galaxy-v1 I can't remember the issue, but I couldn't quite get it to work. The graph has around 131,923 nodes and 544,948 edges. Do you know if fermor should be able to work with that size graph?

Darrick Wiebe18:02:02

Fermor can easily work with graphs that large, but I don't think that layout algo will do well with it. It may work but I'd expect it to take quite a long time. I've been targeting graphs with about 10k nodes max for layout.

👍 2
phronmophobic18:02:46

The https://cosmograph.app/ web app is pretty neat. It uses the gpu to do all the heavy lifting, https://github.com/cosmograph-org/cosmos.

Darrick Wiebe18:02:58

That's pretty cool looking!

Darrick Wiebe19:02:28

I'll be spending some time figuring out how this works... :thinking_face:

phronmophobic19:02:56

Yea, I also spent some time looking into it. It didn't seem too complicated.

phronmophobic19:02:33

I think the main idea is to just get it to run on the GPU

Darrick Wiebe19:02:37

Yeah, pretty nicely architected!

Darrick Wiebe19:02:44

Yeah, but that's not so easy in practice!

👆 2
Darrick Wiebe19:02:35

Is this the technique they are using?

Darrick Wiebe19:02:47

I was watching it run and thinking it does look like they're running in 3d

phronmophobic19:02:49

Yea, it's part of the implementation.

Darrick Wiebe19:02:55

my algo is 2d...

phronmophobic19:02:01

There's is 2d as well

Darrick Wiebe19:02:08

I actually had something like that implemented but it added enough overhead that for the graphs I'm interested it it actually slowed things down because the structure needed to be rebuilt on every step.

Darrick Wiebe19:02:32

it starts winning on really big graphs. But maybe this is a better algo than what I used. I don't recall the name of it...

phronmophobic19:02:33

I spent time trying to learn https://neanderthal.uncomplicate.org/ to see if I could get this working in clojure, but I couldn't get any gpu stuff to work on my mac

Darrick Wiebe19:02:02

Yeah I have the same mac problem. Until Neanderthal finally supports mac I can't use it because my users use macs.

Darrick Wiebe19:02:22

(as do I, but I'd be willing to use aws, etc)

phronmophobic19:02:45

oh, well if you ever find a good option for gpu programming on mac, it would be great to hear back.

Darrick Wiebe19:02:12

I believe JAX and Pytorch now run on mac.

Darrick Wiebe19:02:12

But I think this is using the graphics pipeline. Not sure if the algo could use the deep learning pipeline instead.

🤷 2
Darrick Wiebe20:02:33

So... this is a nice project using the traditional vertex/fragment shader pipeline, but the new hotness is mesh shaders. It looks like this project is also using some deprecated APIs so it may be quite a bit older than it seems based on its recent open source release. I wonder if there is another 10x better version floating in the platonic world waiting to be born.

phronmophobic20:02:06

I have very little experience with any of that stuff. Sounds cool!

phronmophobic21:02:56

> I believe JAX and Pytorch now run on mac. Do you happen to know what underlying API they use to access the GPU?

phronmophobic21:02:47

Ah, I guess this stuff is using https://developer.apple.com/metal/? > Metal backend for PyTorch > The new Metal backend in PyTorch version 1.12 enables high-performance, GPU-accelerated training using MPS Graph and the Metal Performance Shaders primitives.

phronmophobic21:02:16

I wonder if I can generate a wrapper for Metal directly with https://github.com/phronmophobic/clong.

phronmophobic07:03:58

I tried visualizing the graph of all clojure namespaces again with graphviz. It looks interesting... Maybe if I can tweak some parameters it will be better. (fyi, image is pretty big)

Darrick Wiebe16:03:32

Interesting, but I think visualization's usefulness tapers off starting at about 50 nodes and by the time it hits 1000 it is at 0 in most cases. The visualizer I built is actually reasonably nice looking IMO, but that's really a side effect because I built it as part of a physical layout algorithm. I'd recommend querying the graph with the tools I've built in fermor instead, if your goal is understanding the data, then visualizing once you've selected interesting stuff.

Darrick Wiebe16:03:41

But on the visualization side, I'd recommend http://d2lang.com over graphvis.

Darrick Wiebe16:03:31

An example of a view into my compiler using d2:

👍 2
Darrick Wiebe16:03:49

It's a really nice tool.

phronmophobic02:03:53

I think visualizations tend to be more helpful as the data size increases. There's a sweet spot in the middle where it's both easy and helpful to visualize. On the larger end, it's still very useful, but it takes more effort to figure out how to visualize the data.

phronmophobic02:03:41

Was fiddling with graphviz parameters and made this. Every dot is a clojure namespace. Size corresponds loosely to usage.

Darrick Wiebe15:03:07

Perhaps it's useful in some cases, but I think you'd get a lot more value out of a simple query for the top x nodes sorted by in degree on the dependency edge, which would give you the most popular nodes. That gives you the same information this provides, but with the ability to dig further on what you selected. Of course if you need something that looks awesome without necessarily providing deep insight, then in my opinion large scale graph visualization is about as good as you can get.

genekim17:03:35

Wonderful graphviz image, @smith.adriane — did you use command line for this, or some magic to call C library from clojure?

phronmophobic17:03:58

@U6VPZS1EK , I decided to revisit wrapping graphviz based on one of our conversations.

genekim17:03:28

Dude. That is... 🤯 ... epic. So cool!!!

genekim17:03:02

Just curious: what do you think the level of effort would be to use the (apparent) graphviz capability to create SVG, which is directly renderable by membrane? (Although I realize now that same thing could be done by outputing a Byte Array, or whatever, instead of a file...) https://graphviz.org/docs/outputs/svg/ Why I'm asking: immediate uses that jump to mind, related to projects I've wanted to do: • open up another option for visualizing book manuscripts (we've talked about this) — currently restricted to Vega tree-maps. • SVG output would indirectly expose the x,y coords (something you said you wanted to get out of the graphviz library) • have you seen Graphviz Online? https://dreampuf.github.io/GraphvizOnline/. Neat app where you write dot files on right, renders on left. Thought this might trigger some exciting thoughts Kudos on the library, Adrian!

phronmophobic17:03:23

clj-graphviz can already output SVG

🤯 2
genekim17:03:42

(Ha! So many online graphviz editors — here's one with syntax highlighting. https://dreampuf.github.io/GraphvizOnline/ Super interesting that they all encode the dot into the URL! Ha!

phronmophobic17:03:41

It currently only has an option to write the output to a file. It's also possible to render to directly into memory, but it's not yet implemented.

genekim17:03:24

Beautiful!

phronmophobic19:07:02

@ben.sless moving the slowness issues to a new thread as well. Based on previous discussions, it seems like the issue is that the window isn't being redrawn for some reason, but it's kind of hard to tell

Ben Sless19:07:05

Steps to reproduce: Clone repo Run main from CLI as o'er readme Do you happen to have a Linux machine to test on? Is there a chance you're loading a different toolkit by default?

phronmophobic20:07:16

I don't have a linux machine to test on at the moment 😞. I've been using linux on parallels and virtual box, but neither of those seem to manifest the issue.

Ben Sless20:07:51

Jvm version?

phronmophobic20:07:30

the force-layout repo explicitly uses the skia backend

phronmophobic20:07:00

which should isolate the issue from the specific java version

phronmophobic20:07:47

it also uses an updated version of membrane that might fix not redrawing on linux, https://github.com/phronmophobic/membrane/commit/9815b4e381a9bbcbbdb8391e20e99e3f026fb0f5

Ben Sless20:07:16

Hang on, let me report a bit

gratitude 1
Ben Sless20:07:34

until now, I tried upgrading to java 17, and it was using skia

Ben Sless20:07:55

now I tried upgrading the meander dep to the git sha and I get

Syntax error (ClassNotFoundException) compiling at (membrane/skia.clj:1:1).
com.phronemophobic.membrane.Skia

phronmophobic20:07:13

you can't use skia with git dep

phronmophobic20:07:40

hold on, I can try to add a prep step

Ben Sless20:07:52

I hope you don't mind, but it's late here, I'm going to sleep 💤

Ben Sless20:07:05

I'll pick it back up tomorrow, so feel free to leave a bunch of instructions

🙂 1
phronmophobic20:07:06

ok, no problem

phronmophobic20:07:10

thanks for your help!

phronmophobic06:07:51

Unfortunately, I didn't get a chance to put together anything special to diagnose the issue. However, I would be interested to see if the java2d backend now updates as it should for you now. You should be able to test it by checking out the java2d branch of the force layout project, https://github.com/phronmophobic/graph.force-layout/tree/java2d and running the example. No worries if you don't have time or would rather wait for a more comprehensive test diagnostic.

Ben Sless07:07:27

java2d works like a charm

🎉 1
Ben Sless07:07:38

apropo, while I was trying to understand the problem yesterday, I attached visualvm and saw that even moving the mouse across the window allocates like crazy

Ben Sless07:07:38

It leaks, too

Ben Sless07:07:08

I think I found the leak:

(def
  ^{:arglists '([elem])
    :doc
    "Returns a 2 element vector with the [width, height] of an element's bounds with respect to its origin"}
  bounds (memoize #(-bounds %)))

phronmophobic17:07:32

glad the java2d fix is working for you!

Ben Sless17:07:53

any thoughts on the leak?

phronmophobic17:07:09

Membrane has a pretty large surface area since building apps requires a full stack of graphics, events, windowing, state management, component library etc. There's a number of places that trade memory/efficiency for ease of implementation. Basically, there's a lot of room for improvement. It's been pretty surprising how responsive membrane apps feel (at least on my computers) given how minimal some of the various implementations are. I try to focus on the bottlenecks. Right now, I believe that the biggest bottleneck is improving the workflow for building new UIs. Is the memory leak a bottle neck for an application you're trying build?

phronmophobic17:07:45

As far as memory and CPU usage, I think these are probably the most promising areas: • periodically cleaning the image cache (which is also used to cache text draws) • optimizing draw calls • optimizing event handling

phronmophobic17:07:32

With respect to the bounds leak, there's a couple of related thoughts: • It's typical to have a cache that's related to frames. In other words, you cache data for some number of frames. That might be a better fit than a global cache. • Not every usage of bounds is in the graphics loop. • A simple fix might be just to use some sort of weak reference for the cache which might be useful for some other caches. • If I recall correctly, memoizing bounds does have a noticeable impact on performance • If there is a bottleneck related to bounds, a simple workaround is to replace membrane.ui/bounds with a LRU cache or similar

phronmophobic17:07:19

I hope it doesn't sound like I'm brushing off the memory leak. It's definitely a problem that I would like to address at some point. Mostly just trying gauge the severity and potential impact.

Ben Sless17:07:23

Doesn't sound like a brush off at all, those are just priorities. In clj I'd say just stick Caffeine in there as a cache and let it handle things

👀 1
Ben Sless17:07:36

Do that for all caches + option to change cache behavior by user (instantiate a new cache w/ different config and populate it with old one)

👍 1
Ben Sless17:07:53

You can then bound the size of the cache, add ttl, whatever

Ben Sless17:07:43

an orthogonal idea is for shapes to cache their own bounds, just like clojure objects cache their hash

phronmophobic17:07:39

have you used caffeine before? do you recommend a wrapper or just interop?

Ben Sless17:07:49

Just interop, it's easy to use

Ben Sless17:07:15

And it's easy to turn it inside-out into memoization by using the loading-cache

Ben Sless18:07:25

Another problem with memoization in Clojure is that collections equality is slow

phronmophobic19:07:00

components are also memoized so in many cases, you're comparing against the exact same object which is fast.

phronmophobic19:07:45

are clojure's collection equality comparisons much slower than other collection implementations? I though most other collection implementations just don't offer sane equality semantics?