Fork me on GitHub
#membrane
<
2022-07-13
>
Darrick Wiebe19:07:41

Starting with number 2, an edge label is an edge type label. A given node can have edges of multiple types. Another way of thinking about that (and how it's implemented in fermor) is that an edge label is a pointer to which graph the edge is present in. A node can be present in multiple different graphs simultaneously.

Darrick Wiebe19:07:02

so I may have a graph (dw) -friends-> (jh) , etc, and also (dw)-uses->(clojure)

Darrick Wiebe19:07:32

In one view, dw has 1 friends edge and one uses edge. In another view, there is a friends graph and a uses graph, and dw is present in both of them.

Darrick Wiebe19:07:33

The first question: You can not enumerate edges directly but you can enumerate edges on given vertices. To get all edges of a given type, do this: (->> g vertices (out-e :friends))

phronmophobic21:07:08

Is the main entry point fermor.force-atlas/force-atlas ? Is there an example I can check? I tried using a normal graph which which gave an error. It seems like I'm supposed to use a graph created with fermor.force-atlas.graph/make-graph , but I'm not sure what the values for the triples are supposed be.

Darrick Wiebe03:07:36

Sorry, I was afk for a while. Here are a couple of examples of make-graph: This one makes just a simple string. That sounds kind of boring but if you try to lay it out with a force-directed graph algo you'll see that it's actually one of the harder (hardest?) cases.

(def g (make-graph (for [^long i (range 100)]
                     [i (* 10.0 ^double (rand)) (inc i)])))

Darrick Wiebe03:07:07

The algo is tuned right now for about 800 cycles, so run it like this:

(last (take 800 (iterate force-atlas g)))

phronmophobic03:07:39

what are the 3 values of the triplet?

phronmophobic03:07:02

is it [id, x, y]?

Darrick Wiebe03:07:24

[from-vertex-id, edge-weight, to-vertex-id]

phronmophobic03:07:59

ah. i was way off. Does edge-weight affect the layout?

Darrick Wiebe04:07:29

Yeah it does. Here's an example:

metal 1
Darrick Wiebe04:07:33

You can see that it sticks crossings preferentially on low-weight edges

phronmophobic04:07:14

and is the result inspectable via the functions in fermor.core? does it provide coordinates for each node somehow?

Darrick Wiebe04:07:21

I just noticed that it severely misbehaves right now with some other shapes of nets. I'll have to fix it... šŸ™‚

Darrick Wiebe04:07:25

Yeah, the concept in fermor is that each vertex or edge can have a "document" object attached to it. The document can be any arbitrary value. In this case (and the reason it's currently also broken for arbitrary fermor graphs) I have vertex and edge document deftypes in the fermor.force-atlas.graph namespace.

Darrick Wiebe04:07:15

So for any vertex, its document looks like this:

(deftype VDoc [id
               ^:unsynchronized-mutable _position
               ^:unsynchronized-mutable _velocity
               ^:unsynchronized-mutable _prev_velocity
               ^double size ^double mass ^long degree ^long squares]
  IVDoc
  (position [d] _position)
  (velocity [d] _velocity)
  (prev-velocity [d] _prev_velocity)
  (set-position [d v] (set! _position v))
  (set-velocity [d v] (set! _velocity v))
  (set-prev-velocity [d v] (set! _prev_velocity v))
  (swap-position [d f v] (set! _position (f _position v)))
  (swap-velocity [d f v] (set! _velocity (f _velocity v)))
  (swap-prev-velocity [d f v] (set! _prev_velocity (f _prev_velocity v))))

Darrick Wiebe04:07:43

To get a the position of the vertex with id 42:

(-> g
  (fermor.core/get-vertex 42)
  fermor.core/get-document
  fermor.force-atlas.graph/position)

phronmophobic04:07:17

that seems like enough to get up and running

Darrick Wiebe04:07:13

Or all positions:

(->> g g/vertices g/documents (map fermor.force-atlas.graph/position))

phronmophobic06:07:41

ok, I think I got it draw your example graph.

Darrick Wiebe06:07:38

By the way I just pushed a fix. It was drawing certain types of graphs really poorly. A bit better now. some samples:

Darrick Wiebe06:07:54

via

(def g (let [n 500]
           (make-graph (for [^long i (range n)]
                         (let [a (mod i 20)
                               b (if (= a i) (inc i) i)]
                           [a (rand) b])))))

Darrick Wiebe07:07:43

A graph with a bit more structure:

phronmophobic16:07:24

not as pretty as your color versions

phronmophobic17:07:57

Interesting to see how the layout progresses

phronmophobic17:07:40

there's some magic at around step 250

Darrick Wiebe20:07:27

Nice idea with the slider! HSV colors FTW.

Darrick Wiebe20:07:52

There are a few magic time points where the algo changes. Gravity starts at 0, gets enabled and then disabled again. The force algo also changes from global to local. It actually shouldn't really have a bunch of points sitting on top of each other at step 249 I think...

phronmophobic21:07:20

What property do you use for coloring?

Darrick Wiebe21:07:15

vertex id

šŸ‘ 1
chromalchemy17:07:34

These visualizations look wonderful!

phronmophobic19:07:02

@ben.sless moving the slowness issues to a new thread as well. Based on previous discussions, it seems like the issue is that the window isn't being redrawn for some reason, but it's kind of hard to tell

Ben Sless19:07:05

Steps to reproduce: Clone repo Run main from CLI as o'er readme Do you happen to have a Linux machine to test on? Is there a chance you're loading a different toolkit by default?

phronmophobic20:07:16

I don't have a linux machine to test on at the moment šŸ˜ž. I've been using linux on parallels and virtual box, but neither of those seem to manifest the issue.

Ben Sless20:07:51

Jvm version?

phronmophobic20:07:30

the force-layout repo explicitly uses the skia backend

phronmophobic20:07:00

which should isolate the issue from the specific java version

phronmophobic20:07:47

it also uses an updated version of membrane that might fix not redrawing on linux, https://github.com/phronmophobic/membrane/commit/9815b4e381a9bbcbbdb8391e20e99e3f026fb0f5

Ben Sless20:07:16

Hang on, let me report a bit

gratitude 1
Ben Sless20:07:34

until now, I tried upgrading to java 17, and it was using skia

Ben Sless20:07:55

now I tried upgrading the meander dep to the git sha and I get

Syntax error (ClassNotFoundException) compiling at (membrane/skia.clj:1:1).
com.phronemophobic.membrane.Skia

phronmophobic20:07:13

you can't use skia with git dep

phronmophobic20:07:40

hold on, I can try to add a prep step

Ben Sless20:07:52

I hope you don't mind, but it's late here, I'm going to sleep šŸ’¤

Ben Sless20:07:05

I'll pick it back up tomorrow, so feel free to leave a bunch of instructions

šŸ™‚ 1
phronmophobic20:07:06

ok, no problem

phronmophobic20:07:10

thanks for your help!

phronmophobic06:07:51

Unfortunately, I didn't get a chance to put together anything special to diagnose the issue. However, I would be interested to see if the java2d backend now updates as it should for you now. You should be able to test it by checking out the java2d branch of the force layout project, https://github.com/phronmophobic/graph.force-layout/tree/java2d and running the example. No worries if you don't have time or would rather wait for a more comprehensive test diagnostic.

Ben Sless07:07:27

java2d works like a charm

šŸŽ‰ 1
Ben Sless07:07:38

apropo, while I was trying to understand the problem yesterday, I attached visualvm and saw that even moving the mouse across the window allocates like crazy

Ben Sless07:07:38

It leaks, too

Ben Sless07:07:08

I think I found the leak:

(def
  ^{:arglists '([elem])
    :doc
    "Returns a 2 element vector with the [width, height] of an element's bounds with respect to its origin"}
  bounds (memoize #(-bounds %)))

phronmophobic17:07:32

glad the java2d fix is working for you!

Ben Sless17:07:53

any thoughts on the leak?

phronmophobic17:07:09

Membrane has a pretty large surface area since building apps requires a full stack of graphics, events, windowing, state management, component library etc. There's a number of places that trade memory/efficiency for ease of implementation. Basically, there's a lot of room for improvement. It's been pretty surprising how responsive membrane apps feel (at least on my computers) given how minimal some of the various implementations are. I try to focus on the bottlenecks. Right now, I believe that the biggest bottleneck is improving the workflow for building new UIs. Is the memory leak a bottle neck for an application you're trying build?

phronmophobic17:07:45

As far as memory and CPU usage, I think these are probably the most promising areas: ā€¢ periodically cleaning the image cache (which is also used to cache text draws) ā€¢ optimizing draw calls ā€¢ optimizing event handling

phronmophobic17:07:32

With respect to the bounds leak, there's a couple of related thoughts: ā€¢ It's typical to have a cache that's related to frames. In other words, you cache data for some number of frames. That might be a better fit than a global cache. ā€¢ Not every usage of bounds is in the graphics loop. ā€¢ A simple fix might be just to use some sort of weak reference for the cache which might be useful for some other caches. ā€¢ If I recall correctly, memoizing bounds does have a noticeable impact on performance ā€¢ If there is a bottleneck related to bounds, a simple workaround is to replace membrane.ui/bounds with a LRU cache or similar

phronmophobic17:07:19

I hope it doesn't sound like I'm brushing off the memory leak. It's definitely a problem that I would like to address at some point. Mostly just trying gauge the severity and potential impact.

Ben Sless17:07:23

Doesn't sound like a brush off at all, those are just priorities. In clj I'd say just stick Caffeine in there as a cache and let it handle things

šŸ‘€ 1
Ben Sless17:07:36

Do that for all caches + option to change cache behavior by user (instantiate a new cache w/ different config and populate it with old one)

šŸ‘ 1
Ben Sless17:07:53

You can then bound the size of the cache, add ttl, whatever

Ben Sless17:07:43

an orthogonal idea is for shapes to cache their own bounds, just like clojure objects cache their hash

phronmophobic17:07:39

have you used caffeine before? do you recommend a wrapper or just interop?

Ben Sless17:07:49

Just interop, it's easy to use

Ben Sless17:07:15

And it's easy to turn it inside-out into memoization by using the loading-cache

Ben Sless18:07:25

Another problem with memoization in Clojure is that collections equality is slow

phronmophobic19:07:00

components are also memoized so in many cases, you're comparing against the exact same object which is fast.

phronmophobic19:07:45

are clojure's collection equality comparisons much slower than other collection implementations? I though most other collection implementations just don't offer sane equality semantics?