Fork me on GitHub
#asami
<
2022-06-06
>
folcon17:06:42

I'm not sure this is of interest to anyone, but I've been a bit annoyed by the required dependency on graphviz, not that there's anything wrong with it, but if I'm deploying something that does graph generation as part of my app, it's a bit much to ask my end user to have it 😃... So I've been digging into using this: https://mvnrepository.com/artifact/guru.nidi/graphviz-java Which is a java only lib that can act as a stand-in for graphviz. Working with it in clojure is a bit of a pain, but I've written some functions that accept asami + ubergraph's. Would anyone be interested if I bundled that bit into a library and open sourced it?

❤️ 1
quoll17:06:36

Is this in relation to asami-loom?

quoll17:06:49

That project has a whole lot of compromises to make it work with Loom. But if you can convince Loom to detach graphviz, then sure 🙂

folcon17:06:32

I've started trying to do some graph work in asami and step 1 was rendering the output so I can see what I'm looking at. I'm probably going to have to write some convenience functions as the level of resolution asami generates a graph at is not quite at the right level of abstraction for what I'm thinking of. Basically I'd like to transform all the :a/owns so that they look like they're attributes on the node, so the node is an entity itself 😃...

folcon17:06:29

I'm not thinking of changing loom directly, just putting together a lib which could be used alongside it 😃... Similar to asami-loom

folcon17:06:20

I was able to successfully deploy a jar which could then generate and render graphs using this approach.

folcon17:06:14

Once we've established if it's useful or not, or if loom gets pulled into commons, then things like rendering output can be tidied up I think.

quoll18:06:31

A couple of comments… • I’ve been working on asami-loom recently (I haven’t checked anything in). I found a couple of bugs, but not on commonly used functions. I have fixed them, and need to push this. • Asami-loom extends Asami to be Loom compatible by extending protocols onto the Graph implementations in Asami. That needs a good chunk of work for each graph type, and I’m in the process of doing it for Durable graphs. • The Graphviz integration is mostly just generating .dot files. The API for visualizing just launches graphviz as a separate application, passing it a generated file. So there’s very loose coupling

folcon18:06:28

> I’ve been working on asami-loom recently ... Great to hear 😃... > Asami-loom extends Asami to be Loom compatible ... At present what's provided works for my current needs. I will likely start digging into this later. > The Graphviz integration is mostly just generating ... There is loose coupling, however in this case it creates a dependency that I need to solve from a deployment perspective. I can't ask my end users to install graphviz, hence using this approach to bundle the generation into my application. I suspect I'll be in the minority on this, which is why I was asking if anyone else wanted the same behaviour as I'm currently working on it now, so it won't be a lot of work to shove it into a separate library at some point in the next few weeks...

quoll18:06:40

I’d need to look, but I can’t recall any dependencies? It felt like it just does an exec and leaves it. Was that right? There was no library that it tried to talk to

folcon18:06:30

I believe that's right, but from my perspective that's a problem 😃... That's an exec that I need to get on an end user machine path somehow...

quoll18:06:33

Well, it’s not “exec” per se. It calls the system tool to launch the default application that is installed. https://github.com/aysylu/loom/blob/d458f0c0dee9021983c64381b90a470f0178cc8e/src/loom/io.clj#L107-L119

folcon18:06:44

Yep, which is not quite helpful when graphviz doesn't come as standard on machines 😃

quoll18:06:24

true, but it gives a more sensible message, as in “please select the application to open this file”

phronmophobic19:06:06

https://github.com/nidi3/graphviz-java sounds interesting, but I'm not sure "pure java" is the best description (even though they use it in their description). Under https://github.com/nidi3/graphviz-java, it says: • If the machine has graphviz installed and a dot command is available, spawn a new process running dot. • Use this https://github.com/mdaines/viz.js of graphviz and execute it on the V8 javascript engine. This is done with the bundled https://github.com/eclipsesource/J2V8 library. • Alternatively, the javascript can be executed on Java's own Nashorn or GraalVM engine (preferring Graal if both are available).

phronmophobic19:06:13

So it's either using the installed version of graphviz, or bundling a full javascript engine to run a javascript port of graphviz

folcon19:06:22

Yep, it's a bit of a mess, however it's the closest thing I've found to an output that doesn't require me to deploy graphviz on an end user machine. Happy to swap to another approach if you can suggest something better 😃... My goal is to render reasonable graphs inside a UI however. There aren't good answers I've found at the moment 😃

quoll19:06:03

I think a library that can take a Loom source and render it would be useful here, and appropriately decoupled

quoll19:06:05

It might be based on a full JavaScript port of graphviz, but it wouldn’t be coupling it to a project where it wasn’t appropriate

folcon19:06:53

That's fair. I'll leave this be for now then. As trying to figure out how to do that would be a fair few steps from where I'm currently at

quoll19:06:21

The way I was able to use graphviz in the past was to create an Asami graph, require asami-loom (so that the protocol extensions came in), then called with that graph. It was highly decoupled

folcon19:06:35

I think that process nil's out if that's run on machine without graphviz installed or it crashes. I can't quite remember which as it was a fair while ago 😃

quoll19:06:20

It gets all the way up to calling “start” or “open” on the file. That’s when it asks for you to select an app that can open a .dot file

quoll19:06:06

Once I got graphviz installed, and attached it to .dot files, then it worked, with no changes to anything in Clojure-land

folcon19:06:41

Yes, which is unfortunately a problem in an app context as I'll want to generate the output and render it back to the UI without user intervention. Hence going down the embedded route.

quoll19:06:00

I have a version of Loom where I split the namespace into 2: with dot-esc, dot-attrs, dot-str, and the other with: dot, open, open-data, render-to-bytes, and view. The first one is about generating the .dot text, and the others are about file-io and launching the app

quoll19:06:19

It turned out that the dot-str function is ClojureScript compatible, because ClojureScript includes the Google string buffer class, which has an .append method

quoll19:06:53

so the same code will run on ClojureScript so long as the imports are tweaked

folcon19:06:26

Ah, just for clarity, this is running on the clojure side, not clojurescript 😃

quoll19:06:43

OK, that makes sense. However, I think it shows a decoupling that could/should be introduced into Loom: Generation of the .dot file should not be in the same namespace as execution of the engine that renders it

💯 1
quoll19:06:46

ClojureScript makes that decoupling more apparent, since it CAN’T execute anything. It might be able to run an external library (GraphViz in JavaScript), but again, that can be decoupled

folcon23:06:10

Btw, loom/out-degree in asami-loom.index may no longer work with the changed graph structure?

Execution error (ClassCastException) at asami-loom.index/eval51644$fn (index.cljc:53).
class clojure.lang.PersistentArrayMap cannot be cast to class clojure.lang.IPersistentSet (clojure.lang.PersistentArrayMap and clojure.lang.IPersistentSet are in unnamed module of loader 'app')
*e
=>
#error{:cause "class clojure.lang.PersistentArrayMap cannot be cast to class clojure.lang.IPersistentSet (clojure.lang.PersistentArrayMap and clojure.lang.IPersistentSet are in unnamed module of loader 'app')",
       :via [{:type java.lang.ClassCastException,
              :message "class clojure.lang.PersistentArrayMap cannot be cast to class clojure.lang.IPersistentSet (clojure.lang.PersistentArrayMap and clojure.lang.IPersistentSet are in unnamed module of loader 'app')",
              :at [clojure.core$disj invokeStatic "core.clj" 1540]}],
       :trace [[clojure.core$disj invokeStatic "core.clj" 1540]
               [clojure.core$disj invoke "core.clj" 1533]
               [asami_loom.index$eval51644$fn__51694 invoke "index.cljc" 53]
               [loom.graph$eval24928$fn__25005$G__24917__25012 invoke "graph.cljc" 15]
               [fruit_economy.sim.asami$eval66718 invokeStatic "asami.clj" 3]
               [fruit_economy.sim.asami$eval66718 invoke "asami.clj" 192]
               [clojure.lang.Compiler eval "Compiler.java" 7194]
               [clojure.lang.Compiler eval "Compiler.java" 7149]
               [clojure.core$eval invokeStatic "core.clj" 3215]
               [clojure.core$eval invoke "core.clj" 3211]
               [nrepl.middleware.interruptible_eval$evaluate$fn__56324$fn__56325 invoke "interruptible_eval.clj" 87]
               [clojure.lang.AFn applyToHelper "AFn.java" 152]
               [clojure.lang.AFn applyTo "AFn.java" 144]
               [clojure.core$apply invokeStatic "core.clj" 667]
               [clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1990]
               [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1990]
               [clojure.lang.RestFn invoke "RestFn.java" 425]
               [nrepl.middleware.interruptible_eval$evaluate$fn__56324 invoke "interruptible_eval.clj" 87]
               [clojure.main$repl$read_eval_print__9206$fn__9209 invoke "main.clj" 437]
               [clojure.main$repl$read_eval_print__9206 invoke "main.clj" 437]
               [clojure.main$repl$fn__9215 invoke "main.clj" 458]
               [clojure.main$repl invokeStatic "main.clj" 458]
               [clojure.main$repl doInvoke "main.clj" 368]
               [clojure.lang.RestFn invoke "RestFn.java" 1523]
               [nrepl.middleware.interruptible_eval$evaluate invokeStatic "interruptible_eval.clj" 84]
               [nrepl.middleware.interruptible_eval$evaluate invoke "interruptible_eval.clj" 56]
               [nrepl.middleware.interruptible_eval$interruptible_eval$fn__56357$fn__56361
                invoke
                "interruptible_eval.clj"
                152]
               [clojure.lang.AFn run "AFn.java" 22]
               [nrepl.middleware.session$session_exec$main_loop__56427$fn__56431 invoke "session.clj" 218]
               [nrepl.middleware.session$session_exec$main_loop__56427 invoke "session.clj" 217]
               [clojure.lang.AFn run "AFn.java" 22]
               [java.lang.Thread run "Thread.java" 833]]}
This bit specifically I think?
vvvv
(count (disj (apply set/union (vals (spo node))) nil))
The disj gets the below which blows up, though it equally could be that I'm abusing the api =)...
{{:name :good, :quantity 3} {:t 0, :id 1},
 {:name :good, :quantity 1} {:t 0, :id 3},
 {:name :good, :quantity 2} {:t 0, :id 2}}
I'm trying to add attributes all over the place in the graph, which may not be supported...

quoll00:06:21

Using schema, it used to be a map of: {Any {Any #{Any}}} But now it's: {Any {Any {Any {:id long, :tx long}}}} The previous version was like having an innermost map of values to themselves. But now it's a map of values to the statement ID and the transaction ID. So it's similar, but slightly different

folcon00:06:37

Yep, I'm probably going to have to write some convenience functions to handle this.

folcon00:06:31

I'm assuming it's sensible to store node / edge attributes like I'm doing on the graph, or should I follow loom and store them in an :attrs type structure. Loom from my perspective doesn't seem to handle attributes really well? So alternatively I could store them off-graph using the id to track stuff. But I'll have to write a handling layer to manage that... For example:

#asami.index.GraphIndexed{:spo {{:name :raw-material} {{:name :merchant} {{:name :good} {:t 0, :id 1}}}},
                          :pos {{:name :merchant} {{:name :good} {{:name :raw-material} {:t 0, :id 1}}}},
                          :osp {{:name :good} {{:name :raw-material} {{:name :merchant} {:t 0, :id 1}}}},
                          :next-stmt-id 2}

quoll23:06:53

Are you running it against the latest in the main branch?

quoll23:06:50

Oh, I know which change you mean. Yes. It won't work like that, though the update will be very small

quoll23:06:03

I think I can fix this tomorrow

quoll23:06:55

I’m driving up to Baltimore to fly to ClojureD. We're getting there early and I think I can fix this while we're waiting in the airport