Fork me on GitHub
#off-topic
<
2020-07-03
>
p-himik08:07:12

I'm in need for some suggestions. I need to be able to create, manipulate, and serialize a graph. This graph can have cycles, each node is a map where keys are just keywords and values are either regular values of any kind or a link to a node or a vector/map of links to nodes. Possible manipulations include everything - changing/adding/removing nodes and node attributes, without any assumptions. The serialization needs to be fully controlled - a thirdparty library will be the recipient of the data. A very nice thing to have would be a nice user-friendly interface. I'm porting a Python library to Clojure and it would be great if the users could manipulate the graph easily, without me having to write wrappers for all operations. Are there projects that do something like that? Or maybe it's actually pretty simple to implement and I just don't see it?

p-himik08:07:21

As an additional complication - all isolated nodes need to be automatically removed on each graph change. E.g. if I have a graph like

{:root {:children [(node :A)]}
 :A {:x (node :B)}
 :B {:text "hello"}}
and then do (assoc-in graph [:A :x] 1) or something equivalent (but probably using the interface of some graph library), the node :B must be removed.

gklijs14:07:05

All that together doesn't sound simple. My first guess would be a graph db would be a good starting point. https://github.com/gorillalabs/neo4j-clj/blob/master/example/src/example/core.clj but might be overkill. Also depends on size/availability etc.

p-himik14:07:37

Thanks! Oh, it all has to be in memory. Availability and everything around it is not a concern as I'm trying to create a library and not a framework or a platform.

phronmophobic16:07:10

you may want to check out loom and loom compatible libraries like ubergraph. https://github.com/aysylu/loom https://github.com/Engelberg/ubergraph there's even an function for finding isolated nodes: https://cljdoc.org/d/aysylu/loom/1.0.2/api/loom.alg#loners

phronmophobic16:07:50

I believe you can even use your own representation and still have all the algorithmic functions apply if you implement the appropriate interfaces, https://cljdoc.org/d/aysylu/loom/1.0.2/api/loom.graph

p-himik17:07:50

Yep, those were easy to find, so I've checked them out. But using them would really just be shoehorning a library only because it has the word "graph" in its description IMO. You see, it doesn't distinguish between different kinds of edges. Consider this graph, for example (using capitalized keywords to denote nodes):

{:Plot {:left [:Axis], :right [:Axis, :ColorBar]}
 :Axis {}
 :ColorBar {}}
Modeling this in terms of nodes and edges as loom and ubergraph think about them would be... problematic.

phronmophobic17:07:45

edges can have attributes

phronmophobic17:07:50

and you can use your own representation and still use the loom algorithms

lilactown17:07:06

mapgraph might be more up your alley: https://github.com/stuartsierra/mapgraph

lilactown17:07:08

it's less general than loom or ubergraph but more in the realm of what you seem to be looking for: talking about simple data that happens to be a graph

lilactown17:07:32

rather than treating the graph as 1st class and doing graph-y things with it

p-himik17:07:28

@U7RJTCH6J Mmm, yeah, then my statement about "different kinds of edges" is incorrect. Still though - it would be quite hard to model that simple map as a loom graph. @U4YGF4NGM Thanks, I'll check it out!

phronmophobic17:07:11

yea, I'm still not sure it's a good fit for your use case, but if that was the only thing holding you back, it still might have been

lilactown17:07:08

there's a couple other libs out there similar to it. something like datascript could also be nice, if you want advanced query-ability

lilactown17:07:24

like this one has a more specific (and complex) API but it's still all about data, less about graphs: https://keechma.com/guides/entitydb/

p-himik18:07:46

Nice, thanks. I definitely want to avoid DataScript, I'm trying to keep it as close to the real data as possible.

emccue07:07:38

afaik, there are only a few ways to implement a graph

emccue07:07:06

1. Mutable stuff with nodes that point to other nodes. this is usually pretty silly

emccue07:07:05

2. Map<NodeId, Set<NodeId>>, Map<NodeId, Node>

emccue07:07:58

3. Map<NodeId, Set<(NodeId, Attrs)>>, Map<NodeId, Node>

emccue07:07:19

sounds like you want number 3

p-himik07:07:44

I don't understand what you're saying. The code blocks that I have provided above already form a graph. Just not in a way that various graph libraries might want.

lilactown07:07:46

how do you intend to serialize it?

p-himik08:07:25

The way the target library wants it. Every node will be assigned an ID and put in a id->node map, and every node reference will be replaced with {:id id}. That has nothing to do with the aforementioned "ways to implement a graph", if that's what you're hinting at.

lockejan16:07:21

regarding the video streaming topic: discord is IMO the worst option here. We recently switched from googleMeet to discord at work but it keeps crashing big times on a lot of machines here (mostly linux)… Jitsi is also an option to consider. Its open source so it’s even possible to host it yourself if one is keen to do it.

3
noisesmith16:07:10

yeah jitsi is the best

erwinrooijakkers17:07:52

I feel most happy with Jitsi too. The fact that you can host it yourself, you can read the code, and it seems to respect your privacy. Nice feature: see time every participants has spoken.

noisesmith17:07:58

oh yeah - that feature promotes a "competition" to see who can do the most listening :D

walterl01:07:02

I love Jitsi too. Just note that E2E encryption isn't enabled by default. You have to set a key manually, and it only works on Chrom* 83+. See https://jitsi.org/security/

erwinrooijakkers17:07:44

Hi all, is there a command to bump all dependencies in project.clj to their latest version?

noisesmith17:07:25

you might try in #lein but I think lein ancient has an option for this

jsyrjala17:07:37

lein ancient upgrade :no-tests :check-clojure

erwinrooijakkers17:07:47

Fantastic thank you

emccue07:07:34

"Do it, intern"