This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2017-04-21
Channels
- # aws (1)
- # aws-lambda (1)
- # beginners (27)
- # boot (16)
- # cider (1)
- # clara (54)
- # cljs-dev (4)
- # cljsjs (8)
- # cljsrn (25)
- # clojure (148)
- # clojure-dev (2)
- # clojure-finland (1)
- # clojure-france (18)
- # clojure-italy (10)
- # clojure-nl (3)
- # clojure-russia (27)
- # clojure-sg (2)
- # clojure-uk (17)
- # clojurebridge (6)
- # clojurescript (70)
- # core-async (1)
- # css (6)
- # cursive (35)
- # data-science (3)
- # datomic (22)
- # events (4)
- # jobs (18)
- # jobs-discuss (14)
- # leiningen (4)
- # lumo (22)
- # off-topic (20)
- # om (5)
- # om-next (1)
- # onyx (47)
- # pedestal (107)
- # re-frame (43)
- # reagent (1)
- # ring (2)
- # ring-swagger (2)
- # rum (18)
- # sql (15)
- # unrepl (4)
- # vim (61)
- # yada (3)
@wparker we built something hackish to build graphviz, and for the entire session the graph was way too big to be useful
what i found myself wanting was a way to define views over the graph, like, i may not care about all the edges and nodes between Node A and Node B, so I'd like to define a view that shows me Node A -> Node B without all of the intermediate steps
I imagine having something like:
(def views {:everything [FactA FactB ...], :simple [FactA FactC FactF]})
what we built relied on the data returned from inspect, which IIRC required us to fire-rules
im interested in building the graph of all the relationships, not just the relationships that survive after running the rules
The graph would include all logical insertions regardless of whether they were used for anything, but unconditional/external insertions wouldn’t be if they weren’t used for anything
provide functions to retrieve particular subsets of the graph, potentially modify it, etc.
and then look at how to display those views. Basically have a data-oriented approach to the problem that different views could layer on top of
In particular, in our case we’d probably want to combine fact graphs with some other indicators that our specific to our internal frameworks
I’m not clear on what :simple would be - do you mean that if you had A -> B -> C you’d have a direct edge of A -> C?
Also, @devn I’d be curious about why cytoscape.js versus any other graph visualization options you tried or considered - the visualization layer isn’t something I know much about yet
@wparker I did not directly participate in the decision to use cytoscape, but as I recall it was chosen partly for performance, and partly because it made the traversal of subgraphs relatively easy.
basically anything to make it easy to skip the display of 382910 nodes and edges that don't actually tell you much that you care to know about getting from A to C
i think it's often the case that the fact that you got to C suggests everything you'd need to understand about B
for instance, there is a big fan out and fan in between A and C, like A->[B1, B2, B3, B4, B5, BN]->C If C is a specific fact like: MoreThanThreeBs, then who really cares what those Bs were? all I really care about is that by virtue of A, I have the possibility of many Bs, and all I care to know is if I get to state C from A
We have a lot of scenarios where we need to look at hundreds of thousands of facts on our way to a state, but in the end all we really care about is that given a Lab, we determine whether it is normal or abnormal according to a massive set of facts, and then produce an AbnormalLabResult or NormalLabResult. Labl->AbnormalLab/NormalLab is how I'd prefer to view it in a UI rather than render the 500k+ nodes and corresponding edges in between
being able to express something like: "Provided #{A}, render direct edges to #{C, D} through nodes of type #{B}" would be nice
I suppose I imagine that for any collection of rules and facts, there are layers to the view. For instance, I don't care about seeing the source of a rule by default, that's another layer. Similarly, I don't care which specific constraints in that rule were met, unless I do.
I can see where that would be useful for some rules patterns, yes. I’m thinking that it probably wouldn’t be too hard to transform the graph down to that level though?
I'm just listing stuff that I don't think would be totally out of bounds for clara.tools.*
By the way, I think having transformation functions in the fact-graph namespace could be useful, I just haven’t gotten to that point yet
I totally understand for instance, that if you have the raw data, you can do whatever you please, but it would be really nice to not leave that as an exercise for the reader, and instead have a couple of basic built-in tools for specifying what you want to look at
Like, a function which dumps graphviz, even if there is no dependency within clara to actually render it, seems like it might be useful
and then adding on to that: Dumps graphviz where you specify the starting node (the type of fact), and how many hops should be rendered
For that, I’m thinking you’d do something like (-> original-graph remove-intermediate-steps render-to-graphviz)
It would be interesting to generate some anonymized graphs for all of our projects just to get a sense for what transformation functions would be generally useful
I know there are Reasons™ to not include the transformation kitchen sink or unnecessary deps, but I feel like a simple toolkit for rules engine graph generation would be super helpful to have, maybe even as part of core
Yeah, there is often a tradeoff of not tying things together too much versus not providing enough functionality especially if you get into needing other libraries as dependencies. Adding generic data transforms doesn’t seem like it adds too much burden though since it is very easily ignorable if you don’t need them
I’m guessing the types of transforms needed will depend heavily on the domain. For example, in our case, I think a lot of it would revolve around taking a given fact and showing everything either upstream or downstream from that fact, and potentially “backtracking” a bit from some joins on that path, but not the rest of the rules session
If you have a fact A that flows through rules B->C->D->E->F and you don’t get the expected F it would be nice to be able to easily see where it “stopped”