Fork me on GitHub
#off-topic
<
2018-05-16
>
dominicm11:05:53

Is there a name for the concept of taking facts, deriving more facts, and "passing it on"? I don't want to do any mutation really (I don't think so anyway), I just want to read in some facts, keep re-deriving until it settles, and then use the resultant data to create a final data structure. I'm not sure if I could perhaps use clara anyway, with an atom or something like it.

dominicm11:05:08

My end-goal is to generate terraform / cloudformation templates fwiw, based on facts.

cvic11:05:30

Maybe spec would help?

cvic11:05:54

Any examples?

dominicm11:05:17

@cvic nothing super-concrete, but I want to assert things ranging from high-level to low.

[?service :service/name "Foobar"]
[?service :service/dns ""]
might go and produce:
[?asg-foobar :asg/name "Foobar"]
[?load-balancer :load-balancer/target ?asg-foobar]
[?hosted-zone :hosted-zone/name ""]
[?dns-a :type :CNAME]
[?dns-a :sub ""]
[?dns-a :record [reference-to-load-balancer-a-url]]
[?dns-b :type :CNAME]
[?dns-b :sub "www"]
[?dns-b :record ""]
Something else might jump in and set some defaults for ?asg-foobar if they aren't specified explicitly, or another rule doesn't add them ( I don't want to get into ordering semantics too much ). Eventually you end up in a format which looks enough like terraform to generate terraform though.

dominicm11:05:43

In theory you could be cloud-agnostic with the really high-level definitions, but I don't see that as particularly realistic.

dominicm12:05:35

I suppose also being able to say "all https must be TLS 1.1 or above" is a fact too.

alexmiller12:05:07

This is what forward-chaining rules engines do

alexmiller12:05:17

Paula Gearon has done a bunch of work in this area in Clojure

dominicm12:05:42

@alexmiller forward-chaining sounds like it might just be the key word for what I'm interested in. I'll take a look at that talk, thanks!

mikerod14:05:27

@U09LZR36F Also, I’ll give a plug for Clara rules, which has a channel #clara

mikerod14:05:21

It’s a rete-based rules engine in Clojure/ClojureScript. It has a fair amount of usage as of now. That’s forward chaining as well. The domain of facts you can provide is more freeform than only datum/triplet sort of things, but you can also use those if you want.

dominicm14:05:12

I'll take another look. I think the mutation-based API put me off, because I'm particularly uninterested in mutation.

mikerod14:05:29

Clara rules expresses the working memory as an immutable, persistent collection

mikerod14:05:38

There is no mutation in the API, so I’m not sure what you mean there 😛

mikerod14:05:24

You can do things like undo etc, with no trouble. Any time you insert a fact, you get a new session and the original is unchanged. Perhaps something in the docs just mislead you to think it was a mutation based API

dominicm14:05:44

I think some other examples I looked at used println too

mikerod14:05:32

println would be something probably only useful for debugging. The rules use a truth maintenance system so the whole concept is that the forward-chaining engine reaches a fixed-point by evaluating rules until there is no more work to do

mikerod14:05:15

insert! itself is only something you do in the RHS of a rule and it means that it is an effect on the working memory state - so yes, like mutation - but it is a transient state of working memory. The fire-rules loop is about ensuring logical consistency across the rules and reaching a fixed point

mikerod14:05:30

then you query from the resulting state. It is more involved than “mutation”

mikerod14:05:18

It’s better to say that it is a change you want to make to working memory, if the left-hand side conditions are satisfied. It may be re-ran if rules flip from satisfied/not-satisfied during the course of evaluation

dominicm14:05:49

Oh, cool. My unfamiliarity with the whole concept mislead me then I think. I'm finding the world of rules very unfamiliar!

dominicm14:05:12

Why not return something from a rule to add to memory?

mikerod14:05:17

The right-hand side gives you the ability to perform arbitrary side-effects, however, that is typically not a good idea since truth maintenance could make things become satisifed but later not satisfied, the engine is smart enough to retract your inserts if they are no longer justified as true.

mikerod14:05:36

rules are not to be thought of as functions

mikerod14:05:39

they don’t “return things”

mikerod14:05:03

your expressing logical conditions to be satisfied and deriving facts as conclusions in rules

mikerod14:05:25

Behind the scenes, there is a “working memory” that is keeping track of all derived facts

mikerod14:05:51

once the rules reach a “fixed point”, where all conditions are satisfied or not, and no pending inserts/retracts are left, you then can query the final state

mikerod14:05:36

I can do something like:

(def empty-session (mk-session my.rules.ns))

(def added-facts (insert-all empty-session my-facts))

(defn query-after-added-facts (query added-facts some-query))

(def added-more-facts (insert-all empty-session more-of-my-facts))

(def query-after-added-more-facts (query added-more-facts some-query))


Each of these vars would hold immutable stuff and you could compare etc

mikerod14:05:18

The rules just express the logical conditions you need to derive facts, the engine figures out which facts can be derived, then you interact with these results via queries

dominicm14:05:47

A lot of the things you're describing sound exactly like what I'm after. I'm a little confused as to the difference between:

(defrule rule-A
  [?a :foo/bar 10]
  =>
  [?a :foo/baz 20])
and
(defrule rule-A
  [?a :foo/bar 10]
  =>
  (insert! [?a :foo/baz 20]))
Essentially, if it's a bad idea to do things other than insert! in the RHS, why not make it implicit?

mikerod14:05:51

I will note that traditional rete implementations (for forward chaining rules) in the past have been based on a mutable working memory API. Clara is fairly unique in that it doesn’t work that way and instead embraces the Clojure immutable persistent data structure concept. So you can read some of the rete literature, but realize that Clara actually has taken it in the (I’d say better) functional/immutable direction. The concept of forward chaining rules engines though is a thing in and of itself that is worth exploring to be able to understand this stuff. My explanations here may likely not be sufficient to explain this stuff.

dominicm14:05:38

Would playing with clara be sufficient to gain a good understanding?

mikerod14:05:24

> Essentially, if it’s a bad idea to do things other than insert! in the RHS, why not make it implicit? Good question! There are certainly advantages to restricting the RHS in what it can do. However, there are tradeoffs. Clara rules was originally developed and used at Cerner (healthcare IT company) and was meant to be able to deal with the production use-cases there. The original author of Clara chose to make it a bit more general purpose in allowing the right-hand side action to have more freedom. This is the traditional way that it has been in all of the Rete algorithms of the past. The algorithm is completely based on the left-hand side and the right-hand side does “any effects”. There is just a special effect of inserting or retracting facts into working memory. However, this “special” effect is by far the most common and in most cases should be all that is needed.

mikerod14:05:57

I’d say that one reason the right-hande side is more freeform than you proposed above, is because you may want to do some other things there with the information gathered on the left-hand side. example:

mikerod15:05:23

(r/defrule doing-things
  [?a-fact <- A]
  =>
  (let [facts (make-facts-from ?a-fact)]
    (r/insert-all! facts)))

mikerod15:05:58

Also, there are more complex things you can do in the left-hand-side. There are things called accumulators, which reason about collections of matching facts at a time, etc. So things can get a bit more complex

mikerod15:05:28

> Would playing with clara be sufficient to gain a good understanding? I think it can be sufficient. I’ve heard of several people who were learning more about forward chaining rules engines and the thought process behind them by just using Clara to try things.

mikerod15:05:45

There is the #clara channel in Slack too as I mentioned. People (including me) answer questions there often.

mikerod15:05:44

Also, the data model that Clara uses to represent rules is possible to be extended upon. You don’t have to use the DSL as given and can make alternatives where you restricted the way the right-hand-side was expressed for example. At that point though, it is more of a syntactic sugar. I will say that I’ve written alternative “DSL-like” layers on top of Clara’s data model for rules before though to make them contain more metadata about the rule and the RHS and I used that data externally do add extra layers of reasoning about the rule network. I’d consider that stuff to be more advanced topic though. Clara does have some built-in tracing and inspection functionality right now to be able to get a feel for what facts were inserted and “why”.

1
dominicm15:05:46

This is really exciting. I'm taking a break now, but I might have a go at implementing my idea this weekend using Clara. Thanks for your help!

emil0r15:05:21

This was very interesting to read. Thank you!

mikerod15:05:54

Awesome. Let me and/or #clara channel know if you have more questions specific to it.

mikerod15:05:11

@U0545PBND Well, glad you found that to be useful

mikerod15:05:52

Oh, I forgot that I also wrote a blog about the basics of forward chaining with Clara in http://www.metasimple.org/2017/02/28/clarifying-rules-engines.html a while back

borkdude19:05:27

Does anyone here use Arduino boards with Mac High Sierra? I can’t detect the board using an USB port

arrdem20:05:24

@borkdude I'm pretty sure that macOS doesn't come with the USB serial drivers that Linux does so Arduinos don't work out of the box. There should be driver downloads on the main http://arduino.cc site IIRC

borkdude21:05:39

@arrdem tried all kinds of wonky stuff suggested at forums. also https://mac-usb-serial.com/, but to no avail

borkdude21:05:24

Maybe buying a Chromebook for programming Arduino is a better option

roklenarcic22:05:04

use a virtualbox with linux instead?