Fork me on GitHub
#clara
<
2017-09-13
>
Oliver George00:09:06

I'm experimenting to see if I can rewrite my webapp form logic using Clara rules.

Oliver George00:09:37

The existing code does multiple passes: Field definitions and values used to generate cleaned field data and errors Field data and errors used by logic which might (1) show/hide fields, (2) disable/enable fields, (3) change field values, (4) manage relationships between fields... (lots of stuff). I borrowed the approach Django uses for form and field validation, they document it well here: https://docs.djangoproject.com/en/1.11/ref/forms/validation/

Oliver George00:09:58

The current approach has functions which take the form (field definitions, values...) and return a modified version

Oliver George00:09:12

Iteratively manipulating the data

Oliver George00:09:24

I don't think Clara would work in this way

Oliver George00:09:09

My guess is I need each phase to have it's own facts so that each phase of the form processing can use previous phase's facts as inputs.

Oliver George00:09:21

Not sure if I'm explaining that well

Oliver George00:09:08

Specific question: Am I correct that it's a bad idea to try and reproduce the "modify data" approach I'm currently using.

dadair00:09:42

@olivergeorge correct, mutating data in Clara is discouraged (though not impossible). I am doing something similar actually with form field- and form-level validation. I don't do multi-pass internally in the rule engine, but rather the rule engine is called multiple times from (essentially) a reduce, which collects the final modifications to perform at the end

dadair00:09:00

so the rule engine doesn't actually modify the forms, rather it returns what operations to perform in order to modify it

dadair00:09:22

then the reduce can take those, perform them, and send the modified form back to the rule engine, and it goes until it stabilizes

Oliver George00:09:35

Gotcha, thanks @dadair that makes sense

Oliver George01:09:48

@dadair how do you check for stability? data comparison or does clara provide a "no new facts" test

dadair01:09:52

stability from the rules side of things would be it returning from fire-rules, otherwise it wouldn keep going. "stability" in terms of your domain is different. For example, in that external reduce you would query the session for operations, if the query returns nothing, you could know that "the form is stable"

dadair01:09:09

it all depends on how you set up your system

dadair01:09:33

I am in no way an expert on rule systems though, I'm just getting into it myself, that's just the approach I've taken so far

Oliver George01:09:38

Right, interesting stuff. I imagine there's typical examples we can learn from in other similar rule systems.

dadair03:09:29

Anyone have any advice re: calming Cursive x cannot be resolved warnings when using Clara?

dadair03:09:44

for all the LHS bindings

cfleming03:09:08

@dadair Unfortunately there isn’t a good solution for that yet. If the warnings bother you you can turn them off at Settings-&gt;Languages &amp; Frameworks-&gt;Clojure-&gt;Highlight unresolved symbols.

cfleming03:09:32

The drawback to that is that you also won’t get symbols that really are undefined highlighted.

cfleming03:09:00

Sadly until Cursive understands Clara better (which I am hoping to get to soon-ish) that’s the best option.

dadair03:09:22

Is there a pluggable way for lib maintainers/etc to help with those endeavours?

cfleming03:09:18

Not yet, but it’s planned.

cfleming03:09:40

If you were really keen, one thing that would help a lot would be to develop specs for the Clara macro forms.

cfleming03:09:15

I’m hoping to be able to semi-automatically migrate those to Cursive, and I’m sure it would be a valuable contribution to Clara too.

cfleming03:09:33

We’ve discussed doing that in this channel previously, but AFAIK no-one has taken up the torch.

cfleming03:09:40

(me included)

Oliver George03:09:25

@cfleming what format would you prefer specs in? (no promises but curious)

cfleming03:09:00

@olivergeorge Just the specs themselves, i.e. the specs defining the grammar of the macro forms.

dadair03:09:00

Maybe we can get specs done through group effort?

cfleming03:09:18

@dadair That would be awesome, and I’d like to help.

dadair03:09:43

Clara is integral to my work so don't mind putting some time in if it gets better integrated into cursive

cfleming03:09:21

I don’t actually use Clara myself, but I’m keen to use it as an example of migrating a real-world spec to Cursive.

cfleming03:09:31

i.e. specs in a form that would be useful for Clara itself too.

Oliver George03:09:47

Good stuff. Wondered if that's what you meant.

Oliver George03:09:24

Shouldn't be too bad given they've taken the time to document the syntax with railway tracks in the doco (is that the right term)

cfleming03:09:37

Railroad diagrams, yeah

cfleming03:09:48

They’re not complete, but they should be a good start.

cfleming03:09:44

I suspect there are lots of gnarly corners though, or at least that’s what I’m given to understand. But I think there’s a lot of code using Clara out there so hopefully if we can convince users to test it then we should get the edge cases out fairly quickly.

cfleming03:09:15

Having specs would also help non-Cursive users check the grammars too.

cfleming03:09:56

One question would be whether the Clara specs would be best developed in Clara itself or in a separate project.

Oliver George03:09:37

Yeah, I wondered that. I guess it's less intrusive to start with a separate repo. Taking lead from clojure...

Oliver George03:09:44

I have a meeting soon so signing off

Oliver George03:09:59

If anyone can tell me what I'm doing wrong with acc/all here I'd love to know: https://gist.github.com/olivergeorge/e37ff239783e1de8f97804bf472047a4

Oliver George07:09:56

Worked it out. bidi required an earlier version of prismatic so I missed this fix https://github.com/cerner/clara-rules/issues/306

cfleming03:09:06

Ok, cool - I’d definitely like to pursue this.

Oliver George03:09:30

I'll put up my hand to help on the spec front but not familar with clara so someone else would need to lead.

cfleming03:09:57

Ok. I can create a project to hold the WIP specs and have a go at it, hopefully later this week.

dadair04:09:48

I'll help where I can if you post the repo

dadair17:09:27

What's the best way for debugging rules? Cursive breakpoints aren't reachable it seems (not surprising since the rules aren't "called", but rather exist within a session registry and are also macros). Currently I just add print statements and try to reason things out

mikerod17:09:50

Sometimes I end up just mutating some testing atoms state from various points in rules though when I’m confused. The inspection may be a bit of a nicer approach.

mikerod17:09:30

There is clara.tools.inspect and clara.tools.trace that could both potentially be useful. The tests for these features in src/test may be helpful to see some usages.

dadair17:09:23

great I'll take a look at that thanks

mikerod18:09:59

inspection is more human-readable I think

mikerod18:09:16

but the tracing sometimes has good data to work with, but can be a lot so have to pikc it apart some

wparker19:09:16

Clara-rules 0.16.0 is released and on Clojars. Relative to 0.15.2 (the previous released version) this is a bugfix release for https://github.com/cerner/clara-rules/issues/303

dadair20:09:02

Is there a benefit/drawback to doing:

[A (false? x)]
instead of
[A (= ?x x)]
[:test (false? ?x)]

wparker20:09:29

For constraints that can be determined to be true or false without any reference to other facts Clara will evaluate them upfront without trying to join them with anything and if they are false Clara won’t retain a reference to them

wparker20:09:59

So for example, in the latter case, I’d expect Clara to retain a hard reference to every A inserted

wparker20:09:22

while in the former case, it wouldn’t retain a reference to an A without a value of false in field x

wparker20:09:57

In cases where you do joins it can also be more efficient to have Clara do as much filtering as it can on each fact in isolation - apart from evaluating your join criteria, there is some (highly optimized) overhead to keeping track of joins

dadair20:09:27

ok great thanks

dadair20:09:21

other than if you want to use ?x later in the RHS

mikerod21:09:42

@dadair I’d tend to just suggest to not use :test unless you have a use-case that it can’t be avoided

mikerod21:09:55

it just is more opaque to the engine and could mean you’re skipping out on optimizations

dadair21:09:48

makes sense, thanks!

mikerod21:09:22

As far as the actual effects of the example you give here, I don’t know there is a big difference. There is definitely a “little” more overhead to adding the :test node, but I’m not sure you lose any particular optimizations in this particular case. I’d have to mess with it more to be sure though. However, still, I’d just go with my general rule here. 🙂

dadair21:09:56

absolutely, thanks for the help!

Oliver George22:09:24

I'm trying to encode some form logic which disables dependent fields (e.g. can't pick building if site isn't set, can't pick floor if building isn't set...)

Oliver George22:09:58

My rules don't terminate

Oliver George22:09:18

(ns spike-rules.core
  (:require [clara.rules :as rules :refer [insert! defrule defquery defsession]]
            [clara.rules.accumulators :as acc]))

(defrecord Field [field-key field-state])
(defrecord FieldState [field-key field-state])
(defrecord Operation [field-key field-state])

(defrule rule1
  "Insert FieldState based on Field and Operations"
  [?field <- Field (= field-key ?field-key)]
  [?ops <- (acc/all :field-state) :from [Operation (= field-key ?field-key)]]
  =>
  (println :update-fieldstate ?field-key)
  (insert! (->FieldState ?field-key (reduce merge (:field-state ?field) ?ops))))

(defrule rule2
  "Insert field disable operation if a field depends on a field which is not set"
  [FieldState (= ?nil-field-key field-key) (= (nil? (:value field-state)))]
  [FieldState (= ?disable-field-key field-key) (contains? (:depends-on field-state) ?nil-field-key)]
  =>
  (println :disable-dependent ?disable-field-key)
  (insert! (->Operation ?disable-field-key {:disabled true})))

(defsession form-session [rule1 rule2])

(-> form-session
    (rules/insert
      (->Field :site {:value nil})
      (->Field :bl {:value nil :depends-on #{:site}})
      (->Field :fl {:value nil :depends-on #{:bl}}))
    (rules/fire-rules))

Oliver George22:09:14

Perhaps rule1 is bad practice. I'm using it to combine the original field state with any operations which modify it. (in this case adding a :disable flag)

Oliver George22:09:36

@dadair I was trying to avoid having to do the "merge field with ops" manually and refiring rules. That's the approach you were describing, correct?

dadair22:09:08

We actually do it slightly differently. In our case, the rule engine inserts the current state of the form, and the operation the user is trying to perform. Then there are rules that check if that operation is valid given the current state of the form. If it's valid, it returns a transformation of the input action that can actually be performed/persisted by an external state management service. If it's invalid, it transforms the input action to an action that will invalidate the field (or reject the action out-right), which is then again carried out by the external state management service

defndaines22:09:06

Might add in a

(println "?ops " ?ops)
as part of rule1.

Oliver George23:09:12

@defndaines that shows that rule1 is firing repeatedly for the empty ops case.

Loading src/spike_rules/round1.clj... 
:update-fieldstate :site []
:update-fieldstate :fl []
:update-fieldstate :bl []
:disable-dependent :fl
:disable-dependent :bl
:update-fieldstate :fl []
:update-fieldstate :bl []
:disable-dependent :bl
:disable-dependent :fl
:update-fieldstate :bl []
:update-fieldstate :fl []
Hmm, that makes me think it's not doing what I expect. It should have an operation for disabling that field.

Oliver George23:09:19

@dadair thanks, as you say it's slightly different to what I'm focused on but sounds interesting... I'll get to that I expect.

Oliver George23:09:29

Looks like I'm not best friends with acc/all yet

defndaines23:09:35

Maybe check with some?

Oliver George23:09:45

@defndaines I don't think that would work. I want field -> fieldstate even if ops is empty. That way rule2 has a full list of field-state facts to consider.

Oliver George23:09:12

Problem is that acc/all doesn't seem to ever return the inserted Operations facts

Oliver George23:09:57

There's a feedback loop which seems a possible problem: * rule 1 inserts FieldState (before any Operations are inserted) * rule 2 fires because it sees a FieldState which should be disabled * rule 1 sees a new Operation and does truth maintenance to update the field's FieldState

Oliver George23:09:26

Odd thing is that I don't ever see the Operation in Rule 1 via acc/all (the third step described)

Oliver George23:09:55

I'm new to this, likely user error or bad assumptions on how things should work

mikerod23:09:00

@olivergeorge acc/all returns an empty collection when no matches.

mikerod23:09:16

I’m not sure that you’ve already realized that, so pointing it out

mikerod23:09:14

(r/defrule foo
 [?xs <- (acc/all) :from [X]]
 [:test (seq ?xs)]
 =>
 (prn :foo))
Something like that would ensure that the rule wasn’t activated with no matches

mikerod23:09:06

Looks like the acc/all docs could be a bit clearer on that

mikerod23:09:50

Believe it or not, it actually can be useful for it to match with the empty collection in other use-cases.

Oliver George23:09:10

@mikerod Thanks Mike. The curious thing is that an inserted Operation is never returned from (all). I'll take a step back to understand truth maintenance better.

Oliver George23:09:48

Simpler example

(ns spike-rules.round1
  (:require [clara.rules :as rules :refer [insert! defrule defquery defsession]]
            [clara.rules.accumulators :as acc]))

(defrecord Field [field-key field-state])
(defrecord FieldState [field-key field-state])
(defrecord Operation [field-key field-state])

(defrule rule1
  "Insert FieldState based on Field and Operations"
  [?field <- Field (= field-key ?field-key)]
  [?ops <- (acc/all) :from [Operation (= field-key ?field-key)]]
  =>
  (println :rule1.ops ?ops)
  (println :rule1.insert! `(->FieldState ~?field-key ~(reduce merge (:field-state ?field) ?ops)))
  (insert! (->FieldState ?field-key (reduce merge (:field-state ?field) ?ops))))

(defrule rule2
  "disable if read-only"
  [FieldState (= ?field-key field-key) (:read-only field-state)]
  =>
  (println :rule2.insert! `(->Operation ~?field-key {:disabled true}))
  (insert! (->Operation ?field-key {:disabled true})))

(defsession form-session [rule1 rule2])

(-> form-session
    (rules/insert
      (->Field :site {:value nil :read-only true}))
    (rules/fire-rules))
Returns
Loading src/spike_rules/round1.clj... 
:rule1.ops []
:rule1.insert! (spike-rules.round1/->FieldState :site {:value nil, :read-only true})
:rule2.insert! (spike-rules.round1/->Operation :site {:disabled true})
:rule1.ops []
:rule1.insert! (spike-rules.round1/->FieldState :site {:value nil, :read-only true})
:rule2.insert! (spike-rules.round1/->Operation :site {:disabled true})
...

dadair23:09:49

You could write a query to return all operations, if that query returns anything, then your memory does have operations inserted and it's your LHS of rule 1 that is incorrect

Oliver George23:09:06

Okay, thanks will do.

Oliver George23:09:02

Actually, not sure how to do a query when it won't terminate.

Oliver George23:09:05

I appreciate all the help

Oliver George23:09:12

I'll take some time to learn more and try again.