I'm experimenting to see if I can rewrite my webapp form logic using Clara rules.

The existing code does multiple passes: Field definitions and values used to generate cleaned field data and errors Field data and errors used by logic which might (1) show/hide fields, (2) disable/enable fields, (3) change field values, (4) manage relationships between fields... (lots of stuff). I borrowed the approach Django uses for form and field validation, they document it well here: https://docs.djangoproject.com/en/1.11/ref/forms/validation/

The current approach has functions which take the form (field definitions, values...) and return a modified version

Iteratively manipulating the data

I don't think Clara would work in this way

My guess is I need each phase to have it's own facts so that each phase of the form processing can use previous phase's facts as inputs.

Not sure if I'm explaining that well

Specific question: Am I correct that it's a bad idea to try and reproduce the "modify data" approach I'm currently using.

@olivergeorge correct, mutating data in Clara is discouraged (though not impossible). I am doing something similar actually with form field- and form-level validation. I don't do multi-pass internally in the rule engine, but rather the rule engine is called multiple times from (essentially) a reduce, which collects the final modifications to perform at the end

so the rule engine doesn't actually modify the forms, rather it returns what operations to perform in order to modify it

then the reduce can take those, perform them, and send the modified form back to the rule engine, and it goes until it stabilizes

Gotcha, thanks @dadair that makes sense

@dadair how do you check for stability? data comparison or does clara provide a "no new facts" test

stability from the rules side of things would be it returning from fire-rules, otherwise it wouldn keep going. "stability" in terms of your domain is different. For example, in that external reduce you would query the session for operations, if the query returns nothing, you could know that "the form is stable"

it all depends on how you set up your system

Thanks again

I am in no way an expert on rule systems though, I'm just getting into it myself, that's just the approach I've taken so far

Right, interesting stuff. I imagine there's typical examples we can learn from in other similar rule systems.

Anyone have any advice re: calming Cursive x cannot be resolved warnings when using Clara?

eg ^

for all the LHS bindings

@dadair Unfortunately there isn’t a good solution for that yet. If the warnings bother you you can turn them off at Settings->Languages & Frameworks->Clojure->Highlight unresolved symbols.

The drawback to that is that you also won’t get symbols that really are undefined highlighted.

Sadly until Cursive understands Clara better (which I am hoping to get to soon-ish) that’s the best option.

Is there a pluggable way for lib maintainers/etc to help with those endeavours?

Not yet, but it’s planned.

If you were really keen, one thing that would help a lot would be to develop specs for the Clara macro forms.

I’m hoping to be able to semi-automatically migrate those to Cursive, and I’m sure it would be a valuable contribution to Clara too.

We’ve discussed doing that in this channel previously, but AFAIK no-one has taken up the torch.

(me included)

@cfleming what format would you prefer specs in? (no promises but curious)

@olivergeorge Just the specs themselves, i.e. the specs defining the grammar of the macro forms.

Maybe we can get specs done through group effort?

@dadair That would be awesome, and I’d like to help.

Clara is integral to my work so don't mind putting some time in if it gets better integrated into cursive

I don’t actually use Clara myself, but I’m keen to use it as an example of migrating a real-world spec to Cursive.

@olivergeorge This sort of thing is what I have in mind: https://github.com/clojure/core.specs.alpha/blob/master/src/main/clojure/clojure/core/specs/alpha.clj

i.e. specs in a form that would be useful for Clara itself too.

Good stuff. Wondered if that's what you meant.

Shouldn't be too bad given they've taken the time to document the syntax with railway tracks in the doco (is that the right term)

Railroad diagrams, yeah

They’re not complete, but they should be a good start.

I suspect there are lots of gnarly corners though, or at least that’s what I’m given to understand. But I think there’s a lot of code using Clara out there so hopefully if we can convince users to test it then we should get the edge cases out fairly quickly.

Sounds good

Having specs would also help non-Cursive users check the grammars too.

Previous discussion: https://clojurians-log.clojureverse.org/clara/2017-06-29.html

One question would be whether the Clara specs would be best developed in Clara itself or in a separate project.

Yeah, I wondered that. I guess it's less intrusive to start with a separate repo. Taking lead from clojure...

I have a meeting soon so signing off

If anyone can tell me what I'm doing wrong with acc/all here I'd love to know: https://gist.github.com/olivergeorge/e37ff239783e1de8f97804bf472047a4

Ok, cool - I’d definitely like to pursue this.

I'll put up my hand to help on the spec front but not familar with clara so someone else would need to lead.

Ok. I can create a project to hold the WIP specs and have a go at it, hopefully later this week.

I'll help where I can if you post the repo

@dadair Great, thanks.

What's the best way for debugging rules? Cursive breakpoints aren't reachable it seems (not surprising since the rules aren't "called", but rather exist within a session registry and are also macros). Currently I just add print statements and try to reason things out

@dadair some of this may be helpful http://www.clara-rules.org/docs/inspection/

Sometimes I end up just mutating some testing atoms state from various points in rules though when I’m confused. The inspection may be a bit of a nicer approach.

There is clara.tools.inspect and clara.tools.trace that could both potentially be useful. The tests for these features in src/test may be helpful to see some usages.

great I'll take a look at that thanks

@dadair Tests of the tracing like https://github.com/cerner/clara-rules/blob/0.15.2/src/test/common/clara/tools/test_tracing.cljc#L67 may be useful

inspection is more human-readable I think

but the tracing sometimes has good data to work with, but can be a lot so have to pikc it apart some

Clara-rules 0.16.0 is released and on Clojars. Relative to 0.15.2 (the previous released version) this is a bugfix release for https://github.com/cerner/clara-rules/issues/303

Is there a benefit/drawback to doing:

[A (false? x)]

[A (= ?x x)]
[:test (false? ?x)]

other than if you want to use ?x later in the RHS

@dadair I’d tend to just suggest to not use :test unless you have a use-case that it can’t be avoided

it just is more opaque to the engine and could mean you’re skipping out on optimizations

makes sense, thanks!

As far as the actual effects of the example you give here, I don’t know there is a big difference. There is definitely a “little” more overhead to adding the :test node, but I’m not sure you lose any particular optimizations in this particular case. I’d have to mess with it more to be sure though. However, still, I’d just go with my general rule here. 🙂

absolutely, thanks for the help!

I'm trying to encode some form logic which disables dependent fields (e.g. can't pick building if site isn't set, can't pick floor if building isn't set...)

My rules don't terminate

(ns spike-rules.core
  (:require [clara.rules :as rules :refer [insert! defrule defquery defsession]]
            [clara.rules.accumulators :as acc]))

(defrecord Field [field-key field-state])
(defrecord FieldState [field-key field-state])
(defrecord Operation [field-key field-state])

(defrule rule1
  "Insert FieldState based on Field and Operations"
  [?field <- Field (= field-key ?field-key)]
  [?ops <- (acc/all :field-state) :from [Operation (= field-key ?field-key)]]
  =>
  (println :update-fieldstate ?field-key)
  (insert! (->FieldState ?field-key (reduce merge (:field-state ?field) ?ops))))

(defrule rule2
  "Insert field disable operation if a field depends on a field which is not set"
  [FieldState (= ?nil-field-key field-key) (= (nil? (:value field-state)))]
  [FieldState (= ?disable-field-key field-key) (contains? (:depends-on field-state) ?nil-field-key)]
  =>
  (println :disable-dependent ?disable-field-key)
  (insert! (->Operation ?disable-field-key {:disabled true})))

(defsession form-session [rule1 rule2])

(-> form-session
    (rules/insert
      (->Field :site {:value nil})
      (->Field :bl {:value nil :depends-on #{:site}})
      (->Field :fl {:value nil :depends-on #{:bl}}))
    (rules/fire-rules))

Perhaps rule1 is bad practice. I'm using it to combine the original field state with any operations which modify it. (in this case adding a :disable flag)

@dadair I was trying to avoid having to do the "merge field with ops" manually and refiring rules. That's the approach you were describing, correct?

We actually do it slightly differently. In our case, the rule engine inserts the current state of the form, and the operation the user is trying to perform. Then there are rules that check if that operation is valid given the current state of the form. If it's valid, it returns a transformation of the input action that can actually be performed/persisted by an external state management service. If it's invalid, it transforms the input action to an action that will invalidate the field (or reject the action out-right), which is then again carried out by the external state management service

Might add in a

(println "?ops " ?ops)

@defndaines that shows that rule1 is firing repeatedly for the empty ops case.

Loading src/spike_rules/round1.clj... 
:update-fieldstate :site []
:update-fieldstate :fl []
:update-fieldstate :bl []
:disable-dependent :fl
:disable-dependent :bl
:update-fieldstate :fl []
:update-fieldstate :bl []
:disable-dependent :bl
:disable-dependent :fl
:update-fieldstate :bl []
:update-fieldstate :fl []

@dadair thanks, as you say it's slightly different to what I'm focused on but sounds interesting... I'll get to that I expect.

Looks like I'm not best friends with acc/all yet

Maybe check with some?

@defndaines I don't think that would work. I want field -> fieldstate even if ops is empty. That way rule2 has a full list of field-state facts to consider.

Problem is that acc/all doesn't seem to ever return the inserted Operations facts

There's a feedback loop which seems a possible problem: * rule 1 inserts FieldState (before any Operations are inserted) * rule 2 fires because it sees a FieldState which should be disabled * rule 1 sees a new Operation and does truth maintenance to update the field's FieldState

Odd thing is that I don't ever see the Operation in Rule 1 via acc/all (the third step described)

I'm new to this, likely user error or bad assumptions on how things should work

@olivergeorge acc/all returns an empty collection when no matches.

I’m not sure that you’ve already realized that, so pointing it out

(r/defrule foo
 [?xs <- (acc/all) :from [X]]
 [:test (seq ?xs)]
 =>
 (prn :foo))

Looks like the acc/all docs could be a bit clearer on that

Believe it or not, it actually can be useful for it to match with the empty collection in other use-cases.

@mikerod Thanks Mike. The curious thing is that an inserted Operation is never returned from (all). I'll take a step back to understand truth maintenance better.

Simpler example

(ns spike-rules.round1
  (:require [clara.rules :as rules :refer [insert! defrule defquery defsession]]
            [clara.rules.accumulators :as acc]))

(defrecord Field [field-key field-state])
(defrecord FieldState [field-key field-state])
(defrecord Operation [field-key field-state])

(defrule rule1
  "Insert FieldState based on Field and Operations"
  [?field <- Field (= field-key ?field-key)]
  [?ops <- (acc/all) :from [Operation (= field-key ?field-key)]]
  =>
  (println :rule1.ops ?ops)
  (println :rule1.insert! `(->FieldState ~?field-key ~(reduce merge (:field-state ?field) ?ops)))
  (insert! (->FieldState ?field-key (reduce merge (:field-state ?field) ?ops))))

(defrule rule2
  "disable if read-only"
  [FieldState (= ?field-key field-key) (:read-only field-state)]
  =>
  (println :rule2.insert! `(->Operation ~?field-key {:disabled true}))
  (insert! (->Operation ?field-key {:disabled true})))

(defsession form-session [rule1 rule2])

(-> form-session
    (rules/insert
      (->Field :site {:value nil :read-only true}))
    (rules/fire-rules))

Loading src/spike_rules/round1.clj... 
:rule1.ops []
:rule1.insert! (spike-rules.round1/->FieldState :site {:value nil, :read-only true})
:rule2.insert! (spike-rules.round1/->Operation :site {:disabled true})
:rule1.ops []
:rule1.insert! (spike-rules.round1/->FieldState :site {:value nil, :read-only true})
:rule2.insert! (spike-rules.round1/->Operation :site {:disabled true})
...

You could write a query to return all operations, if that query returns anything, then your memory does have operations inserted and it's your LHS of rule 1 that is incorrect

Okay, thanks will do.

Actually, not sure how to do a query when it won't terminate.

I appreciate all the help

I'll take some time to learn more and try again.