To the behavior side of what is occurring, i believe that this is due to the nature of RHS retracts… The State of the session would likely be as follows: Begin Loop 1: Nothing Step1.1: No Widget found, unconditionally insert default widget (adds activation to queue) End Loop 1 Begin Loop 2: Fire pending activations (first “adding widget”) Step 1.2: Flush default widget Step 2.2: A default widget is available and matches the condition on “set-widget-attrs”, insert custom widget, retract default widget (adds activation to queue) End Loop 2 Begin Loop 3: Fire pending activations (first “setting material”), Insert Custom Widget1 (added to pending updates), Retract Default widget (retracted immediately) Step1.3: Retraction removes the default widget from the session, triggering rule ‘add-widget’ to add an activation as there are technically no other Widgets in the session (only pending) Step2.3: Custom widget1 inserted and flushed End Loop 3 Begin Loop 4: Fire activation (second “adding widget”) Step 1.4: Flush default widget Step 2.4: A default widget is available and matches the condition on “set-widget-attrs”, insert custom widget(2), retract default widget (adds activation to queue) End Loop 4 Begin Loop 5: Fire pending activations (second “setting material”), Insert Custom Widget2 (added to pending updates), Retract Default widget (retracted immediately) Step1.5: Retraction removes the default widget from the session, triggering rule ‘add-widget’ however due to Custom widget 1 being in the session, no activation is added Step2.5: Custom widget2 inserted and flushed End Loop 5 If the above doesn’t make sense, don’t worry its a jumbled mess anyway. While i would like to say that the above is defective, but i have trouble with that… as, if truth maintenance was used, the flushing of the Custom widget should trigger the appropriate clean up of the originating default widget. Unconditional inserts are an attempt to supersede clara’s own logic… and it generally bites unless used sparingly. Additionally RHS retracts add a whole other layer of complexity that should also probably be avoided in general… In my opinion, RHS retracts(retract!) is counter to the general idea that a rules engine is going at… That being that the rules are independent, and only concern themselves with their own LHS and RHS, and the engine can figure out how one rule interplays with another. retract! upends that by allowing one rule to arbitrarily “muck” with another rule. All of that being said, i would probably write the rules slightly differently and i believe that leaving intermediates around is fine.

(ns retraction
  (:require [clara.rules :as r]))

(defrecord Widget [name material length parts])
(defrecord OutputWidget [name material length parts default])

(defn pick-material []
  (first (shuffle ["steel" "silver" "gold"])))
(defn pick-length [mat]
  (condp = mat
    "steel" (+ 10 (rand-int 20))
    "silver" (+ 5 (rand-int 10))
    "gold" (rand-int 10)))

(r/defrule add-widget
  [:not [OutputWidget (false? default)]]
  =>
  (let [new-mat (pick-material)
        len (pick-length new-mat)]
    (r/insert! (->OutputWidget "w1" new-mat len [] true))))

(r/defrule correct-widget
  [Widget (= ?name name) (= "unknown" material) (nil? length) (= ?parts parts)]
  =>
  (let [new-mat (pick-material)
        len (pick-length new-mat)]
    (r/insert! (->OutputWidget ?name new-mat len ?parts false))))

(r/defrule map-widget
  [Widget
   (not (nil? length))
   (not= "unknown" material)
   (= ?name name)
   (= ?material material)
   (= ?length length)
   (= ?parts parts)]
  =>
  (r/insert! (->OutputWidget ?name ?material ?length ?parts false)))

(r/defquery all-output-widgets
  []
  [?w <- OutputWidget])

Sorry for the wall of text.

@ethanc Thanks for the lengthy response. Couple of followups: • Re: insert-conditional! I only did this because I couldn’t get it out of the infinite loop — I expect this is the first issue that most people encounter. • I ran the code above and it works, however I am puzzled by the Widget records. I must be missing an important concept here, They are never inserted so where do they come from? I have been thinking about how to build these types of structures where the attributes of the records have conditions/constraints on what values they can take (sometimes based on other attributes and other times on other record type attributes). In the above examplle these rules are contained in the functions like pick-length. Conceptually I think it would be better if these were created using rules, especially when they get more complex. I see in your example you have added a default attribute which is used to determine when the OutputWidget is complete. In general, how is this done if there were multiple rules that need to be triggered to fill in a record. - do you have a map {attribue-done-flag -> state} for each attribute and built this up as the rules get fired? or just have a longg list of flags? I expect this is a fairly common issue.

Ignore the second point above — I see that only the add-widget rule is actually fired — so the other two don’t do anything hence no need for Widget record at all.

Is there an upper limit on how many facts clara can deal with before experiencing performance issues?

Super vague question, I know

As such, I'm ok with a very vague answer...

In a hypothetical scenario where I have rules segmented by accounts as such:

(defrule hypothetical-example
  [:account (= account "accont-number") (= ?city city)]
  [:route (= ?road road) (= ?zip zip)]
  [:path (= ?magic-code magic-code) (= road ?road) (= city ?city) (= zip ?zip)]
  [:another-thing ...]
  [:woof ...]
  [:moo ...]
  =>
  (println "do something" magic-code))

{:fact-type :path :city "new york" :road "1st road" :zip 93934}

@pdmct, > Re: insert-conditional! I only did this because I couldn’t get it out of the infinite loop — I expect this is the first issue that most people encounter.  most likely, the concept of truth-maintenance is something that requires a bit of a read in, but once understood, generally it should become second nature. > I ran the code above and it works, however I am puzzled by the Widget records. I must be missing an important concept here, They are never inserted so where do they come from? Ignore the second point above — I see that only the add-widget rule is actually fired — so the other two don’t do anything hence no need for Widget record at all. (edited) I will still answer it, as its a valid questions and others might be interested. There are multiple places it could come from, in the example above. Im creating a “default” widget. In the event that the session does not have a widget inserted, it will fabricate it.

(r/defrule add-widget
  [:not [OutputWidget (false? default)]]
  =>
  (let [new-mat (pick-material)
        len (pick-length new-mat)]
    (r/insert! (->OutputWidget "w1" new-mat len [] true))))

(-> session 
  (r/insert (->Widget "ExternalWidget" "copper" 12 [<some parts>]) 
  r/fire-rules)
 

@ethanc thanks for the detailed explanation. Yes the truth maintenance part takes a bit of practice to grok .. getting there. Re: the individual facts yes I considered that would be a useful approach. > Personally, i would use a new model for each transformation and slowly(rule by rule) build to the final. Do you mean something like:

(defrecord BaseWidget [name])
(defrecord MaterialWidget [name material])
(defrecord LengthWidget [name material length])

@petr, It is a vague answer… It depends on the composition and number of the rules and the number or type of facts inserted. For example, Clara is extremely performant… if all of the facts inserted do not match the fact-type of any rule present. 😄 As a more serious answer, we have noted that performance breaks down in cases where sessions have large cartesian products. If the rule that you mentioned had each fact have 100 or 1000 instances, the product area would grow quite fast and would likely cause a degradation in performance. That being said, if you are able to “pre-filter”(add intermediate rules) to reduce this product size, generally the performance is pretty decent. Considering it as a whole, the real answer is going to have to be, it will greatly depend on real world data. One of the reasons that i don’t tout performance, or create rule-based performance test, is because there are too many factors that can fudge the numbers. Which could lead to potential “bad taste” for consumers as they might feel like the metrics are skewed.

@ethanc Thank you for the answer, it's much appreciated. in that case, would you recommend the following:

(defrule hypothetical1
  [:account (= account "accont-number") (= ?city city)]
  [:route (= ?road road) (= ?zip zip)]
  [:path (= ?magic-code magic-code) (= road ?road) (= city ?city) (= zip ?zip)] ;; THere are potentially thousands of these - ONLY one will match though
  =>
  (insert! {:fact-type :found-path ?magic-code}))

(defrule hypothetical2
  [:account (= account "accont-number")]
  [:found-path (= ?magic-code magic-code)]
  [:another-thing ...]
  [:woof ...]
  [:moo ...]
  =>
  (println "do something"))

I definitely understand where you are coming from in terms of promising performance. I'm just trying to determine if I need to approach this problem in another way

potentially yes, however if you were able check/validate, say, “city”/“zip” and remove invalid prior to the join. (this might not be applicable to your system) that would likely be more ideal.

So basically run a filter prior to the facts being asserted? That would be difficult as every "account" has different fields. city,`zip`, and road were just examples I pulled out of my head