Fork me on GitHub
Konstantinos Tzanidakis14:01:01

Hi all, I want to serialize/deserialize the working-memory separately so I started off by using a simple implementation for IWorkingMemorySerializer that serializes to an atom (as per Clara's docs) and deserializing from the atom by derefing it. That works perfectly fine, in that I am able to run queries and get the same results from both the original and deserialized session. However, if I add the extra step to write to fressian and read fressian to the ser/deserializer of the IWorkingMemorySerializer , all queries in the deserialized version of the session return with nil: eg. ({:?crn nil}) while the same query on the original is returning ({:?crn {:fact-type :client-representative-notification, :name "Alice", :client "Acme", :level :high}}) . I should note that I am fairly new to clara rules. Has anyone come across something like this before?


@konstantinos562, Do you have a small snippet, it would help me reproduce it locally?

Konstantinos Tzanidakis15:01:24

Hi @ethanc do you need the rules as well?

Konstantinos Tzanidakis15:01:08

(defrecord WorkingMemoryAtomSerializer [input output]
  (serialize-facts [_ facts]
    (reset! state (write-fressian facts)))
  (deserialize-facts [this]
    (-> @state io/input-stream read-fressian)))

Konstantinos Tzanidakis15:01:12

with (def state (atom nil)) in the namespace

Konstantinos Tzanidakis15:01:36

and the read and write fressian as follows

Konstantinos Tzanidakis15:01:40

(defn write-fressian [obj]
  (let [baos (ByteArrayOutputStream.)
        wr   (frz/create-writer baos)]
    (frz/write-object wr obj)
    (.toByteArray baos)))

(defn read-fressian [b]
  (-> b frz/create-reader frz/read-object))

Konstantinos Tzanidakis15:01:47

sorry for the long posting all

Konstantinos Tzanidakis15:01:50

NB: I have replaced fressian with nippy/freeze and nippy/thaw and it works perfectly fine now. I am not sure what the issue was (most probably I am doing something wrong there)


@konstantinos562 not sure what frz ns is referring to


You may just not have setup good read/write handlers for your facts


Fressian’s defaults tend to be insufficient

Konstantinos Tzanidakis15:01:36

sorry it's the requirement: [ :as frz]


or they will cause many clj data structures to become Java structures - or not the same as before


you can see Clara defines a more robust set of fressian handlers for clj to clj serialization


however, there really is no compelling reason to not just use nippy if you have it set up


I wouldn’t have used Fressian to impl Clara’s session serialization if I were to do it again today. it is difficult to work with in what it provides by default and I don’t think it is any better than something like nippy or perhaps a few others. It is also not documented super well and at the “primitive” level makes some unavoidable Java collections that are awkward to convert from (and also wasting perf time)

Konstantinos Tzanidakis15:01:11

for this example, I have not set up any handlers but the facts are fairly simple

[{:fact-type :client-representative :name "Alice" :client "Acme"}
 {:fact-type :support-request :client "Acme" :level :high}]


(this is all pluggable and could be implemented with something else on Clara’s side just as well too)


try to Fressian write/read that and see that it returns you the facts in the same structure


leaving Clara session/rules out of the test


that may give some insight

Konstantinos Tzanidakis15:01:13

@mikerod simply using the fressian write/read does work correctly in REPL

Konstantinos Tzanidakis15:01:46

yes, I think I will use the nippy as it seems to be something in the fressian side that I cannot track at the moment


> simply using the fressian write/read does work correctly in REPL You’ve analyzed waht is returned? is it still a clj vector with 2 clj maps etc?

Konstantinos Tzanidakis15:01:24

thank you very much for the prompt responses and help!

Konstantinos Tzanidakis15:01:50

@mikerod yes it returns exactly the same facts that are serialized; i.e. a vector of two clj maps


I’d have to experiment some to look close, can’t right now


where things don’t work out-of-the-box with conversions and types


Under “Default handlers”


However, didn’t mention anything with vectors/maps there


I immediately see that types at least aren’t preserved

(let [r (serde-obj [{:fact-type :client-representative :name "Alice" :client "Acme"}
                    {:fact-type :support-request :client "Acme" :level :high}])]

  (type r))
;;= java.util.Arrays$ArrayList


this isn’t immediately meaning Clara wouldn’t accept it - but it is already on shaky ground

Konstantinos Tzanidakis15:01:25

@mikerod thank you very much on this!


you could attempt to just add the handlers from clara.rules.durability.fressian


only issue there being you may not want to deal with the “identity preserving” parts potentially


of how you’d do that


has 2 dynamically scoped bindings you add for it’s internal caching


you can see it via pform/thread-local-binding


d/node-id->node-cache doesn’t matter for working memory - so you can leave that off


d/clj-struct-holder is required I’m fairly sure - it is used to preserve object identity during (de)serialization