Fork me on GitHub
#clara
<
2020-01-07
>
Konstantinos Tzanidakis14:01:01

Hi all, I want to serialize/deserialize the working-memory separately so I started off by using a simple implementation for IWorkingMemorySerializer that serializes to an atom (as per Clara's docs) and deserializing from the atom by derefing it. That works perfectly fine, in that I am able to run queries and get the same results from both the original and deserialized session. However, if I add the extra step to write to fressian and read fressian to the ser/deserializer of the IWorkingMemorySerializer , all queries in the deserialized version of the session return with nil: eg. ({:?crn nil}) while the same query on the original is returning ({:?crn {:fact-type :client-representative-notification, :name "Alice", :client "Acme", :level :high}}) . I should note that I am fairly new to clara rules. Has anyone come across something like this before?

ethanc15:01:51

@konstantinos562, Do you have a small snippet, it would help me reproduce it locally?

Konstantinos Tzanidakis15:01:24

Hi @ethanc do you need the rules as well?

Konstantinos Tzanidakis15:01:08

(defrecord WorkingMemoryAtomSerializer [input output]
  d/IWorkingMemorySerializer
  (serialize-facts [_ facts]
    (reset! state (write-fressian facts)))
  (deserialize-facts [this]
    (-> @state io/input-stream read-fressian)))

Konstantinos Tzanidakis15:01:12

with (def state (atom nil)) in the namespace

Konstantinos Tzanidakis15:01:36

and the read and write fressian as follows

Konstantinos Tzanidakis15:01:40

(defn write-fressian [obj]
  (let [baos (ByteArrayOutputStream.)
        wr   (frz/create-writer baos)]
    (frz/write-object wr obj)
    (.toByteArray baos)))

(defn read-fressian [b]
  (-> b frz/create-reader frz/read-object))

Konstantinos Tzanidakis15:01:47

sorry for the long posting all

Konstantinos Tzanidakis15:01:50

NB: I have replaced fressian with nippy/freeze and nippy/thaw and it works perfectly fine now. I am not sure what the issue was (most probably I am doing something wrong there)

mikerod15:01:11

@konstantinos562 not sure what frz ns is referring to

mikerod15:01:23

You may just not have setup good read/write handlers for your facts

mikerod15:01:35

Fressian’s defaults tend to be insufficient

Konstantinos Tzanidakis15:01:36

sorry it's the requirement: [clojure.data.fressian :as frz]

mikerod15:01:55

or they will cause many clj data structures to become Java structures - or not the same as before

mikerod15:01:33

you can see Clara defines a more robust set of fressian handlers for clj to clj serialization https://github.com/cerner/clara-rules/blob/master/src/main/clojure/clara/rules/durability/fressian.clj#L231

mikerod15:01:47

however, there really is no compelling reason to not just use nippy if you have it set up

mikerod15:01:03

I wouldn’t have used Fressian to impl Clara’s session serialization if I were to do it again today. it is difficult to work with in what it provides by default and I don’t think it is any better than something like nippy or perhaps a few others. It is also not documented super well and at the “primitive” level makes some unavoidable Java collections that are awkward to convert from (and also wasting perf time)

Konstantinos Tzanidakis15:01:11

for this example, I have not set up any handlers but the facts are fairly simple

[{:fact-type :client-representative :name "Alice" :client "Acme"}
 {:fact-type :support-request :client "Acme" :level :high}]

mikerod15:01:19

(this is all pluggable and could be implemented with something else on Clara’s side just as well too)

mikerod15:01:47

try to Fressian write/read that and see that it returns you the facts in the same structure

mikerod15:01:54

leaving Clara session/rules out of the test

mikerod15:01:11

that may give some insight

Konstantinos Tzanidakis15:01:13

@mikerod simply using the fressian write/read does work correctly in REPL

Konstantinos Tzanidakis15:01:46

yes, I think I will use the nippy as it seems to be something in the fressian side that I cannot track at the moment

mikerod15:01:17

> simply using the fressian write/read does work correctly in REPL You’ve analyzed waht is returned? is it still a clj vector with 2 clj maps etc?

Konstantinos Tzanidakis15:01:24

thank you very much for the prompt responses and help!

Konstantinos Tzanidakis15:01:50

@mikerod yes it returns exactly the same facts that are serialized; i.e. a vector of two clj maps

mikerod15:01:55

I’d have to experiment some to look close, can’t right now

mikerod15:01:25

where things don’t work out-of-the-box with conversions and types

mikerod15:01:43

Under “Default handlers”

mikerod15:01:59

However, didn’t mention anything with vectors/maps there

mikerod15:01:52

I immediately see that types at least aren’t preserved

(let [r (serde-obj [{:fact-type :client-representative :name "Alice" :client "Acme"}
                    {:fact-type :support-request :client "Acme" :level :high}])]

  (type r))
;;= java.util.Arrays$ArrayList

mikerod15:01:09

this isn’t immediately meaning Clara wouldn’t accept it - but it is already on shaky ground

Konstantinos Tzanidakis15:01:25

@mikerod thank you very much on this!

mikerod15:01:53

you could attempt to just add the handlers from clara.rules.durability.fressian

mikerod15:01:15

only issue there being you may not want to deal with the “identity preserving” parts potentially

mikerod15:01:57

of how you’d do that

mikerod15:01:26

has 2 dynamically scoped bindings you add for it’s internal caching

mikerod15:01:40

you can see it via pform/thread-local-binding

mikerod15:01:53

d/node-id->node-cache doesn’t matter for working memory - so you can leave that off

mikerod15:01:18

d/clj-struct-holder is required I’m fairly sure - it is used to preserve object identity during (de)serialization