Fork me on GitHub

@ryanbrush: question for you -- we have rules which are generated dynamically, and we were noticing that our runs with caching turned on were missing the cache. Turns out it was due to ordering.


What do you think about using sets to check for cache hits?


Honestly, this might be an instance of "not clara's responsibility" -- I think we're going to wind up managing our own cache using an LRU or TTL cache


just curious what you thought -- took a bit of tinkering to figure it out


and I guess I don't see why it couldn't be a set


or coerced into one


the generated source rules, i mean


@devn I'm open to the idea of using a set to improve cache hits. Come to think of it, if the code that generates rules produces a set (rather than a list) that is passed to mk-session, then that might actually trigger a cache hit by itself. We should be able to think of mk-session as a memoized function. If passing a set of rules into mk-session doesn't hit the cache then we could probably change that.


yes, making it a set makes it a cache hit


@devn However, managing your own cache might be a better idea, anyway, since putting all rules into a set (or doing a deep comparison in a sequence) could get expensive for a lot of rules. You could probably do something more efficient in user code if you know that things changed or didn't.


i guess my thinking was that it was a little subtle why at first. i opened up yourkit to figure out what was slow, found myself in mk-session, beta and alpha creation, yadda yadda, noticed a lot of DynamicClassLoader business, saw a bunch of eval's which i believe are the generate LHS and RHS fns


finally noticed we were just getting cache misses, diff'd the generated rules, noticed the order issue, yadda yadda


i couldn't really think of a good reason why clara couldn't just make the decision to take whatever collection is passed to mk-session and make it a set, or if someone really wanted duplicates, (sort-by hash [...]) or something


Yeah, that's fair. Just want to understand any performance implications there might be but I think we can do that.


for really large numbers of rules, for instance?


@ryanbrush: One More Thing™ -- Do you have any thoughts on analyzing/visualizing the time-complexity of build-network provided a set of rules?


General question is: How much does the design of the rules being added to a session effect the performance of build-network? For instance, a really shallow logic graph vs one with a lot of depth.


Wondering aloud if there's any potentially actionable information about the complexity of the network that would be useful to someone building an expert system.


"your logic graph is really shallow, which means build-network is going to be slower"


@devn good question, but probably not one that I can answer on my mobile right now. ;) I'll post something more thoughtful when I get the chance.


@ryanbrush: ha, no worries! @enn and I were working together today and I popped open Slack and realized I asked you a question about Clara at 10PM on Christmas Eve. You owe me nothing, and I'm very grateful for all of your help.