This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-06-13
Channels
- # admin-announcements (1)
- # beginners (10)
- # boot (15)
- # cider (9)
- # clara (195)
- # cljsrn (24)
- # clojars (20)
- # clojure (46)
- # clojure-android (1)
- # clojure-germany (15)
- # clojure-greece (16)
- # clojure-nl (1)
- # clojure-russia (13)
- # clojure-spec (28)
- # clojure-uk (44)
- # clojurescript (104)
- # clojurex (1)
- # component (7)
- # css (2)
- # cursive (27)
- # datomic (92)
- # dirac (12)
- # emacs (5)
- # lambdaisland (3)
- # lein-figwheel (36)
- # mount (87)
- # off-topic (8)
- # om (102)
- # om-next (3)
- # onyx (30)
- # pedestal (3)
- # re-frame (26)
- # reagent (20)
- # robots (4)
- # specter (18)
- # spirituality-ethics (1)
- # untangled (127)
- # yada (11)
devn: thanks for the heads up. Yeah, my original demo is likely to not be all that close to what I end up going with. However, it is good to know where the hiccups are right now with the default impl's of print-dup
and I forgot about this existing issue there.
@mikerod: it actually prompted me to do a general jira survey for "dup" issues. I was unable to figure out how to get JIRA to search for #= despite several attempts at various quoting and escaping schemes
actually, i wound up doing a walk over most of JIRA yesterday. I don't know that it was "enjoyable", but focusing on the critical and high priority issues and reading them through was worth the time I think
I try to keep up with it somewhat, but it'd be good to get a refresher over the bigger existing issues
it moves slow enough that if you give it a good look post-release, you'll know what's up for awhile
I've done a reasonable amount with print-dup
before, but it is a lesser documented area and as you've shown here, there are caveats
heh, speaking of print-dup: I wrote a thing awhile back that took all of the sexps from the #C03S1KBA2 IRC channel going back to 2009
I used print-dup to serialize the entire collection of results out to a file at some point, and i remember finding some tricky bits, but i don't remember details anymore
despite the "first rule of hash equals club is don't talk about hash equals", i remember it being quite solid back in 1.2, 1.3
there were at least 30k random examples serialized to a file, and read successfully, including Swing and all sorts of other nonsense
though it's one of those things that Rich and Co. have basically said: use at your own risk, i have a hard time imagining it going away
Anyway, as long as I mentioned that project: http://getclojure.org/
It's nothing to write home about, and since it was all sandboxed, there are plenty of things which didn't pass the sniff test and were subsequently not run in the sandbox, but there's some interesting stuff in there, usually further into a search. A search for comp
has results for the first few pages that are like (comp comp comp comp)
for example, which is not very interesting.
yeah, print-dup
is weird. It seems to be, in some ways, a recommended approach to serialization of Clojure structures, but it is still not really fully "official"
i had dreams of building a sort of hoogle for clojure. codeq + clojure.tools.analyze + query over sexps, something like that
it is primitive though. to use it successfully, you have to implement your own back references etc.
i'd like to understand clara internals better, but i think i need to start smaller and implement a dumb version of rete before I'll be of much use
devn: yeah, I actually find the codebase to be approachable, but it is good to have some background
clara.rules.engine is almost something you could look at stand-alone to reason about the way the Rete network propagates data around
builds the rete graph etc, but I think you can sort of gloss over that and just look at clara.rules.engine to get a good feel for how data moves around
is it crazy to think about building some instrumented visualization of the graph construction?
writing a toy version of your own could be a good learning experience though I'd imagine
and also, it may not visualize the graph like you are saying. Visualization has different sort of levels you may be interested in.
some of them useful at a domain level so people using our stuff can understand why a decision was rendered
I found it to have a really good overview of Rete. It is certainly dated and lacking many of the optimizations and extensions to Rete present in Clara, but it is a good base.
there are a few key chapters to it. it is long, but some of it isn't really all that relevant unless you are curious about the test system they were writing back in the 90s or whenever it was. 😛
I won't speak completely for Ryan, but I think he had some inspiration from http://herzberg.ca.sandia.gov/
Jess has good documentation here though. Including on some Rete extensions like accumulators.
i keep tinkering with the idea (yet another project that is on hold due to the amount of time i have) of a datomic-driven rules engine
reified transactions a la Tim Ewald (sagas) here: http://www.datomic.com/videos.html
I haven't done much with Datomic other than listen to a bunch about it and read up on it. I've liked a lot of the ideas etc there and hope to work with it at some point
i know that there have been at least two rules engines built on datomic which are not open source (sad face)
Oh, also on Rete. I follow Drools stuff a fair amount too. Drools has pushed the bar with optimizations and has a lot of good extensions and documentation as well.
the datomic thing is most interesting to me i think because most rules engines (as far as i understand it) do not address temporal concerns
this is partly why durability is interesting to me: "show me what would have happened as of a point in time", or "show me what it would look like if the rule base contained this rule, without committing it"
"Complex Event Processing, or CEP, is primarily an event processing concept that deals with the task of processing multiple events with the goal of identifying the meaningful events within the event cloud. CEP employs techniques such as detection of complex patterns of many events, event correlation and abstraction, event hierarchies, and relationships between events such as causality, membership, and timing, and event-driven processes."
I know Ryan has said a little on the topic of dealing with temporal "events" as well, but I'm not sure how much its been thought out yet
Clara durability of session state (working memory) is going to be important to the work I'm doing right now, so that should keep progressing. We haven't needed anything more complex as of yet.
we deal with a lot of sparse data, and so a fair amount of work is done to validate concepts, those concepts are then turned into facts. i'd sort of like to extend rules to the problem of validation and creation of concepts
but there's a sort of ETL-like process that varies a lot. i imagine a sort of rule-based ETL
yeah, certainly, just haven't had the stomach for it, because i think there's tooling i'd want to build to manage this process and have decent visibility into it
well if you come up with good tooling ideas that'd help on the Clara end, I'm sure we'd all be happy to hear about them
our current push this release round is probably going to center on performance and then this durability thing
As I said, I don't feel like I have a good enough handle on internals to say: "here's a great idea!" because I worry that the performance overhead will be insane, or that i'll be unable to capture some key piece of information necessary to the idea without reworking some significant piece of Clara
but thinking more on what can be done for visualization, "explainability", etc should stay imoprtant
https://github.com/rbrush/clara-rules/blob/master/src/main/clojure/clara/rules/listener.cljc
it's extremely important for us to say: "We recommend these actions based on the following information related to the patient, provider, etc."
mostly just keeping tracing sort of information in data structures that can be hooked on by whatever tooling
with a sort of decision tree that people with a clinical background can use to validate those recommendations
so far, what we've mostly done - which has been sufficient - was to just show the support "chain"
we either manually write rules to do so, or we built some automatic helpers around insert's
(defrule my-rule [?s <- SomeFact] => (insert! (map->AnotherFact {:something blah :support [?s]})))
I committed some code as part of an early spike on clara which added a :contributing-factors {}
k/v in the RHS
yes, we have implemented some ideas around this. not in Clara directly, just in our own projects
we've defined protocols or something as well that can sort of walk the "contributing facts" chain
you might need to merge multiple :contributing-factors across the results of queries in order to get the real picture
we've sort of just "flattened" the chain up to certain points where there is a meaningful user-defined name
or at least, the visualizations ive come up with are like: oh god, no nurse is ever going to look at this
so it's sort of hierarchical, but flattened out just to where we have some sort of meaningful name to show end-users
the bindings which wind up being worth a damn, again due to sparse data, can be troublesome also
you could mark fact types in some way or another htat are "internal details" vs meaningful though
(defrule [?f <- ImplDetailThing] [?f2 <- Meaningful] => (insert! (map->Fact {:val something :support [?f ?f2]})))
I'm now generating records (using eval, don't hate me) that pull from a big old map of domain concepts => data definitions
but then when walking the chain later, dropping out stuff in the chain/tree that isn't useful
yeah, could think of a protocol like ISupported (depending on how you like to name things)
it may make sense if you are syaing you are trying to make data types dynamically driven off of config
just be aware the the rules engine has some liberties it can take as far as when it fires rules
rules may fire in different orders due to non-determinism (we'd like to minimize), optimizations, etc
just never get too reliant on rule order unless you have logical rule order that with eventually be consistent due to truth maintanence
(defrule rule-1 [MyFact] => (insert! (->AnotherFact )))
(defrule rule-2 [:not [AnotherFact]] => (insert! (->AlternativeFact)))` etc
yes, that's really what we deal with, but i do worry a bit that if that's not the case in the future, we're going to be sad pandas
appreciate the conversation, and am pleased to hear that i'm not the only one who's doing this whole :support thing. people on our team just started using it
@devn: FWIW, I agree with Mike that the engine and (I’ll add) the memory are probably the best places to start. When I was first getting familiar with the internals of Clara I found it useful, and still do if I’m unsure of what some rule will look like in the network, to just create a session from that rule and look at the rulebase generated. That is, something like (-> (mk-session [rule-in-question] :cache false) .rulebase :alpha-roots) and just poke around the rules network generated.
I’ve found that if I’m not dealing with the compiler, it is very possible to just deal with the rules network and forget where it came from. The memory is even more isolated in some ways in that it mostly is just a composite data structure, albeit one very much tailored to the needs of the engine.