2026-07-30 datahike | Clojure Slack Archive

datahike 2026-07-30

markaddleman 2026-07-30T01:06:11.436669Z

@whilo Just wanted to say thanks for the quick turn arounds on my planner issues thanks3

whilo 2026-07-30T01:07:12.676749Z

You are welcome! I am happy to help solve issues users hit in Datahike to make sure to get it ready for 1.0.

whilo 2026-07-30T01:07:46.083259Z

I am also curious what you are doing with Datalog @markaddleman in general.

markaddleman 2026-07-30T01:11:41.466209Z

Happy to! The short version: Datahike is the basis for a knowledge-graph populated and queried by LLMs. The LLM extracts subject–predicate–object facts out of unstructured prose into Datahike, then reasons over them with Datalog. That’s why I keep landing on the planner around recursive rules and get-else totality specifically — I lean hard on both.

whilo 2026-07-30T01:17:06.758379Z

That makes sense. I have implemented a Roam-like wiki as part of #C09622F337D. Getting this ready to open source rn.

whilo 2026-07-30T01:17:31.658119Z

Would be happy to share work on building a good default wiki schema and functionality.

markaddleman 2026-07-30T01:17:40.673979Z

Oh very cool!

whilo 2026-07-30T02:56:03.347339Z

Is the new query planner+engine faster for you? I assume you used the old engine before.

markaddleman 2026-07-30T14:03:28.194769Z

Yep, I’m using the old engine right now. I’m hoping to switch to the planner once the core use case is unblocked

markaddleman 2026-07-30T21:51:33.685359Z

Thanks for the latest fix! I think the planner is completely unblocked for me now. I ran The Turn of the Screw through our extraction pipeline into a datahike graph (~12k datoms) and timed the queries:

┌────────────────────────────────────────────┬────────────────┬──────────────────┐
 │                   query                    │ ratio (OFF/ON) │     verdict      │
 ├────────────────────────────────────────────┼────────────────┼──────────────────┤
 │ coherence self-join                        │          75.7× │ planner wins big │
 │ not-join / negation                        │          2.49× │ wins             │
 │ join-heavy 5-way                           │          2.09× │ wins             │
 │ get-else provenance (statement-rows, real) │          1.47× │ wins             │
 │ aggregation (sub-2ms)                      │          0.64× │ loses (trivial)  │
 │ bounded recursive reachability (#113)      │          0.38× │ loses ~2.6×      │
 └────────────────────────────────────────────┴────────────────┴──────────────────┘

What each one is, product-side: • coherence self-join — consistency check: do the extracted claims contradict each other? (e.g. flag a character whose stated beliefs conflict) • not-join / negation — gap detection: what’s required-but-absent, or explicitly denied • join-heavy 5-way — faceted retrieval: statements matching several conditions at once (holder + attitude + polarity + topic…) • get-else provenance (statement-rows) — the workhorse: the review table listing every extracted statement with its source citation • aggregation — rollups (belief counts per character, etc.) • bounded recursive reachability — nested theory-of-mind: tracing chains of belief (“what does A believe that B believes about C”) The one loss is recursive-rule eval (the belief-chain traversal, ~2.6× slower) — wierdly, the same rule seeded in reverse wins 8–118×.

whilo 2026-08-01T09:53:03.298759Z

I can take a closer look at this, maybe https://github.com/replikativ/datahike/blob/main/doc/graph-algorithms.md are helpful. It would also be interesting to point Claude the recursive parts of the engine and let it experiment and compare to other recursive engines, maybe Souffle.

👍 1

Clojurians Log v2

datahike 2026-07-30