This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-06-14
Channels
- # aws (6)
- # babashka (31)
- # beginners (69)
- # biff (9)
- # boot (9)
- # bristol-clojurians (1)
- # calva (20)
- # chlorine-clover (2)
- # cider (8)
- # cljsrn (24)
- # clojure (25)
- # clojure-norway (4)
- # clojure-spec (29)
- # clojure-uk (7)
- # conjure (23)
- # datahike (5)
- # datomic (39)
- # emacs (4)
- # fulcro (4)
- # graalvm (11)
- # honeysql (1)
- # lambdaisland (1)
- # leiningen (8)
- # liberator (1)
- # libpython-clj (3)
- # malli (6)
- # mxnet (1)
- # off-topic (94)
- # pedestal (13)
- # re-frame (4)
- # releases (2)
- # shadow-cljs (8)
- # spacemacs (22)
- # sql (9)
- # vim (1)
Hello. Just about beginning to play with datomic. I have a specific need and I think datomic fits the bill, I was wondering if anyone could give me pointers. I need to build a dossier system of sorts where there are notes and other details maintained for several candidates. As this detail changes through time, it would help for us to have historic information visible about each candidate
In Mongo, this would be a series of records, timestamped, but all really containing mostly the same information
So given that I want to maintain a history and given that the schema can be flexible, Datomic sounds right, am I correct in assuming this?
so let us say, I make a series of assertions on :notes , then later on, I’d like to look at :notes, not just as what the latest is, but all the :notes accrued over time
This should be (trivially?) possible right?
Thanks
@U8VE0UBBR as a non-official source on Datomic, I advise against using Datomic's historical features for giving users access to revisions: https://vvvvalvalval.github.io/posts/2017-07-08-Datomic-this-is-not-the-history-youre-looking-for.html Datomic is not a bitemporal database. A priori, I recommend making one entity per note revision, as you would do with a regular database.
So you suggest an additional attribute that records the version as well, as opposed to relying just on timestamps
I see what you are saying here, history changes are fine as long as the shape remains the same
the second the shape changes
that becomes more complex
However, even with what you are saying, its easier to use datomic here isn’t it, given the use case?
Datomic can be easier to use, for general reasons not related to modeling revisions, such as flexible schema, expressive reads and writes, ease of data sync, etc.
Alright. In this case, I have a very specific need of having to look at history
You may be fine just storing one entity per revision or per change, if your queries aren't highly sophisticated.
Otherwise might want to look at bitemporal dbs like Crux, but there are many other aspects to consider than historical query features.
As of now, I just want a history of notes per person let us say
so let us say some attribute was added only at tx 200, what is the cost to me as a developer if I query for that attribute in an earlier transaction?
The main question for assessing Datomic against such use cases is how complicated your historical queries are
Pretty much no branching, straight ahead, give me everything you’ve got on person x, at most limited by a specific duration
Assuming incredibly low performance needs, never more than a handful of users at any point
From what I am reading of the schema change, I could potentially backfill necessary data, which might not even be necessary for certain types of attributes
Well you can’t backfill such that it looks as if it was transacted in the past. That is the limitation of relying on datomic history for revisions
Datomic history is more like (immutable, not branching) git history than like time-series records
How does using a predicate directly in the query (https://docs.datomic.com/cloud/query/query-data-reference.html#predicates) compare to querying the data then performing the predicate? I assume the predicate runs somehow before the join?
the answer is in the docs:
> The predicates =
, !=
, <=
, <
, >
, and >=
are special, in that they take direct advantage of Datomic's AVET index. This makes them much more efficient than equivalent formulations using ordinary predicates. For example, the "artists whose name starts with 'Q'" query shown above is much more efficient than an equivalent version using starts-with?
errr. wait < works on strings to compare the first two letters?
;; fast -- uses AVET index
[(<= "Q" ?name)]
[(< ?name "R")]
;; slower -- must consider every value of ?name
[(clojure.string/starts-with? ?name "Q")]
That seems really oddwhat does it mean "they take advantage of datomics AVET index" does that mean the comparison is done using information in the index as well? like when we say index im thinking "alice" "bob" "zack" so using the index in the context of (< ?name "d") would mean that zack is returned and the operation do this never actual had to look at the string zack because i was stored in location that was marked like "d-z" or something.
If the query planner can see the attribute you are using, know it has an avet index, and see the comparisons and their values it can figure out a subset of the values in the index to seek instead of seeking the whole thing
for 'on-prem' i understand you have to add an avet index for an attribute by doing a transaction. i did a search through my cloud db and i don't see a db/index attribute. Do i need to add avet indexs for the queries that use index e.g d/index-range to work? and if so, how?
awesome, thanks!