This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-03-10
Channels
- # admin-announcements (2)
- # alda (16)
- # beginners (22)
- # boot (55)
- # cider (11)
- # cljs-dev (5)
- # cljsrn (7)
- # clojure (115)
- # clojure-art (1)
- # clojure-finland (3)
- # clojure-greece (18)
- # clojure-madison (2)
- # clojure-norway (2)
- # clojure-poland (50)
- # clojure-russia (122)
- # clojure-uk (9)
- # clojurescript (28)
- # core-async (7)
- # cursive (4)
- # data-science (4)
- # datomic (65)
- # devcards (3)
- # dirac (91)
- # editors (7)
- # emacs (9)
- # hoplon (1)
- # jobs (5)
- # jobs-discuss (10)
- # juxt (2)
- # keechma (7)
- # ldnclj (31)
- # leiningen (3)
- # off-topic (22)
- # om (129)
- # om-next (1)
- # onyx (6)
- # proton (15)
- # protorepl (2)
- # re-frame (49)
- # reagent (13)
- # remote-jobs (1)
- # ring-swagger (2)
- # yada (19)
assuming i have a database value which has a certain entity id present within it, and i then use (as-of db some-point-in-time-before-that-entity-id exists) , and I do (d/with ...) on that as-of db, with a tx like [:db/add entity-id some-attr some-value], why does that succeed still? surely it should blow up with an invalid entity-id , since it technically doesn't exist at that point in time?
man, it's so tempting to start using Datomic for a project I'm working on right now, because I saw the video about resource ownership/authorization, and it just felt so elegant to solve it that way (transform the database object into a new database that only contains objects that are "owner" by the current user)
@greywolve: weird indeed, are you positive about the entity id not existing before that ?
@lmergen: what complexity are you worried about ?
ah yes
biggest impediment is the limit on the number of processes IMO
I do deploy it on AWS, and it's OK operations-wise
and we're only a 2-devs startup
but you do have to plan and accomodate for it
but it seriously goes against my gut feeling right now... it's a beautiful thing, but it would be awesome if there would be some datomic-as-a-service with AWS or so
val_waeselynck: yeah i set the entity id to about 2 years back, so pretty sure š when i restore the entire db back to a point in time just before that entity existed, then d/with correctly blows up when i try that tx
@lmergen: amen to that
apps + 2 transactors + ddb
itās not that big of an issue. there are details, but itās not rocket surgery
@lmergen: official stance on this is that your write volume must be consistent with the upper limit on your database size (10 billion datoms)
@lmergen: having said that, it can be interesting to use a hybrid architecture with some NoSQL database for collecting most of your data and Datomic to post-process it
so a lot
and I'm trying to find a solution to query our data warehouse, mix & match it with our relational database, and not having to worry about the slow query time of our data warehouse
worrying about running two transactors is the least of your problems, then
someone suggested me into this direction, to "pre-warm" the pipelines with our data warehouse data...
I'm not an expert in this kind of deployment š maybe you shoud just contact the guys at Cognitect
yeah I'm already in contact with one of their sales people, but I think what I'm after is actually some consultancy on this matter.. hmm
I think Datomic aims to fulfill the role of a traditional relational database ā with better scaling characteristics than most SQL's ā more than the role of a "big data" solution.
@lmergen yeah, Datomic is not a solution to this problem, it plays in the oltp space. Likely in your situation your sql database will need to be able to utilize pre-calculated historical aggregations done on warehouse data, in the appropriate shapes, so that you can use sql across both datasets. In the warehouse/sql world this is an ETL problem, one needs to orchestrate pushes from the warehouse into some version (slave/standby/replica) of the oltp system. This usually is very messy and complicated. What's appealing about Datomic from an architectural perspective is being able to look at this problem from a pull perspective, with layers of declaratively-defined caches
as in, I don't think it's possible to do this in realtime, we simply need to schedule periodical jobs
people are doing all sorts of fancy stuff to make results appear faster, e.g. https://www.mapr.com/developercentral/lambda-architecture, but all of this brings huge amounts of complexity
@greywolve: branching off of an as-of
point with with
isnāt supported.
with
works against the db without the filter. as-of
dbs will filter out prospective transactions from with
when queried against. At present, prospective branching from points in the past isnāt supported.
Hi all, Iāve been examining the schema of a Datomic database, using this kind of query:
clj
(d/q '[:find ?attr
:where
[_ :db.install/attribute ?e]
[?e :db/ident ?attr]]
db)
But I get retracted attributes as well.
Does anyone know how to filter out retracted attributes?Iām getting back attributes like
:customer-account-state/customer-username_retracted5632a067-4048-4186-8610-8e5286596ebe
Iāve inherited this code, with no handover, so thereās a possibility thatās this is some non standard craziness and I just havenāt found the place where it comes from.
@arthur.boyer: You canāt retract attributes. That looks like renaming (with a retracted+sha convention) only, and renaming is a typical way to deprecate an attribute. In general, retracted values will only appear in history databases, and you have to bind the fifth position of the datom (the ?added) portion to see if it was an add or retract op.
Ok that makes sense. Iāve run queries binding the fifth position and they came back true. Iāll do some more digging and see if thereās a way I can clean them up. Thanks.
if you donāt want those values returned by a schema check, I guess a regex against string rep of the ident would be fine (assuming those are only cases of āretractedā)
Thatās what Iāve been doing, but it feels like a nasty hack, and not what I think idiomatic datomic should look like.
So, what do you do if you do want to retract an attribute? Do you retract all the datoms that use it?
hey all, I have a question on the pull syntax bnf grammar (copied from the pull api site):
pattern = [attr-spec+]
attr-spec = attr-name | wildcard | map-spec | attr-expr
attr-name = an edn keyword that names an attr
wildcard = "*" or '*'
map-spec = { ((attr-name | limit-expr) (pattern | recursion-limit))+ }
attr-expr = limit-expr | default-expr
limit-expr = [("limit" | 'limit') attr-name (positive-number | nil)]
default-expr = [("default" | 'default') attr-name any-value]
recursion-limit = positive-number | '...'
I just tried putting a default-expr in as the first item in a map-spec and it worked. so iām thinking the doc on the site is incomplete? or am I missing something?a more common pattern is to migrate data to a new Datomic database (by i.e. replaying the log with filter/transform) with a finalized schema. That works if you want to drop a lot of the initial modeling learning process, say. But I think the possibility of retracting attributes could introduce more operational complexity than keeping them and removing them from queries.
@ethangracer: do you have a repro or just a code example of the specific expression that works and violates the grammar? I can take the repro case to the team and see whether the grammar or the behavior is correct.
@bkamphaus: Thanks for that, I think the practical upshot now is that I can safely ignore things named retracted and we can consider a more radical solution later.
@bkamphaus: sure, for this sample the idea is that I want to load todos from the server. but maybe I haven't sent any todos to the server yet. In which case I want to return an empty vector (to distinguish between loaded / not loaded from the server). So the bnf compatible query is
(d/pull db [{:todos [:id :title :is-completed]}] __id-for-todo-list__)
which returns nil
if that todo-list has no todos. If instead, I write:
(d/pull db [{(default :todos []) [:id :title :is-completed]}] __id-for-todo-list__)
then the pull returns []
, even though this is not documented by the bnf@ethangracer: thanks, Iāll investigate and get back to you.
@bkamphaus: sounds good, thanks