This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-03-17
Channels
- # announcements (7)
- # babashka (56)
- # beginners (114)
- # bristol-clojurians (4)
- # calva (22)
- # cider (7)
- # clara (1)
- # clj-kondo (17)
- # cljs-dev (1)
- # clojure (93)
- # clojure-europe (8)
- # clojure-italy (5)
- # clojure-nl (2)
- # clojure-uk (79)
- # clojuredesign-podcast (18)
- # clojurescript (108)
- # code-reviews (6)
- # cursive (3)
- # data-science (16)
- # datomic (151)
- # duct (7)
- # emacs (10)
- # events (1)
- # fulcro (76)
- # luminus (8)
- # off-topic (3)
- # other-lisps (2)
- # pathom (8)
- # re-frame (5)
- # reitit (8)
- # schema (9)
- # shadow-cljs (37)
- # specter (3)
- # sql (17)
- # tree-sitter (2)
- # yada (9)
It's a shame that when they added support for strings as temp-ids, they didn't make strings work as temp-ids for the entity
function, so it's easy to have tx-fns that use entity explode in your face. I'm not sure I think they would consider this a bug, but it's the kind of sharp edge that feels all-too-common in datomic.
you mean like (d/entity some-db "a-tempid-string")
?
we usually just transact, and then do (d/entity after-db (get tempids some-tempid))
@matthavener Yes, that's what I mean. But I have some transaction functions that internally use entity
, which can't handle those string tempids, so the transaction will abort. I don't see how transacting is an option, since the problem is that the tx-fns won't run.
That's why as far as I can tell the solution is just not to us those string ids if you have a situation like this.
@pmooser that’s really a type error: those tx fns cannot accept a tempid as an argument
whatever work they are doing requires knowledge they cannot have--silently doing nothing seems wrong
@favila I mean it depends on what you think it means for datomic to support strings as tempids - I would say the error is incompletely supporting it. It's not like the docs clearly state you can use them only in certain places as strings.
I mean it's hard to say what is intended in this case, since (as often happens with datomic) it's hard to know which of these edge cases are intentional and which aren't, and the documentation does little if anything to clarify.
In any case, in terms of concrete things, in my situation it just means we either have to walk some data structures and replace strings with temp-ids, or just avoid the strings where possible. In this case it's a little unfortunate, since strings as tempids are a convenient affordance for things like client-generated data.
edge cases are often not called out or checked and you only know they are edges by experience and intuition
I think it's also just that despite clojure being a great language, they don't have the impression that this is a significant sort of quality problem. There are some truly shocking bugs in things like core.async - and they're known, and they'll probably never be fixed.
[{:db/id tempid :upserting-attr bar} [:txfn tempid]]
is wrong whether tempid is a string or a record.
@favila Not in general, no. I mean, I don't walk these structures I'm transacting unless I'm doing some modification of the data anyway.
I'm not sure what you are implying - that I can't use a tempid of any kind as an argument to a tx-fn?
(I probably just don't understand the particularity you're trying to illustrate with upsert)
I mean if :txfn needs to read tempid, these two transactions are “equivalent” if [:upserting-attr :bar] exists:
even though the state of the db is the same and arguably the tx should do the same thing
that’s why I argue, if a txfn is expected to read an entity provided as input, it should hard-fail if that input is not resolveable (i.e. a tempid)
they can read the environment (&env, the db), but they cannot read “in progress” transactions
what tempid resolves to is not known until the very end, and then all fully-expanded adds and retracts are applied simultaneously, set-wise
my point is that there’s a syntatic transformation occurring, and if that transformation needs to read the environment to perform the transformation, it can only do so with what is available to it syntatically
@favila It's going to take me a couple minutes to respond, because I want to go experiment with what you are saying a little bit.
and tempids are not resolveable syntatically. some other opaque tx fn could emit an assertion which changes its resolution
only when the full set of primitive asserts/retracts is known and the syntax is fully expanded can tempids be resolved
Ok, so I sort of understand what you mean, but I don't think it's even quite correct.
You're right that the tempid passed to your tx-fn won't be magically converted to the db/id of the upsert,
but if the tx-fn uses the tempid to assert some things, datomic will correctly eventually resolve that tid to the upserted db/id.
So I don't completely understand the point you were trying to make, or what it has to do with my original point, which is that half-implemented features (ie, temp-ids as strings) are unfortunate sources of complexity and sharp edges.
Ok, I'm probably just being really dense, but so? What does it have to do with the changelog that says you can use strings as tempids, when they are not substitutable?
Like it's hard for me to understand the idea of that the implementation of entity
isn't wrong in the sense that either it should work for all representations of temp-ids, or for none of them. I'm not quite sure how you think some other solution serves the users.
Well, what you just said isn't true, as it happens, depending on what you mean by temp-id, which is my point.
I concede absolutely, that a string tempid and a record tempid should fail in the same way; however, whichever way they fail, you still need to do the same checks
zoom out and re-examine what you're trying to achieve. calling d/entity on a tempid is a detail, what is the overarching goal?
My code works, I worked around the fact that entity freaks out if you call it using a string on a tempid
Maybe I mistakenly gave you guys the impression that I'm trying to fix a bug - the bug is fixed. It's just yet another case of having to understand datomic from its behavior than from any kind of real specification.
well I can't help with wishes 🙂 what semantics would it even have to call d/entity on a tempid? like what would it return?
Exactly what it does if you call it on a tempid. In fact, entity confuses me in that it will return something for any integer value you pass it, even if the entity doesn't exist. I can't explain that, just as I can't explain why it only works for certain representations of tempids but not others.
What I would have it do, if I could control it, is: whatever entity does, do it for all representations of tempids, and not behave differently depending on representation of tempid.
datomic stores datoms, not entities. d/entity provides a projection of datoms as a map. So, what does it mean for an entity to exist?
If we accept your definition, presumably it must mean an entity with id E exists if at least one datom has ever been asserted with E in the first position of the EAVT tuple.
I don’t think it’s a meaningful question. an entity is a key upon which to join datoms
it matters for what d/entity returns. (d/entity 9999) => {:db/id 9999}
(assuming 9999 has no assertions)
Why should entity
behave differently for different representations of the same concept?
(I have no idea the relevance of anything else you're talking about to THIS, which is my fundamental point/question. As far as I can tell, all of these existential questions about entities have absolutely no bearing on this, which is fundamentally a behavioral question.)
it’s an implementation detail that it’s not, namely that tempid records encode a negative number, which represents a tempid
the argument to d/entity got passed to d/entid
(or moral equivalent) at some point, and that’s why you have what you have
The utility of having entity work for a tempid is that you can still use entity to check if there are existing values for an attribute that you intend to set (and there's actually no worry about them being asserted elsewhere in the tx, since the existence of that would create multiple assertions, which would abort the transaction anyway).
Now, if I haven't communicated that clearly, I don't think that I can really explain it any better.
If entity returned nil for tempids, it would be fine, I'd just make sure that I would query for the existing values, and you'd have EXACTLY the same problem, as the query couldn't see anything else you asserted in the same TX but hasn't been committed yet.
which is why I was arguing, in both cases, the only appropriate thing is to check and abort
I don't need to abort - the transaction in my particular case is well-formed and meaningful - even with the result of entity
being passed a temp-id.
the check changes and is much more convenient if d/entity is consistent (really d/entid)
so you are using this tx fn in such a way that you have some guarantee from the application that it will never try to change something the tx fn will read in a way that would affect its functioning, nor ever compose this tx fn in the same tx with anything else that might do the same?
I agree completely on all counts that entity should behave consistently - and ideally, the behavior would be documented or specified somewhere.
1. In this particular case, the tx-fn makes assertions of a particular attribute. Something else making an assertion of the same attribute for the same entity somewhere else in the tx would be an error anyway, since you're not allowed to make ambiguous transactions. So no, that's not a problem. 2. Do you think that all functions we write, and especially transaction functions, are arbitrarily and infinitely composable? I assure you this is not a general property of transaction functions.
The fundamental problem here, as far as the argument you're making, isn't even about entity behavior, because as I said above, even if this didn't work with entity
at all, you'd just have to query for existing attribute values, and that would have the same potential issue you're worried about.
I think the problem that concerns you, then, is whether we can really have tx-fns that work on both existing and new entities, correctly in both cases.
for 1) I don’t know exactly what you are doing so perhaps you are avoiding this case, but the upserting case from earlier is what I was thinking of. 2) absolutely not, which is why the caution about throwing if someone accidentally supplies a tempid to a tx-fn that is expected to read the value of the tempid. Correct to your last two paragraphs.
What I will say is that the tx-fn in question that I wrote is pretty specialized, and I acknowledge the wisdom in what you're saying in general - at the very least we have to be careful, and in many circumstances, having entity return non-nil when something isn't really there could create problems.
not knowing your problem specifically, I would prefer using a sentinel value to a tx fn (say, nil) to indicate “I am minting a new entity id, it has no assertions before this point” rather than a tempid to communicate the same.
I like the clarity of your idea - in my particular case, it would just require slightly different code.
although of course, you may still need the tempid in order to hang new assertions off of it
Yeah, if I had upserting attributes in the entity, what I'm doing wouldn't work at all.
Since I do not have that case (currently), that means the presence of a tempid does actually indicate a new entity.
How do I get all attributes on a matched entity? Say I have a database with movies. And I want all movies before a certain year. How would I do that?
(d/q '[:find ?movies
:where [?movies :movie/release-year ?myear]
[(< ?myear 2013)]] db)
Right now this only returns the entity id. How would I change this to include all attributes on that entity?for your sanity, though, production code should generally be explicit about attributes
there’s also :keys
in query, if you just want a natural name for each member of the result tuples
user=> (d/q '[:find (pull ?movies [:movie/title :movie/directors]) :where [?movies :movie/release-year ?myear] [(> ?myear 2013)]] db)
[[#:movie{:title "Paddington", :directors [#:db{:id 17842874695549022}]}]]
How would I go further and retrieve the directors’ as well?