Is there any way to make datomic with sql storage use a different table name than datomic_kvs?

@favila not at present. The last time somebody hit this I think they found they could do a level of indirection on the SQL side.

In a transaction, I’m stuck trying to retrieve the (temp) entity id or the (temp) transaction id and using it as a value for unique attribute. Use case: I want to create a low-cost, shortish unique id for an entity that will survive backup/restore and other cross-database operations. db:/id seems like the obvious choice, but I am slightly worried about their stability across backup/restore.

Is there any way to have the tempid usable as the value for a custom attribute? What would be the db.valueType of such an attribute?

Squuids are unsavory due to length.

I’ve considered an alias to :db/id, but I think that would suffer from the same stability issues across backup/restore.

Alternatively, can anybody confirm that :db/id is intended to be stable across backup/restore?

We think we've discovered a bug in Datomic related to string tempids being resolved incorrectly (two entities get mixed up in single transaction under certain conditions). Does anyone know what's the proper channel to report it to the Datomic team?

@cch1 Cognitect has warned that :db/ids are not guaranteed to survive backups/restores and should not be used as stable identifiers. (but they have been stable so far). I don't get how a tempid would help you though!

What do you mean specifically by "retrieve the (temp) entity id"?

By “retrieve the (temp) entity id” I mean “reference the (temp) entity id so that I can assign it as a value to my attribute”.

Thanks for confirming my concern about the stability of :db/id across backup/restore.

I'm still not sure what you mean. Can you show me a transaction?

{:db/id "user"
 :my/identifier "user"}

the tempid there is literally "user"

Yes.

there's nothing else to retrieve?

That’s enough to illustrate my example. In that case, :db/id is assigned a value (reported in the tempid resolution key of the transaction) and my custom identifier uses the same value. Significantly, my attribute does survive backup/restore because it is stored independently of the :db/id.

Does that make sense?

You mean you want the entity id which finally replaces the tempid?

yes. I want it to be a value of my custom attribute.

so, if :my/identifier is a ref type, this is done automatically

if :my/identifier is not a ref type, there is no replacement done, what gets written in is literally whatever you put

So can I create an explicit self-reference? Or a ref to the transaction itself?

That’s an interesting idea.

tempids are placeholders for real entity ids

Right. So a reference to myself would work.

the final step of tx expansion is to replace all tempids with ids

yes, it would work, but also be pointless for what you seem to want, which is a short stable identifier

I see. It would not be stable.

also, just use the db id

the db id is already there, why add another attr that just asserts itself?

it's pointless

IFF it were stored separately, it would be stable across backup/restore. But as a reference, that would probably not work.

What I really need is access to the tempid creation mechanism as a value, not a reference/entity-id

Again, I still don't see what that would accomplish

tempids are syntax

Unique identifier, stable across backup/restore, short-ish, cheap.

tempids are none of those things

squuid works, but too long

No, but the resolved value is all of those things except stable across backup/restore.

My hope was to store it as a value.

but the resolved value is the db id itself

Maybe not possible.

That’s OK.

The resolved value is unique, short-ish and very cheap (already computed in fact). I just need to get it assigned as a value during the transaction. But maybe that is not possible.

why does it need to be in the value slot of a datom? that I don't understand

Because otherwise I suspect it is not guaranteed to survive the backup/restore process.

As an entity id, it could be regenerated as long as the references were presevered.

As a value, it should be stable.

yes, but suppose datomic renumbers db/ids at some point. your "unique id" mechanism is now broken, because newly-issued ids may collide

you need to control the issuance mechanism too

That is a very good point.

Sigh. Maybe use squuids-as-a-long then.

so, we have created autoincrement counters in datomic for this purpose

Using a db fn?

external systems couldn't tolerate uuids because they were too long, and we couldn't find an encoding method that could compress them enough

yes, with db.fn

OK. Point to example? You have been very helpful.

it has the limitation that you can only increment one counter once per TX

Your scenario sounds exactly like mine. I was hoping for a solution that leveraged the internal uniqueness of tempids, but your point about clashes post-restore is the kiss-of-death for that.

That limitation might be OK…

I also vaguely heard that there are counter services (which just issue guaranteed-monotonically-increasing counters), or you could build something with zookeeper; and call these from within a tx fn

(you have to tolerate gaps in ids from failed txs)

anyway, let me dig up our solution

gaps are ok too.

Your solution will be appreciated. ZK might work too since we are already using it.

Gotta run, work emergency

our solution doesn't have gaps and doesn't require coordination with other services, but has the one-increment-per-tx limitation

Thanks for your help, @favila

@cch1 https://gist.github.com/favila/f33518c7e72a4079b5948d2f853053b0

Thanks!