Clojurians Log v2

You can turn on closed schema if that's what you want. That will throw an error if you insert anything that is not defined in your schema (sounds like the opposite of disabling the schema on write).

https://github.com/juji-io/datalevin/blob/ede16dd86ff5fcd9fb90d4bcbf632470a695b2c1/CHANGELOG.md#added-7

I think it is 😅 . I'd try it out shortly. thanks.

There's also :validate-data? which will throw errors if you insert the wrong type

I normally use both

It's not crucial in my use case; I'm indexing a ton of data and wouldn't want attributes to be created without my knowledge. I'm currently using malli, but would prefer to do without it for a tiny extra performance gain

What would be the desirable behavior instead? Ignore the unwantted attribute? Error out?

From the description, you seem to want to have a whitelist of attributes, and ignore anything not on that list?

Isn't this easier to do in user code than adding something to DL. It seem to complicate things quite significantly, as there are multiple paths of data ingestion.

I'd want to throw an error or get some kinda value that indicates failure or extra attributes created; So that I'm alerted that I didn't handle an attribute in my schema.

Yh, I guess user space is the way to go

so this is a form of more strict schema check. Sure. File an GitHub issue, we can probably add this. I think an entry of a whitelist in the option map should do it.

@huahaiy If I might ask one more question.. I'm ingesting json data from external sources; obviously, the keys aren't namespaced. Is there a way to auto namespace a map insertion (as well as remove the namespace on fetching)? I'm also currently doing this in user space, but wondering if there's already something similar

> File an GitHub issue, we can probably add this Will do. thanks!

JSON in/out is on the roadmap, when we get to that, auto name spacing can be dealt with as an option.

Would it be useful to add an entry of :attributes-created in the transaction report?

On the other hand, if you are doing bulk ingestion, I would recommend not transacting things, instead, use init-db and fill-db, it's much faster. Of course, you will need to do more work in user code to translate your data into datoms. Transaction slow down is mainly due to having to check existing data (a lot of reads), and these reads also use the same read/write transaction, so everything is done sequentially.

the speed difference is at least 4x

Yes, and :attributes-created would be useful. I added that to the feature request as an alternative.

> the speed difference is at least 4x I've only sample with a few thousand of entities per tx, so never really noticed any slowness. the actual data would be millions per tx, so I'd have to look into this. Much appreciation for your work @huahaiy

Thanks for these suggestions! @olajeremy123

Or, perhaps, more generally, a way to get the schema that resulted from schema-on-write? So you could compare it with what you started with, and either adopt the changes or address the bugs that caused them.

That sounds good too

What is the goal of turning off schema-on-write?

schema-on-write just assigns an :db/aid to an unseen attribute, That's about it. So there's really not much to it.

Could you describe what the issues are?