Any experiences with allowing non-programmers to query analytics data? Our main db is datomic but we can't expect our marketing/business people to learn datalog. They need simple queries but giving them the ability to write/tweak those themselves (as they could with an SQL db) would be useful.

@pesterhazy you'll probably get closest to this with https://github.com/zcaudate/adi

@jethroksy that seems to try to do many things (we already have a schema etc.)

right, it wasn't designed for your use case

adi feels to me like a graphql-like wrapper for datomic

knowledge of datalog here is not necessary, so you probably could architect something along the lines of adi

I see

I was thinking more along the lines of - excel export based on pre-defined queries - hooking up queries to an SQL db (sqlite? mysql?)

haven't seen anything like that

@pesterhazy because the default return for a query is a set of tuples, it’s fairly trivial to write a function to wrap query results that exports to a comma/tab/arbitrarily delimited text file, then import it into whatever db or system you like. I did this myself several times to get stuff into a SQL store or other tabular data analysis tool (I use pandas in Python, others use R, Tableau, etc.). Of course there are also two common feature requests that would make this more trivial: (1) some kind of SQL mechanism to query Datomic (SQL->Datalog or otherwise) or (2) ability to download tuples from a query in console as CSV.

The team was aware of these when I was there, but there’s also a high bar of quality that has to be cleared especially re: supporting SQL queries (because a lot of people want the ability to plug in their analysis tool of choice, which means support for all kinds of arbitrary crazy generated SQL queries no sane human would ever write).

@bkamphaus I empathize and understand a generic SQL query layer would be hard to get right

feature request (2) you mentioned would get us pretty far as well

I suppose I could easily build something like that myself

the limitation is that it would have to be a long-running process as just booting up and connecting to datomic and loading segments brings query times up to 3min (long feedback loop compared to a quick "SELECT * from orders")

I don’t follow all the logic there. The Datomic system isn’t otherwise up?

the transactor is running

oh booting up a new peer to do this?

yes

as in a Jenkins job

that's what we've been doing (and what I would do in a typical SQL setting); it works fine but it's just not a great experience for interactive use because of the startup time

what’s the time sensitivity of these queries? could you just periodically store/cache the query results elsewhere?

yes, we also do that (push a CSV to S3)

all of that works to some extent, I'm just thinking out loud if there's an elegant possibility I've missed

I’m trying to retract an entity. But when I run @(datomic.api/transact conn [[:db.fn/retractEntity [:db/ident :item/ident-name]]]), I get an error saying IllegalArgumentExceptionInfo :db.error/invalid-attribute Schema change must be followed by :db.install/attribute or :db.alter/attribute. Any thoughts on why this is happening?

@annataberski it looks like what you’re trying to retract is an attribute. You can rename things and alter some aspects of an attribute ( http://docs.datomic.com/schema.html#Schema-Alteration ) - but can’t remove it outright.

gotcha. thanks @bkamphaus

Been puzzling over a weird bug for some time now, curious to understand more about how Datomic history databases work and whether this is actually expected. What we’re seeing is multiple [e a v t] facts share the same e a t but have different v values. How would this be possible?

When you assert a new fact (i.e. Marshall now loves ice cream) against a cardinality one attribute, that will expand into 2 datoms in that transaction. One is the retraction of the current value and the other is assertion of the new value. Both will have the same E, A, and T, but different V and Op (or added) values

@bmays ^

god bless 🙏

Bind [e a v t op] and you'll see the difference

Makes sense now, didn’t know retractions are automatically added

They will for cardinality one attributes