Fork me on GitHub
#datomic
<
2022-06-10
>
valtteri10:06:16

Hi, I have a two-part question: • Our client is running Datomic Pro and now they need to implement a full featured search into their application (full-text search, faceting etc.). One obvious idea is to index the data from Datomic to Elasticsearch or similar engine. I was wondering if there are other known good solutions in this space? • The client originally developed a multi-tenant system. However, years went by and now their biggest concern is "enterprise capabilities" (security especially) and there's pressure to start providing single-tenant option to certain clients. Does Datomic have some kind of a story for going from multi-tenant to single-tenant? I mean in both technical and licensing terms. Two questions above are related since I'm supposed to architect a solution for search that would also be future proof in single vs. multi-tenant sense. Thanks!

nottmey11:06:37

regarding your first question: this was the answer I got, when I asked a similar question last time (feed elasticsearch or similar and break down your query)

tatut12:06:19

or your nodes could have local lucene indexes, like xtdb does it... you might take a look at https://github.com/xtdb/xtdb/tree/master/modules/lucene even if it isn't directly usable

enn13:06:46

I’m not aware of anything off-the-shelf to index Datomic to ES. we rolled our own indexer that consumes the transaction log and updates ES.

pyry17:06:23

+1 to "roll your own indexer"

valtteri19:06:12

Thanks for all the responses! Local Lucene might be too low-level so I guess indexing to ES seems like a viable way to go forward. Still need to think about single-tenancy.. Both Datomic and ES feel too large and expensive if there’s a need to spin up a new environment for a single customer.

valtteri19:06:09

Or.. maybe that’s how it is and the price-tag for single-tenancy needs to be set accordingly. :thinking_face:

nottmey19:06:09

For search there is also something like http://algolia.com, maybe they have a better pay per use model

dazld11:06:04

algolia is wildly expensive - we’re using https://typesense.org for now, and it’s a solid piece of engineering. it’s been on my mind to write up how we’re putting all of this together - graphQL in front of datomic & typesense, and a tx log watcher that invokes updates.

👌 1
nottmey11:06:51

@U3ZUC5M0R nice recommendation, did not know about typesense. Do you know whether they comply with european data protection law? (because it seems like an US service and our customers want to search in european users data)

dazld11:06:50

worth asking them - if you self host, then clearly it’s ok, but the cloud is different

👌 1