Fork me on GitHub
#datomic
<
2019-02-26
>
p14n13:02:37

Is there a way of referencing an entity at a point in time? I want a record to reference the value of an exchange rate at the time it was created. I can create a copy of the various bits of info, but I just wondered if I could instead simply reference the entity id and db and have that realised in the query

benoit14:02:51

@p14n You could store the transaction id or the date of the transaction of the point in time you're interested in.

p14n14:02:25

Yup, I was thinking along those lines. Means that I'd need to perform 2 queries (one to get the tid and one to get the value)

benoit14:02:58

You will have to query with a database value at the time you're interested in anyway I think. But if this value is going to be queried a lot I would store it separately in the database instead of having to query the old database value every time.

p14n14:02:17

I think you're right. I was just checking as referencing another entity at a point in time seemed like a 'natural' fit for datomic

favila16:02:32

if you need to store this reference in datomic itself, you can create a ref attribute that points to the tx

favila16:02:26

e.g. {:snapshot/entity eid :snapshot/time tx-eid}

parrot 5
Joe Lane18:02:11

Does anybody know the current query group instance size options?

Joe Lane18:02:36

I cant find them documented anywhere (I’m sure they’re subject to change, which may be why)

jaret18:02:24

@joe They are in the QG CF template. You can choose from the sizes there. Let me pull it up.

jaret18:02:38

79     "InstanceType": {
  80       "Description": "Cluster node instance type",
  81       "Default": "t2.medium",
  82       "AllowedValues": [
  83         "t2.medium",
  84         "m5.large",
  85         "i3.large",
  86         "i3.xlarge"
  

Joe Lane18:02:31

@jaret Follow up question. If I were interested in using ~10 GB of that tasty 500 GB NVMe SSD on the production i3.large instances (or from an m5.large query group) are there any datomic cloud issues around that? Can I write to a directory on disk from an ion? The use case is a Lucene index where documents are filtered by data coming back from datomic. I know this is a potential can of worms if I don’t keep the index size managed well, but lets assume this can be done. Asking for a friend who is estremely dissatisfied with ElasticSearch…

Joe Lane18:02:05

Said friend is also happy to do the index management off on a query group, so as to not affect the primary groups what so ever.

jaret19:02:56

@joe You should be able to do something like that. We make minimal use of disk, and have done nothing to get in the way of using the boxes (that we know of). The important caveat here is we have not tested/vetted writing to a directory from an Ion or implementing Lucene etc. So I cannot officially recommend the approach and it remains untested/unsupported. All that being said, you should try/test it and let us know how it goes. 🙂

👍 14
Joe Lane19:02:23

My “friend” is extremely excited

😆 4
steveb8n20:02:04

@U0CJ19XAM what was it about elastic search that your friend didn't like? I'll need this in future so really useful to know.

Joe Lane21:02:39

Can you ping me about it later tonight?

Joe Lane03:02:42

We have some search scenarios which require joins and subselections of data. Right now we have a system that encodes this stuff in elasticsearch and operationally its been difficult to keep everything in sync (we are doing Change Data Capture off a mysql binlog). When it was originally built we didn’t have datomic as a stable time basis so we were losing data whenever there was downtime. It was a nightmare. Now we have datomic cloud as a stable basis for our indexing. Our usecase is small enough and specialized enough that everything can fit on a single machine for now, and we benefit greatly from first doing a datalog query to limit the documents we are going to search against. Hope that helps @U0510KXTU. I’m happy to talk more over DM.

steveb8n04:02:15

Yes that does help, thanks. I think there might be an OOTB elastic search integration coming for Datomic cloud as well. Will be interesting to compare these two options

Joe Lane15:02:03

Interesting, where did you hear that?

steveb8n20:02:05

was mentioned in some Cognitect talk somewhere. can’t remember specifics but I remember thinking I’ll wait to see how that looks