Fork me on GitHub
#datomic
<
2015-06-08
>
robert-stuttaford08:06:38

<!channel> as can be seen in the topic, i’m doing a hangout this friday. if you’re planning to attend, I’ve made a google form with some questions, and i’d appreciate any input you have for me. thanks! https://docs.google.com/forms/d/1opX6Br27woFrJvih4pMPuOpTwlWB4il0EYiHuRwcwYI/viewform

danielcompton08:06:52

Ouch 1am in NZ

gjnoonan09:06:49

@danielcompton: It will be worth staying up for simple_smile

haduart09:06:43

@robert-stuttaford looking forward to it simple_smile

robert-stuttaford09:06:57

@danielcompton: sorry about that. curse you, solar system!

tjg09:06:27

Hah, and a bunch of us in Bonn/Cologne Germany will be at the ClojureBridge workshop at 5pm. Perhaps we'll come early and watch it on a big screen... 😛

mad15:06:07

June 12th, 3pm UTC +2 OK, I’ll be there!

robert-stuttaford18:06:06

@luke on large strings, would sticking those on their own entities in their own db partition help at all?

robert-stuttaford18:06:16

thereby containing the slowness to reads of those datoms

luke18:06:18

That's getting into implementation details a bit deeper than I know. I don't think putting them on their own entities would make that big of a difference.

luke18:06:42

Not sure about partitioning - I don't know how that works with adaptive indexing.

robert-stuttaford18:06:31

ok. part of the problem is that they impact the read-side perf of the rest of the nice, small datoms around them. i thought perhaps if they’re isolated, that might help when using only those small datoms

robert-stuttaford18:06:02

i guess for it to work you’d have to be careful about when you touch them, which is pretty much the same territory as just using a blob store

luke18:06:03

it seems logical but someone with a much deeper knowledge of the implementation than I do would need to answer that.

robert-stuttaford18:06:24

do you perhaps have such a person handy? simple_smile

luke18:06:49

I'll ask.

robert-stuttaford18:06:35

thank you! i’d like to make mention of it in that hangout i’m doing on friday. would like to put the partition idea forward, but only if it’s actually a good idea!

tcrayford18:06:22

finna guess that's top secret though

tcrayford18:06:13

@robert-stuttaford: think the blessed story is just to use a squuid pointing at whatever blob store you're using for datomic still though

robert-stuttaford18:06:49

yeah. it was just an intuition that i thought i’d sanity check. after all, i can actually talk to the people, unlike some other databases i’ve used in the past simple_smile

luke18:06:06

@robert-stuttaford: So I forgot momentarily that partitioning is by entity, not by attribute. So you'd have to put all the of the "heavy" attributes on their own entity.

robert-stuttaford18:06:35

yes. i was aware of that - they’d have to hide behind a ref

luke18:06:35

At that point it depends on your usage patterns if the extra ref traversal cost outweighs the "heavy" attribute cost.

luke18:06:11

Probably a benchmark is the only way to tell for sure, absent Rich or Stu themselves weighing in to my question simple_smile

robert-stuttaford18:06:10

yes. i guess my question really is, is this a viable strategy for dealing with this tradeoff of Datomic, and if so, when do you know when it’s time to a) use this approach and b) stop using this approach in favour of a blob store instead

robert-stuttaford18:06:39

again, just a thought experiment; just wondering if this approach would actually help or not simple_smile

robert-stuttaford18:06:11

datomic does kinda have some nice properties which are valuable to use even on bigger data -grin-