Fork me on GitHub
#onyx
<
2017-05-20
>
lmergen10:05:33

hey, i’m trying to figure the exact semantics of checkpointing / barrier synchronization

lmergen10:05:37

i’m a bit confused at the moment

lmergen10:05:47

while refactoring the sql plugin to 0.10, it appears as if the barrier snapshot doesn’t occur until at least 20 segments have been emitted

lmergen10:05:09

which causes the crash to fail, unfortunately

lmergen10:05:35

when i’m looking at https://github.com/onyx-platform/onyx-sql/blob/0.9.x/test/onyx/plugin/input_test.clj#L56, it appears as if the “old” logic threw an error after exactly 1 checkpoint had been written

lmergen10:05:47

with the new ABS, what would be a sensible semantic to make this happen ?

lmergen12:05:10

i’ve gone ahead and submitted a PR with this test case failing — i can use an extra pair of eyes on this one https://github.com/onyx-platform/onyx-sql/pull/22

lmergen12:05:37

other than that, input / partitioning / output / updates are working

lucasbradstreet15:05:59

@lmergen good stuff. I'll review it today. Thanks!