Fork me on GitHub
#onyx
<
2017-04-04
>
lellis02:04:10

@lucasbradstreet org.onyxplatform/onyx-datomic "0.9.12.0"

lucasbradstreet02:04:05

@lellis good to know. I don't think it makes much difference but it's good to know it hasn't happened in 0.10

shaunxcode05:04:56

@michaeldrogalis how did you settle upon rocksdb btw?

hunter14:04:48

@michaeldrogalis @lucasbradstreet i am going to switch back to 0.9.15 for this topology for now. it consistently dies, corrupting the tenancy-id. The only log message that indicates what the issue might be is 2017-04-04 08:42:10.323 INFO default o.a.c.f.state.ConnectionStateManager - State change: SUSPENDED ... apache curator suspending zookeeper cconnections?

michaeldrogalis15:04:03

@shaun-mahood Limited number of options for a fast, embedded K/V store. RocksDB and LMDB were the two front runners. We had good evidence that RocksDB worked well for this use case since Samza was using it for something similar. We’re likely to try LMDB with Pyroclast. If you’re interested, ping us in 4 weeks and we’ll let you know how they compare.

lellis18:04:45

Hello again! First of all, @lucasbradstreet the problem persisted in version 0.10.0.0-beta10 and datomic 0.9.5561. But i discovered the problem occur when i set the plugin to start from beggining of database. If i set the :datomic/log-start-tx 2038 (this number indicates the next tx in my database) everything works ok, with no exceptions. So my workaround was to set these tx to from beggining. But i have one question, if my peers going down for some reason, the zookeeper will keep the last TX and when i init again my peers the beggining will be theses zookeeper tx right? Or will beggining from 2038 (for example)?

hunter18:04:37

last TX in zookeeper

lucasbradstreet18:04:49

@lellis That is very good information. Maybe I can figure out a work around then. If you want to be able to resume from job to job, it’ll do it automatically in 0.9, or via resume points in 0.10 http://www.onyxplatform.org/docs/user-guide/0.10.x/#resume-point