Fork me on GitHub
#onyx
<
2016-09-05
>
mariusz_jachimowicz11:09:34

The method (defmethod extensions/subscribe-to-log ZooKeeper is sending some exception to the channel via

(catch java.lang.IllegalStateException e
         (trace e)
         ;; Curator client has been shutdown, pass exception along
         (>!! ch e))
       (catch org.apache.zookeeper.KeeperException$ConnectionLossException e
         ;; ZooKeeper has been shutdown, pass exception along
         (trace e)
         (>!! ch e))
       (catch org.apache.zookeeper.KeeperException$SessionExpiredException e
         (trace e)
         (>!! ch e))
       (catch Throwable e
         (fatal e)
         (>!! ch e)
so when platform is applying entry from log via
(when-let [entry (<!! ch)]
        (let [result (extensions/apply-log-entry entry (:replica @replica-state))
then entry is an exception end there is no defmethod fot this kind of entry The question is - should this exception be applied into replica or should it be ignored?

lucasbradstreet11:09:18

Hmm. Right, so for the peers we do check if it's an exception, and we restart the peer because the zookeeper connection is in a bad state. However, I assume you're talking about the dashboard / playing back the log for monitoring

mariusz_jachimowicz11:09:58

yes, I am playing with the dashboard currently

lucasbradstreet11:09:52

For, say, the dashboard, it should also check whether the value is an instance of throwable. If it is, it should treat it as a connection loss, maybe display something to the user, and then try to subscribe to the log again / re-establish the connection.

lucasbradstreet11:09:01

ie. I would escape back to the same logic / handling that you were thinking of adding for when it can't connect in the first place

mariusz_jachimowicz11:09:01

Yes, am going to this direction for implementation of showing connection problems in the dashboard.

vladclj13:09:36

I use plugin onyx-http, and when post data to server I get error in onyx.log, help me someone)

Exception message: Output of write-batch does not match schema: {(not (= (name (:onyx.core)) (namespace :onyx.core/written?))) invalid-key}[m
        [1merror[m: {(not (= (name (:onyx.core)) (namespace :onyx.core/written?))) invalid-key}
       [1mjob-id[m: #uuid "70f04a36-867f-484f-9658-9dc2883c093d"
     [1mmetadata[m: {:job-id #uuid "70f04a36-867f-484f-9658-9dc2883c093d", :job-hash "f9a4c4fa748dfd6e702b9f3be3163e74f2ed046a32180b45c83f2905711139"}
      [1mpeer-id[m: #uuid "a8ffeeea-13ab-404e-9fd0-e1e917bd6ec9"
       [1mschema[m: {#schema.core.OptionalKey{:k :onyx.core/batch} [Any], :onyx.core/seal-ch Any, :onyx.core/outbox-ch Any, :onyx.core/kill-ch Any, :onyx.core/log-prefix java.lang.String, :onyx.core/workflow (constrained [(constrained [(pred task-name?)] edge-two-nodes?)] vector?), <#C1923ED97|schema>.core.OptionalKey{:k :onyx.core/state-ch} Any, :onyx.core/id java.util.UUID, <#C1923ED97|schema>.core.OptionalKey{:k :onyx.core/state-log} Any, :onyx.core/peer-replica-view Any, ...}
    [1mtask-name[m: :send-global-order
         [1mtype[m: :schema.core/error
        [1mvalue[m: {:onyx.core/seal-ch #object[clojure.core.async.impl.channels.ManyToManyChannel 0x5cba17f0

lucasbradstreet13:09:24

Oh dear. Yes, it shouldn't be using onyx.core/written since it's a plugin

lucasbradstreet13:09:38

I can fix the plugin. I assume this is via with-test-env?

lucasbradstreet13:09:35

K. This is because with-test-env is doing extra schema checking. What version of Onyx / onyx-http are you using? I can cut you a snapshot

lucasbradstreet13:09:36

Actually, onyx-http 0.9.9 doesn’t assoc to onyx.core/written?. Can you try using onyx 0.9.9 and onyx-http 0.9.9.0 if you’re not already?

vladclj13:09:50

I try 0.9.10.0-beta1 and 0.9.9.0

lucasbradstreet13:09:07

Those are the versions that give the error?

lucasbradstreet13:09:42

k, investigating

lucasbradstreet13:09:22

Are you using any other plugins?

lucasbradstreet13:09:53

That should not give that error

vladclj13:09:07

[org.onyxplatform/onyx-kafka-0.8 "0.9.9.0"] [org.onyxplatform/onyx-http "0.9.10.0-beta1"] [org.onyxplatform/onyx-metrics "0.9.9.0"] [org.onyxplatform/onyx-seq "0.9.9.0"]

lucasbradstreet13:09:56

You’re not happening to use onyx-s3 or anything are you?

lucasbradstreet13:09:12

oh nevermind. that’s the ancient deprecated one

lucasbradstreet13:09:11

Can you check your own code for the string

onyx.core/written?

lucasbradstreet13:09:19

not sure how that could be happening with any of the recent plugins

vladclj13:09:40

one moment

vladclj13:09:36

this string only in onyx.log file((

lucasbradstreet13:09:59

Can you call lein deps :tree and make sure it’s actually using onyx-http 0.9.10.0-beta1?

lucasbradstreet13:09:19

and maybe try lein clean? 😮

vladclj13:09:47

I think I found the problem we use two onyx-http plugins)

vladclj13:09:44

first plugin is created by our co-worker v.solovyov

lucasbradstreet13:09:33

Is there any reason you can’t switch to the “official” one? We brought @v.solovyov’s version in house

lucasbradstreet13:09:55

I think we pretty much maintained compatibility, but if there’s something missing let me know and I will add it.

v.solovyov13:09:15

heh, my version probably was just left in project.clj unnoticed

lucasbradstreet13:09:33

Yeah, it sounds like it 🙂

vladclj14:09:51

thanks for help)

lucasbradstreet14:09:28

No worries. Glad you figured it out.