submit-job throws:

No implementation of method: :-coerce of protocol: #'hasch.benc/PHashCoercion found for class: java.math.BigDecimal

I some BigDecimal configuration entry in lifecycle.

@isaac: ah, we have seen something similar to this before. We will need to implement hasch hashing on big decimals

@isaac I encountered this a few days ago. I was able to work around it in my project by doing as @lucasbradstreet said:

(ns hasch.extensions                                                                                                                                        
  (:require [hasch.benc :refer [magics PHashCoercion -coerce encode-safe]]
            [hasch.platform :refer [encode]]))

(defn ^bytes str->utf8 [x]
  (-> x str (.getBytes "UTF-8")))

(extend-protocol PHashCoercion
  clojure.lang.BigInt
  (-coerce [this md-create-fn write-handlers]
    (encode (:bigint magics) (encode-safe (str->utf8 this) md-create-fn))))

Notably we also had to add :bigint to their magics map.

thanks

Sure thing. I’ve been meaning to send a PR their way for it.

Hi folks! I'm trying to figure out how to do something like this: Suppose I have a job with three tasks. First one reads some data from db and produces a list of segments. Second one takes those segments one by one and processes them somehow. The third one should perform some action (write status "I'm done") once all of the segments from first task were processed by the second task. I'm not sure how to implement that third step. Currently looking at combination of global window + :punctuation trigger. Is it the right direction. Is it possible at all?

@jetmind that would work. Onyx internally does something similar to checkpointing "I'm done" for each segment in order to ensure every segment gets processed at least once.

Another option would be to have an output plugin write the "I'm done" message for your segment. Could you elaborate on your use case a bit more?

The problem is I don’t want to write “I’m done” for each segments, but only after all of the segments were processed

@jetmind: I would checkout the lifecycle hooks Onyx provides, one will probably do what you need. http://www.onyxplatform.org/docs/user-guide/0.9.10-beta1/#lifecycles

@jetmind You’d need a window, and a way to convey to that window what “completion” means.

It sounds like you’re trying to dynamically signal the completion of batch of data that is determined at runtime. You’re going to need to communicate out-of-band to let the downstream task know what to expect.

That’s pretty much what I thought, thanks for confirming my understanding