hi Lucas is there a way to pass arbitrary values to :flow/post-transform functions? i have yeller logging inside one, and i need to provide it with configuration

i suppose i could grab the catalog from the event and dig around in there to find the config i injected for tasks

@robert-stuttaford: yeah, I think I would tend to inject that config when I need it, rather than have it part of the segment.

yup. that’s what i’ve done; remove it from the segment. just wondering about the pattern for /post-transform, as it doesn’t support the :onyx/params capability

Well, if you really need it at that point, I'd inject it into the event map via a lifecycle and then grab it back out in post-transform. But I would tend to inject it in whatever task that is handling the transformed messages instead

it’s the error flow cond which all tasks flow to

so injecting it via the task is not ideal

so i could use an :all lifecycle to inject config, but then how do i pass the config to the lifecycle

Ok so what I'm trying to say is that you should inject it as a param to your handle-error task

that’s simply (constantly true) right now

Since that's what handles all the exceptions, right?

the post-transform is what does the yeller logging

i can’t remember why we set it up this way

IMO that's the wrong way to go about it.

i updated the snippet above. you’re saying we can drop the post-tranform and do the logging inside the pred?

and then use onyx/params to pass config to the pred

Post transform is only really for transformation. Where it's flowing to is where you want to do any side effects

What is the tag-error task doing atm?

it just adds a :highstorm/error true tag to the transaction

so that we can know which txes caused issues later

Ok that makes sense. I guess I can see why you want to log it in the task throwing the exception so that you have the full exception object

One second

@robert-stuttaford: so what I’m thinking is you build a lifecycle like this:

https://github.com/onyx-platform/onyx-datomic#inject-conn

https://github.com/onyx-platform/onyx-datomic/blob/0.8.x/src/onyx/plugin/datomic.clj#L523

by returning the info you want in a map from the lifecycle function, you can inject the yeller config into the event map

Note that the lifecycle map has the config in this case

make sure to turn off the params injection though, because you wouldn’t want that on all the tasks

what is inject-conn returning?

The other thing worth considering is that if you setup a timbre appender that logs to yeller, it should all of these exceptions to yeller anyway

Lifecycle functions always return maps

see the second link

ok. let me get my head around all of this and i’ll come back if i have questions

so the root cause is this:

(defn log-segment-error [config {:keys [datom tx] :as segment} exception-obj] (if (yeller/env-is-yeller-env? config) (yeller/report config exception-obj {:environment (:environment config) :custom-data segment}) (log/error datom "\n\n" (pprint-str tx) "\n\n" exception-obj)))

when yeller is enabled, log to yeller. when not, log to disk (for dev)

i’ll shout in a mo once i’ve tried all this on

alrighty

so it’ll put whatever i put in the map (in the example you provided, at :datomic/conn) into the top of the event map?

so that i can just read it right out of the event

Yep it just gets merged into the event map

jobs are stopping cleanly .. and quickly! hooray!

and the error handling appears to be working as well

Excellent!

testing the yeller appender now

oh drat

clojure.lang.ExceptionInfo: Unfreezable type: class yeller.timbre_appender$make_yeller_appender$fn__7913 yeller.timbre_appender$make_yeller_appender$fn__7913@18c41315

nippy is choking on the appender

Hmm. Are you putting it in a segment?

I assume you're not trying the event map approach yet?

as an onyx/param

ok will have to rethink that

brb meeting

Oh if it's an onyx/param you might have it in the wrong order

I mean your function might think it's a segment

K later

hey Lucas, i’m getting this every now and then as i use the reset: Missing file for cnc: /var/folders/5h/lrqzt7gx61gg_zkqt47pmfzr0000gn/T/aeron-robert/cnc

my understanding was that this is all automatically managed now?

On startup or shutdown?

Embedded media driver, right?

embedded yeah

on startup

this is locally in dev

You're sure the previous instance is completely shut down first?

i’ll admit it might not be

That's my first thought - that the previous one is deleting it as the next one is startup up

ah, that sounds likely

Maybe we should drop the deletion on shutdown now. We do it to clear the space

doing a fresh run now. pretty close to having everything stitched up on the error handling front. then i can move to test servers and work on getting two nodes processing tasks, which is the last issue i have

Cool

it’s happening on a fresh jvm startup

Ok interesting. Actually throws an exception and won't start up?

yup

java.lang.IllegalStateException: Missing file for cnc: /var/folders/5h/lrqzt7gx61gg_zkqt47pmfzr0000gn/T/aeron-robert/cnc
java.lang.IllegalStateException: Could not initialise communication buffers
     clojure.lang.ExceptionInfo: Error in component :messaging-group in system onyx.system.OnyxPeerGroup calling #'com.stuartsierra.component/start

And definitely starting a media driver huh?

i.e. embedded-media-driver? true

😊

i just checked, i’m not

how’d that change...

Mmm

sorry

No worries

just tested a transactor death. yeller+timbre is working!

Nice! In that case you don’t need to inject the lifecycles, right? Just use timbre to log.

well, we still need to tell it whether to log to yeller or not

Michael said that lifecycles only manages what happens when errors occur, we still need flow conditions to actually log to yeller

or do i misunderstand things?

i can control whether the appender is included or not based on env, so that’s ok

You setup yeller as a timbre appender right? I just mean you can use timbre's info, warn and error from your flow conditions

oh i see

Rather than having to inject in the yeller config

hmm

we are using the yeller custom-data arg to include the segment

Ah

(yeller/report config exception-obj {:custom-data segment})

which is hella useful

Yeah fair enough. That's kinda handy

s/kinda/hugely

so, we’re using logback. as soon as i provide an :onyx/config key to peer config, all our logback config stops working

i’d like to keep the previous setup - onyx.log for onyx, and our own two logfiles (hs.log and error.hs.log) via logback, but continue to use the yeller-timbre appender. right now, with :onyx/config set, all the onyx logging is coming to stdout instead, and, weirdly, so too is all the stuff usually destined for the logback files.

also, with the yeller appender included, i am now getting exceptions on this line

https://github.com/onyx-platform/onyx-datomic/blob/0.8.x/src/onyx/plugin/datomic.clj#L353

clojure.lang.ExceptionInfo: Unfreezable type: class clojure.lang.Delay clojure.lang.Delay@7d707e47

testing with the appender disabled to verify

ok. the delay error is still happening. i’ll try to track down why. definitely want to figure the log file config thing out, though

Huh. That delay issue again.

For the logback stuff, maybe timbre is messing with it. Could you post your onyx log config?

it’s literally just this:

{:appenders {:yeller-appender (yeller/timbre-appender config)}}

Ah, so you’ve basically switched off all the other appenders

-blush-

i’ll update-in + merge

I don’t think that’ll help. Basically if you supply a config it replaces the existing one automatically

Oh, I thought it should merge

Ah, the rotor one is only used if you don’t supply one

true. i can’t merge

see https://github.com/onyx-platform/onyx/blob/0.8.x/src/onyx/static/logging_configuration.clj#L20

cool, i’ll duplicate that stuff and include yeller

i wonder what the delay issue is? it’s a fresh onyx/id, so it can’t be old ZK state

the onyx/id is based on git sha now

Yeah, you’ve seen that one before IIRC

yup. you did something clever and it went away

can you post the stack trace?

I don’t know if we did anything last time

i sure can, in about 20

I thought it was something in your code

just getting the logging fixed

it’s quite late for you, isn’t it?

Yeah, almost time to sleep

ok. please don’t stick around on my account let’s catch up tomorrow!

No worries. G’night!