Fork me on GitHub

@lucasbradstreet Thanks, I'll look into that for #1. Do you have any recommendations for question #2?


@stephenmhopper not sure whether the job tracking bit should be done within Onyx itself. As for the segments fitting in memory issue, yes you will want to setup an input source using an onyx plugin. onyx-seq is a good way to do this to get started as it’s fairly flexible, though you may outgrow it later


I see. So I can use onyx-seq with something that returns a lazy-sequence of values and it should avoid memory issues? At what point does one "outgrow" this plugin?


the primary problem is that the way it recovers from failures isn’t really ideal for all use cases


since it expects a seq, the way it recovers is to just drop the first n elements. For example, if you’re doing something like reading a big file / parsing it / etc, maybe a plugin that knows more about the medium and that could directly seek to the recovery point would be more efficient in the face of failures


that makes sense


Okay, I think I have enough to keep moving for now


a little while ago a nice library to create flowcharts, graphs and other diagrams was mentioned. Does anyone recall the name of that library? (It wasn't graphviz...)


awesome, thanks


how could I forget a name like that 🙂