Thanks! I'll take a look at this as well.

unless i misunderstood that would imply building a dataset with labels. I'd probably go a different route: building a recommendation engine (with tf idf ?) and tune it manually

For the binary approach, would I have to label articles that shouldn't go into my curation? Or could I get away with just labeling the 2k I have as "should" go into curation and find matches on that?

What is tf idf and what does manual tuning entail?

For most ML classifiers you need both positive and negative examples. Some algorithms are sensitive to the ratio of positive and negatives in your training set too.

Damm, that's not really a feasible option for me then.

Can't you just use the articles in your feed that you rejected?

Often a good approach for building a classifier is to start with simple ML algorithms then slowly increase the complexity when you can get no further value out of it. e.g. • manual keywords that you make up yourself (no AI) • Naive Bayes • Paragraph2Vec • Neural Network • etc You will want to evaluate how your model performances, typically you can trivially calculate Precision, Recall and F1 scores to see how well it is doing.

I don't keep track of every article I've rejected and not every article in the feed that is not in my accepted list should be considered rejected.

Any automed way I can think of that would get a bunch of "rejected" articles would be biased

You can probably build a basic classifier with about 500 to 1,000 positive and negative examples. So you could go through your negatives and filter them by hand.

One issue you may hit is that your criteria for the model to train on is too broad and across too many domains. e.g. if you couldn't explain it concisely to a friend and they would do exactly the same accepts/rejects as you then your target is not clear enough. One way around this is to build multiple classifiers (e.g. one classifier just to find interesting ML scientific papers that are at least 10 pages long and cover NLP - this example is less subjective)

Hmm... perhaps. Not too thrilled about doing that manual labeling. I'm going to try a non-ML approach first by filtering the articles in my feed by length. Since 99% of my sources are technical anyway, it might be "good enough" for now.

Agree - starting with an algorithm and heuristics sounds like a good start. You can add some keyword matching in there and give a score for the number of keywords that match).

Do you have any recommendations for algorithms for gauging the "quality" of a piece of text? Maybe something that measures the number of "filler" words or something along those lines?