Well, it's hard to help with a problem if you don't know what the problem is. ;)

Fair enough! For context I recently started http://crosspromote.io/, where creators can post their newsletters. The information they post relevant to this problem is the audience size and the content tags describing what their newsletter is about. It's my responsibility to send out an email to two creators who run newsletters in a similar niche with similar audiences, in order to facilitate an introduction. Matching is fairly simple, since I just match them if their audiences are within 25% of each other, and the both share at least one content tag. With that out of the way, I'm trying to figure out an algorithm for how to best decide on who to send emails to. The main issue is a self-imposed one; I want to limit the number of emails sent out per week per creator. I can start off with a uniform limit for everyone, but the end goal is to have it be chosen by the creators themselves (so some newsletters might want only 2 or 3, others maybe 5 or 8, whatever). Ultimately I'm trying to optimize for emailing the most unique creators. I figured this kind of looks like a graphing problem, where the emails are edges and the nodes are individual newsletters. My initial approach would be to devise a weighting algorithm for each edge, based on the total number of edges the nodes have and how many remaining edges they'll have if that edge is selected (relating back to the limit of emails per week per creator). Then use some version of a longest shortest path algorithm that tries to hit most nodes with the least amount of weights. Having the edge weights be dependent on each other seems a bit complicated though, so I was just going to see if I could ignore that and instead rerun the algorithm x amount of times. Am I on the right path here? I'm sure there's some better solution, since this is just my intuitive approach, and I have a feeling there's A LOT of literature on this kind of problem.

> I have a feeling there’s A LOT of literature on this kind of problem I love these kinds of problems and there’s no shortage literature around for solving these problems. maybe we should ask a few more questions…

Fire away, I'm open to answering any.

> Then use some version of a longest shortest path algorithm that tries to hit most nodes with the least amount of weights do have anything working yet around this? can the data structure fit in memory and is that an assumption you can safely make?

I’ve played with Loom a bit, it’s fairly good at helping solve problems like this: https://github.com/aysylu/loom

I just got the nodes + edges loaded into a undirected graph using https://github.com/Engelberg/ubergraph . Yes I think for now we can assume it will all fit in memory without a problem.

Okay, I was going to say, if this is something you see growing in time, then using something like Neo4J or Spark Graph might be a better option

I do still need to make the weighting algorithm for edges though before I can run the longest short path algo

Yeah I'll worry about that when it becomes an issue. For now it would just be overengineering

yeah, first step is make sure you can solve the actual problem haha, so it sounds like you’re trying to optimize for the number of nodes you can reach based on budget

so a greedy algo might not be best

Oh I thought the longest part of the longest shortest algo meant it tried to hit as many nodes as possible, but with the shortest path

> meant it tried to hit as many nodes as possible, but with the shortest path hmmm, so the path with the highest ratio of nodes / sum-of-weights-traversed

I guess, I'm not 100% sure though

Looking at it more, it seems like the path approach might not be appropriate at all

getting closer haha, for that calculation it seems like it could be O(n!).

I suppose for this problem, and others like it, start with a few simple base cases, see if working through the solution to this pokes holes in your assumptions about how you’ve constructed the solution, maybe this isn’t a graph problem at all

Yeah I'm going to hack on it a bit more and look at what algs are available on ubergraph

cool, if it isn’t super proprietary keep us posted

also, if this is a problem that scales and needs to “run in production”, I’d again consider looking into Neo4J or Spark Graph (both are amenable to Clojure).

Thanks for the suggestion. Hopefully it gets to the point where that is a big problem!

well, not too big

http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.html I'd recommend reading through this, as both an overview of some graph algorithms that may be useful (example code in Rust) and as a caution against reaching for Spark when processing this data.

Your problem sounds in general like it lands in the realm of Collaborative Filtering, which Wikipedia has a good overview of: https://en.m.wikipedia.org/wiki/Collaborative_filtering

You might also try looking at or asking in the #data-science channel for more resources.

Awesome, thank you! I'll check out those resources.

ugh, I had no idea there was data science channel

Yeah not a fan of the default to hide most of the channels

Hmm my initial impression of collaborative filtering is that it's meant for matching things together based on large amounts of data points. But that's not really the case here. I only have 2 data points to match on, and it's not going to get any bigger. Instead I need a way to optimize filtering. Maybe my understanding is too shallow though?

I would agree to be careful w/ Spark fyi - that’s long tangential discussion though, but point well made @U03QQS7341W - have you looked at Algorithms for the Intelligent Web? The Manning series book, I seem to recall there are some examples that are similar outlined in it.

https://livebook.manning.com/book/algorithms-of-the-intelligent-web/chapter-4/

No I haven't, but I'll take a look. Thanks!

chapter 3 & 4 covers recommenders and collaborative algos like respatialized mentioned. I read this when the first edition dropped ages ago, but it’s still relevant today.

We have this kind of adjustable standing desk and it’s 78x50cm. Works very nicely with laptop. It’s a Finnish brand “Selka Mahtuva” but I guess something similar is available elsewhere.

Oh cool, they seem to sell those to other countries as well https://selkastore.eu/

Ah this is nice! Thanks, I will check them out.

Not the cheapest ones but quality is excellent.

Can you attach a monitor arm?

I don't see why not :thinking_face:

I don't use external monitors though

I would recommend http://uberdesk.bg ;-)

You can order it with plot of any size and shape. Успех!

Thanks, I will check them out, благодаря :)

I actually tried it! The table tops are not solid wood, I could barely attach the legs and they came off again eventually. Will probably work with more solid desks, but I would avoid learning carpentry if possible :)

Fair enough

Doesn't seem like it's supported: https://github.com/jamiemcg/Remarkable/issues/155

Yeah just saw that, now looking through the code to see how easy it would be to add

I live in Chicago. I would be happy to meet up for a bit depending on the time and place. I'm curious why your asking in the clojure slack.

@U0DJ4T5U1 good question - interested in meeting folks working in Clojure; in my community I don't meet many other Clojure programmers. I am mostly interested in Clojure for data munging, math and science applications. I like the pragmatic data centric approach. Clojure reminds me of the things I like working with the J programming lang

Sure, i would be happy to chat about that. I don't do much in the way of statistical analysis however. Let me know as time draws closer where and when you have in mind and I'll try to meet you.