Hi @U0J6U23FW, was checking out https://sendgrid.com/docs/for-developers/parsing-email/setting-up-the-inbound-parse-webhook/#other-inbound-parse-documentation, will check out Postmark APIs as well, Thanks. I actually wanted to discuss the architecture of the notification system of which this is a small part. I missed out on explaining that. Here’re the details: Say there’s a list which has items and multiple people can act on the list. Upon any update on the shared list, all other participants should be notified of it (like how the Github PR comment works). And anyone in the shared list should be able to respond to that email and it’d trigger an action (like posting the Github comment) and that in turn updates the list and sends out emails again. I’d like to discuss the system design for this.

How do you see yourself solving this ? What does your current design look like ?

Hey @U716Q56R2, here’s the design I’ve come up with:

Notes:
ECS = Email communication System.
Using sendgrid as an example Email Service Provider.

- Have color coded the flow. ECS has two parts, appropriate path would be chosen based on the event of send or receive.
- Send Email flow begins by reading data from `UpdateList` Topic. (Follow Blue)
- Receive Email flow begins by receiving Webhook callback (Sendgrid). (Follow Orange)

What does ECS stand for here ? Entity-Component-System ?

ECS = Email communication System. 

It's the name of the Notification system I'm designing. Currently focused on email but would be later extended to include SMS/Push.

The blue flow: I’ll have a consumer/subscriber subscribed to UpdateList Topic. So any new message that comes (as “Update Message”) will then be immediately stored in the database, then the db-id will be sent to the background worker. The background worker will fetch the record from the db and resolve the necessary details like the list info, thread meta, email template and the email address. Once the data is processed, the processed data would then be stored in the db again. It’d then send the email to other subscribers/recipients using Sendgrid* API and the response would be stored in the db. The orange flow: Once the email is responded to, it’ll hit the Sendgrid* Inbound parser webhook and will send the callback response to ECS. The callback response payload will be stored in the DB and then the db-id will be sent to the background worker. The background worker will then resolve the list-info and will publish a new message to the UpdateList topic. Which will cause re-sending of the email notification to the rest of the subscribers.

I would separate the inbound (webhook that happens to be hit by sendgrid inbound service) from the outbound (sendgrid API) in that diagram. When you say, later extensible to SMS/Push, you probably mean you intend to have more outbound channels. Once we do that we notice that the flow becomes pretty much unidirectional, looking like :inbound -> (store-inbound) -> :queue-a -> (handle-inbound-and-prep-outbound) -> :queue-b -> (send-outbound) -> :outbound.

> When you say, later extensible to SMS/Push, you probably mean you intend to have more outbound channels. Yeah, that’s right. > Once we do that we notice that the flow becomes pretty much unidirectional, looking like `:inbound -> (store-inbound) -> :queue-a -> (handle-inbound-and-prep-outbound) -> :queue-b -> (send-outbound) -> :outbound`. The intend is to have multiple selectable outbounds later. So yeah, it makes sense to separate it.

Thanks!

multimethods are a useful building block for dispatch. (and their combining cousin, weavejester/intentions (on GitHub))

> useful building block for dispatch Where and when would you recommend implementing this? weavejester/intentions ~ It’s not under active development though, not sure if that’s something to think about - seen a lot of Clojure libraries that are abandoned before they make it to 1.0.0

A lot of Clojure libraries work fine and require no further patching 🙂 Intentions is not a framework or something. It is one useful construct. It is tested, it works. There is hardly ever any breaking change in anything clojure. Also, Semver is not very popular in clojure, largely because breaking changes are uncommon. Many projects follow a Major.Minor.CommitCount numbering, where Major.Minor bumping is an indication of the amount of new things or changed things.

W.r.t. to multimethods, for instance for your problem of sending notifications, imagine that a notification looks like { :notification/content "bla bla" :notification/title "Bla." :notification/href "" :notification/recipient-id some-user-uuid-here :notification/channel :notification.channel/sms } . You can write (defn send-notification [notification] ...) with a big case or cond based on the value associated to the :notification/channel key. Or you can (defmulti send-notification :notification/channel) and then (defmethod send-notification :notification.channel/sms) . More examples at https://clojuredocs.org/clojure.core/defmulti

Supporting several channels can be tacked on top of it without changing the single-channel schema. Say you expand your notification schema to have :notification/channels (plural) on a notification map. You can then use any looping construct. A for would be clear enough to me - some people would prefer reduce . Would look like (defn emit-all-channels-of-notification [notification] (for [ch (:notification/channels notification)] (send-notification (assoc notification :notification/channel ch)))) .

Interesting, I’ve not yet dived into the details of code-implementation yet but this does give me stellar pointers on how to go about it when I do. multimethods are cool! Thanks a lot for the clear and consie explanations @U716Q56R2, appreciate it 😄

You're welcome 🙂 I was hoping to be corrected in my shortcomings by someone - you know what they say about being wrong on the Internet right - but ... apparently I'm saying consensual stuff here.