Fork me on GitHub
#aws
<
2016-08-30
>
luposlip18:08:33

Hi guys. I’m currently having 2 docker containers in AWS ECS serve the Clojure based backend for a service I’m developing. They have a HTTPS/WSS load balancer in front (`AWS ELB`). The backend is completely stateless, except of course for the database. This all works perfectly, until I need to push data to the connected users, via Sente (https://github.com/ptaoussanis/sente) based websockets based on recent changes. The changes come from a single user, and arrives via the connected websocket. The changes need to be communicated to the other connected users. Problem is that the other users sockets are connected to only one of the containers, which may not necessarily be the same container as the initiating user is connected to. In other words - the atom containing the connected users are split between the 2 containers. Do any of you guys have a simple way of centralizing this notion of connected users? I’m thinking about using Datomic to store the connected users (thus distributing the connected users via the Datomic Peers), but are there better/simpler ways of ensuring a single repository of websocket-connected users across containers?

alandipert19:08:59

@luposlip memcache or redis are good ways to store sessions

alandipert19:08:29

the simplest fix is to use polling, not websockets 😄

alandipert19:08:04

websockets can make session tracking pretty hairy, so i prefer polling unless there's a compelling reason not to

alandipert19:08:23

and with datomic especially, polling is an awesome way to do it

alandipert19:08:36

since the data you're polling via datomic query is automatically edge-cached, usually

luposlip19:08:43

Thanks @alandipert. I’m not exactly sure I get what you mean about polling. Is it long polling vs. websockets you refer to? The auth part is working really fine, and everything works really nice with both push and pull, as long as the push/pull is only needed between the user(s) connected to the same container.

alandipert19:08:02

right, long polling from browser to web server

luposlip19:08:25

ahh, but that will give me the same issue, because the user is still only connected to a single container, right?

luposlip19:08:51

"Polling via Datomic Query" - from the client?

alandipert19:08:46

long polling with a timeout of some kind, or just straight-up short polling even

alandipert19:08:21

but yeah, my experience with push has been: try to use websockets, sessions are hard, and i end up sending heartbeat messages anyway to show "problems connecting" flash msg in client

alandipert19:08:37

then i figure if i'm sending heartbeats, may as well just poll and simplify everything

alandipert19:08:01

when my poll hits the server, with datomic, to check if something is different is efficient, since the poll usually boils into a query that's been cached at that server

alandipert19:08:52

it seems like that for most people websockets are an indirect way to escape the problems of the database they're using, since their database probably doesn't stream changes to connected peers soundly

luposlip19:08:49

well, perhaps you’re right. At least Datomic solves the synchronization of the connected peers (as in backend containers/servers). But not the one of the connected users 🙂

alandipert19:08:26

whether a user is connected or not isn't something you can keep in datomic?

luposlip19:08:11

I also use “heartbeat” pings every 15 seconds. But they aren’t that necessary, because the Sente socket library actually emit events to signal socket disconnection etc. If you haven’t tried Sente, I’ll recommend you to look at it 😉

alandipert19:08:33

i'll check it out sometime, it's been years since i made something like this

alandipert19:08:37

the last thing was a video conference site

luposlip19:08:57

Yes, I can keep that in Datomic. And I think that’s the way to go. Then I don’t have to introduce an extra component to the mix.

luposlip19:08:34

Let me know if you need input when you look at it, will happily return your favor! 🙂

luposlip19:08:41

Thanks for your input.

kenny19:08:38

@luposlip If you need real time and don't want to manage the web socket infrastructure you could check out something like https://pusher.com/ or https://www.pubnub.com/

luposlip19:08:51

Also nice input @kenny, will take a look at them. In the current project I don’t think I’ll use them though, because I’ve built everything (and it works), except the shared-across-containers repository of connected users.

alandipert19:08:06

yeah, for the project i cut my teeth on we looked at and were impressed by pusher, but couldn't use it because of HIPAA

luposlip19:08:17

ahh, good input. Can you use AWS in compliance with HIPAA?

alandipert19:08:41

parts of it, under various circumstances - depends what you need to store

alandipert19:08:48

we weren't on AWS either tho

alandipert19:08:39

it was a video chat app for doctors/patients, i'm pretty sure unless you're a healthcare app of some kind hipaa isn't a concern

alandipert19:08:59

and if it is, you need a lawyer to really know