Fork me on GitHub
#off-topic
<
2021-07-08
>
nate sire14:07:30

I am trying to convince myself to build a 64 core Ryzen Threadripper setup to test Clojure concurrency models on... someone tell me I am not crazy... instead of just using AWS?

❀️ 2
chucklehead15:07:18

just so we're clear on the rules, do you want to be talked out of it?

πŸ˜‚ 9
chucklehead15:07:14

you can get 64 core AMD or Graviton instances for around $1/hour on the spot market

3
nate sire15:07:28

yes... because it's $5k just for the CPU

nate sire15:07:11

so it is roughly equal for the 1st year.... $12 per day

nate sire15:07:54

I also need to put the rig in a server closet with air conditioner

chucklehead15:07:57

if you leave the instances running 24/7 and don't pay for your own power

nate sire15:07:43

hmmm... AWS sounds more realistic

nate sire15:07:12

I wanted to run all my unit tests in parallel... run a 1000+ test suite in seconds

Joshua Suskalo15:07:37

Considering you can set up nrepl and cider to work over tramp for file editing and repl usage, working from aws is much better if your only goal is to test performance.

nate sire15:07:09

parallel unit testing is my main daily goal... tramp?

nate sire15:07:24

looking tramp up now

Joshua Suskalo15:07:25

tramp is emacs' facility for remote file editing.

πŸ‘ 3
nate sire15:07:44

you edit your files on a server... no local setup?

Joshua Suskalo15:07:59

The only local setup is your emacs config

nate sire15:07:13

that is super interesting...

Joshua Suskalo15:07:18

Then tramp uses ssh tunnels to handle commands and files on the remote side.

Joshua Suskalo15:07:24

Cider has explicit support for tramp.

❀️ 2
nate sire15:07:40

I'd need to think... that could replace entire need for a local Docker setup

Joshua Suskalo15:07:03

Potentially. As long as your developers are allowed to have ssh access to aws devboxes

nate sire15:07:25

sure... they are allowed... just not allowed AWS keys

nate sire15:07:48

and stable internet

Joshua Suskalo15:07:59

This can also enable collaborative editing since multiple people can open cider connections to the same nrepl server, and work in the same repl. Not good if they aren't communicating, but potentially hugely helpful if they're pair programming over a call.

nate sire15:07:25

I am going to try it... cider + tramp

Joshua Suskalo15:07:46

tramp is also included in base emacs, it works any time you open a file with a path that matches their regex

nate sire15:07:47

because I had problems with docker working across osx and windows... did finally solve it

nate sire15:07:33

a local docker setup could just serve as a backup

nate sire15:07:38

if internet is down

Joshua Suskalo15:07:53

Could be a good option.

Thomas Moerman17:07:38

The joy of building such a rig is probably worth it ;-)

πŸ€“ 3
vemv17:07:02

> I am trying to convince myself to build a 64 core Ryzen Threadripper setup to test Clojure concurrency models on yeah if it's for learning a cloud offering is much less of an unnecessary commitment still I wouldn't discard having a beefy computer at home. You'd be surprised how many existing clj programs / tools get automatically better with more cores :) I heard the next Mac Pro will have 40+ cores, presumably w/o breaking bank

nate sire17:07:21

wow... 40 cores on a laptop. hmm. Although almost every laptop I ever had... had over heating problems. Then the heat offgasses the toxins in the plastics.

nate sire18:07:56

One time... I had my thinkpad on a plastic table... the gasses left a brown ring around the laptop where the two had interacted with the air... I am a big fan of aluminum laptops for that reason.

nate sire18:07:57

I think I will build the Ryzen system since I can get my money out of it after year 1.

πŸ‘ 4
vemv18:07:36

(I had meant tower mac pro)

dpsutton20:07:47

does anyone connect to a socket repl on a remote machine? how do you connect and do you worry about leaving a port open? Do you use ssh tunnelling?

borkdude20:07:25

even with nREPL I use ssh tunneling

dpsutton20:07:15

so you bind port X locally to the nrepl/socket port on the remote machine and then nc localhost X or connect to localhost X?

borkdude20:07:12

at work, we expose the port on the server in a VPN

borkdude20:07:41

and on my work laptop, in the VPN, I just do lein repl :connect -h server -p 1234

borkdude20:07:09

but I've also used the SSH tunneling approach

πŸ‘ 3
myguidingstar20:07:09

always connect via ssh or vpn. Ensure the remote machine only open the port for local network interfaces and not all the internet. There are bots that scans for open port automatically, you won't be safe for long

dpsutton20:07:57

yeah trying to figure out how to securely do this locally. that's why i was hoping the port tunneling would be secure

dpsutton20:07:19

then the only port is 22 for ssh and then using that to bind port 50505 or whatever

myguidingstar20:07:27

on second thought, I may have not made myself clear enough. When I say "open the port for local network interfaces", "local" means local to the remote machine, not your local machine. So be careful. The keyword here is network interface. Let me know if you understand this

dpsutton20:07:09

i do not πŸ™‚ My networking knowledge is quite deficient. And i'm hoping to find a guide for how to do this (or something analgous) so I can be confident i'm not exposing my machien to the world

myguidingstar20:07:32

ok, let me try to explain things in small steps. Please be patient

myguidingstar20:07:51

ah, there will be less to worry if firewall is used in the remote machine. Is it a linux box?

hiredman20:07:04

I am a big believer in something like https://gist.github.com/hiredman/86aeb916b478d9e57cbce8e0e678babd which just tunnels the repl over http(s) instead of on its own infrastructure

dpsutton20:07:59

(defn verify [id signature]
  true)

dpsutton20:07:05

interesting. i'll have to read that more thoroughly after work

hiredman20:07:49

yeah, the crypto bits aren't there, you'll need to pick your favorite

hiredman20:07:46

the assumption is you are using https to protect the connection anyway, the verify and signing stuff is really jsut for user authentication (public key, multiple uses, similar to something like ssh)

myguidingstar20:07:34

insteresting, though I suspect that would end up in more work, given that ssh/vpn is ubiquitous and easy to find doc or help

hiredman20:07:50

it to some degree requires no additional work, if your app is a running a webserver then you already have the the port exposed, you don't need to mess with the firewall, you don't even need ssh access to the server

hiredman20:07:55

to be clear, that is just a sketch, the public crypto stuff needs to be filled in, and should not be used as is

myguidingstar20:07:31

I'll finish my suggestion above. For the traditional approach, you need ssh/vpn client in your local machine, ssh/vpn server, start a repl server however you like plus a firewall in your remote machine. Forget about the network interface I talked about - firewalls solve the problem with that

dpsutton20:07:35

interestingly, i tried it last night with a pop os machine and i couldn't connect to the port. checked and ensured firewall rules were absent. so lots to learn and figure out

hiredman20:07:51

I have some similar code at work for tunneling a repl over a http://socket.io connection, which uses ed25519 keys for challenges and response

dpsutton20:07:11

you have the most interesting gists

seancorfield21:07:48

Mostly... I start the VPN, set up an ssh tunnel, and then connect to that as if it was localhost, since I use a Socket REPL for everything (local or remote).

ghadi21:07:49

I'm very compelled to try tailscale as soon as I have a free minute

dpsutton21:07:09

oh nice. can you share a redacted command for "set up an ssh tunnel"?

dpsutton21:07:35

is it just ssh -D 50505 user@host?

ghadi22:07:25

add -L <LOCAL_PORT_TO_EXPOSE>:<REMOTE_HOST>:<REMOTE_REPL_PORT>

ghadi22:07:43

to your normal ssh invocations

ghadi22:07:34

you do have to specify the remote host, even though it's commonly the same host that you're ssh'ing to

ghadi22:07:43

but it need not be

dpsutton22:07:16

contemplating a beefy windows machine as code server and could eventually end up on macbook air or other thin client if its seamless.

tvaughan22:07:45

echo -en                                                              "\
Host *                                                               \\n\
  LogLevel quiet                                                     \\n\
  StrictHostKeyChecking no                                           \\n\
  UserKnownHostsFile /dev/null                                       \\n\
  ForwardAgent yes                                                   \\n\
  ServerAliveCountMax 2                                              \\n\
  ServerAliveInterval 300                                            \\n\
Host bastion                                                         \\n\
  HostName 10.0.0.10                                                 \\n\
  User ec2-user                                                      \\n\
Host instance                                                        \\n\
  HostName 10.0.0.11                                                 \\n\
  User fedora                                                        \\n\
  LocalForward 50505 localhost:50505                                 \\n\
  ProxyCommand ssh -tF .sshrc bastion -W %h:%p                       \\n\
" > .sshrc
for SSH_PRIVKEY in "$BASTION_SSH_PRIVKEY" "$INSTANCE_SSH_PRIVKEY";      \
do                                                                      \
  echo "$SSH_PRIVKEY" | grep . - | ssh-add - > /dev/null 2>&1;          \
done
ssh -tF .sshrc instance env                                             \
  FOOBAR="foobar"                                                       \
  bash --login

tvaughan22:07:24

Extracted from a Makefile. Tunnels through a bastion host into aws, and port forwards to a vm on a private vpc.

paulocuneo00:07:01

reminds me of that space craft remote debugging story https://www.youtube.com/watch?v=_gZK0tW8EhQ

nate sire13:07:08

if you have a static ip... your internet provider might provide one... I always restricted port 22, remote machine, to only allow my ip address

nate sire13:07:21

that will keep you safe from most ssh attacks

nate sire13:07:42

until you setup a VPN which is the defacto standard these days

nate sire13:07:25

AWS offers OpenVPN which is the most flexible vpn standard... but I did have to patch Ubuntu one time to get it to work... Windows, OSX and Linux all can differ on how they use VPN protocals

henrik21:07:09

Has anyone tried something like https://tailscale.com/ for this?