are you using rsync or similar to make your life easier ?

can you chop it up in smaller blocks?

@U052852ES Yeah, I am just transferring a few folders at a time, but the VPN connection will randomly close. If I could transfer it all in one go it would take around 48 hours it seems… @U0524B4UW Hm, I didn’t consider that at all. Do you think it would work with a SMB network drive?

rsync should work with SMB. I kinda remember doing that 12 years ago

and it zips things on the wire, if not already done so

Ok, better try it out then. Just got disconnected again. It seems to intentionally disconnect me because I am using too much bandwidth. At least that’s the message I get.

yeah in that case use something that can restart (which I think rsync can)

i don't think rsync cares about the type of filesystem - networked filesystems should be fine as long as they aren't weird (i think SMB will be fine, but i'm not sure about, say, S3 via FUSE). it compresses and can restart, and has some options that can help you with large files if you are seeing failures part-way through transferring large files (e.g. https://fedoramagazine.org/copying-large-files-with-rsync-and-some-misconceptions/ )

some details on how rsync calculates source/target file deltas (and so avoids transferring whole files after a previous transfer has partially succeeded): https://rsync.samba.org/tech_report/node2.html

If you use rsync -P it'll do partial

so it can be restored from that point onwards after a failure

I normally do rsync -vahP <src> <dest>

Do it with a torrent?

I don’t control the originating server.

It’s a brittle samba directory connected to by some proprietary software that the IT department wrote which establishes athis connection to when you log in to their VPN. This directory contains ~2 years worth of TIFF file uploads that have been gradually put there over that period by some people operating a scanner in some basement of the Danish Royal library.

I’ve asked to get direct access, but they say it’s not possible.

Upload to S3 and then download at the other end?

Or perhaps look at tailscale? Not sure if relevant.

I don’t control the originating server.