This seems like a sorely needed effort at a conceptual unification after over a decade of fiddling with layer architectures and buying better performance on credit by throwing ever more TPUs and data at every problem

Reminds me a bit of this brief essay from a few years back: http://www.argmin.net/2018/01/25/optics/

I also appreciated the animals analogy 😉 https://miro.medium.com/max/1400/1*hJfNh6YVzv9gBff4cFR3Hw.webp

Wow very interesting reads!

I am halfway through 😅 Do you have any good tips on reading material for algebraic topology and differential geometry?

gosh no, other than read all you can and do problems