Remotely Hosted Development Servers: Performance Challanges

Question

Our company is planning, requiring all dev (coding) work to move to ,

1. Remote servers, with each individual dev given their own desktop on server
2. Devs will access these using standard laptops

Reason

1. Security
2. better hardware \ configuration available at lower cost, say 64 GB RAM 1 TB SSD as compared to getting something like this on a laptop
3. Serves and Devs will be on different continents
4. We will be connected to atleast 1 VPN (my company's) and more if connecting to client systems, as we need to connect clients VPN to access their db. So real possibility will be connected to 2 VPN to work \ debug.

Question

1. Have you worked on such a setup ?
2. what are the pitfalls from developer perspective
3. How do you measure slow response, latency on the actual server which leads to developer desktop on server to respond slowly.