High latency to Azure vm compared to GCP

Hello Yogesh Singhal

I understand that you've been experiencing latency issues between your Azure VMs and GCP VMs, particularly during data transfers over VPN.

Here are some steps to help troubleshoot and potentially reduce the latency:

1. Ensure the VPN is optimally configured. Check for unnecessary hops or routing inefficiencies. Verify that your VPN Gateway SKU (vpngw1) meets your bandwidth needs. If it's under heavy load, it might contribute to latency.
2. Consider deploying your Azure VMs in a Proximity Placement Group (PPG) to minimize network latency within Azure. This can improve communication speed between your VMs.
3. If not already done, enable Accelerated Networking on your Azure VMs. This feature can help reduce network latency and increase throughput.
4. Check resource utilization to ensure your VMs are not throttled on CPU or disk I/O, as this can contribute to perceived network latency. Monitor their resource utilization to identify any bottlenecks.

Since you mentioned different latency results from your laptop (4ms to GCP, 36ms to Azure), please share the following details over private messages to help narrow down the issue for further troubleshooting:

• Share the ping, traceroute and PsPing results from your laptop to Azure VM and GCP VM both?
• You can perform SockPerf (for Linux) or Latte (for Windows) to measure actual network latency.
• Use iPerf to measure actual throughput and identify bottlenecks.
• Please share the VM CPU and disk I/O metrics.
• Have you tried adjusting any VPN MTU settings?
• what is the data transfer rate and data path from Azure VMs and GCP VMs?
• If possible, please share the tunnel bandwidth or Gateway S2S Bandwidth metrics on Azure VPN Gateway to understand the VPN data flow rate.

Refer this How to validate VPN throughput to a virtual network

Please add a comment below if you have any further questions.

Dear Yogesh Singhal,

Thank you for sharing your question on the Q&A Portal regarding latency between Azure and GCP. I appreciate your engagement and the detailed explanation of your scenario it’s very helpful for understanding the issue.

To address your concern about high latency over the VPN connection, here are a few suggestions you might consider.

VPN Gateway SKU Upgrade: Since you’re using VPNGw1, upgrading to a higher SKU (e.g., VPNGw2 or higher) could improve throughput and reduce latency, especially for cross-cloud scenarios.

ExpressRoute or Peering: For lower and more consistent latency, Azure ExpressRoute (or GCP’s Partner Interconnect) might be a better fit than VPN. Alternatively, explore direct peering options between Azure and GCP if available in your region.

Network Path Optimization: Check if traffic is taking an indirect route. Tools like Azure Network Watcher or third-party solutions can help analyze the path and identify bottlenecks.

Proximity Placement: Ensure resources in both clouds are deployed in the same geographic region (Central India in your case) and, if possible, in zones with direct connectivity.

I would also recommend reviewing Azure’s performance benchmarks for VPN and comparing them with your observed metrics.
Thanks again for raising this topic it’s valuable for others who might face similar challenges. Let me know if you’d like more details on any of these points!
Best regards,

Alex

P.S. If my answer help to you, please Accept my answer

we tried all your intial things and nothing worked.

Placement grop is not applicable

accelrated networkign enabled

no load on VPN gw , its underutlized and even vm are underutlized.

data trasnfer from azure to gcp using DD command is 23 MBps.

how to adjust VPN Mtu?

Hello Alex, i will try moving moving azure resources to india west ( mumbai) region as the GCP is in mumbai only.

do i need to move all my resources? like VPN , storage or just moving 1 VM will be good for testing?

Hi Yogesh Singhal

To adjust the VPN MTU, you need to configure this on your on-prem VPN device. Navigate to WAN or VPN settings, look for MTU size, and adjust it based on your test results.

Additionally, moving to India West (Mumbai) could help improve latency since GCP is already in that region. You can test this by moving one SAP Hana VM and VPN Gateway.

You can also test by creating a new VPN in India West and running the latency tests. If the results are good, you can switch the connections to the new VPN.

Could you please let me know the connectivity or data path between the Azure VM and GCP VM for the data transfer? it is like Azure VM >> VPN Gateway > S2S connection > GCP VPN >> GCP VM?