The solution was to turn off all offloading in the Virtual Network adapter. I disabled the following for both ipv4 and ipv6:
- IPv4 Checksum Offload
- Large Send Offload Version 2
- TCP Checksum Offload
- UDP Checksum Offload
The virtual adapter has various types of offloading enabled, which can cause a problem if the physical adapter is not server grade hardware.