I have been trying to automate the testing of our NIC cards with VXLAN, and it has been nothing but a nightmare. I am using the 'net-dvs' commands per VMware NIC certification guides, and the documentation is rife with errors. Anyway, I finally was able to configure two ESXi5.5 hosts and send encapsulated traffic between two virtual machines; one on each host. However, there is still a VXLAN issue. It is erratic in operation. By erratic I mean a test case would work, at the most, two times in a row, and then it would fail. In this case, failure is defined as bi-directional traffic does not pass between two virtual machines, one on each host. Run it again, and it would mayber pass or fail. I finally decided this is an ESXi 5.5 issue since it occurs with two different vendors NICs, and different drivers.
I upgraded my ESXi hosts to 6.0 hoping that would resolve the issue. Wrong.
After loading 6.0, my first thought was that VXLAN is now correctly working. I was able to run my test case 10 times, and it passed each time. However, after looking into it, I realized it is passing because the traffic is not encapsulated VXLAN traffic. It is regular traffic going between two VMs. Apparently, VXLAN is missing from ESXi 6.0
6.0 Issues:
[root@dut1088:~] vmware -v
VMware ESXi 6.0.0 build-2494585
[root@dut1088:~]
This is what I have found with ESXi 6.0.
First, the VXLAN module ‘vdl2’ is missing.
[11:29:00] (AddVxlan) COMMAND vmkload_mod vdl2
[11:29:06] (AddVxlan) RESPONSE: ['VMKMod_ComputeModPath(vdl2) failed: Not found', 'vmkload_mod: Can not load module vdl2: not found']
Is there another module that replaces ‘vdl2’? If so then what is its name? If ‘vdl2’ is still used then where do I find it for 6.0? It was included in the default ESXi5.5.
However, the other commands for configuring VXLAN appear to work, except the traffic is not encapsulated.
[12:10:37] (AddVxlan) COMMAND net-dvs -s "com.vmware.netoverlay.layer0=vxlan" -p globalPropList TEST_DVS
[12:10:42] (AddVxlan) RESPONSE: []
[12:10:42] (AddVxlan)
[12:10:43] (AddVxlan) COMMAND net-dvs -s "com.vmware.net.vxlan.vmknic=1" -p pgvmk_vxl_1 TEST_DVS
[12:10:48] (AddVxlan) RESPONSE: []
[12:10:48] (AddVxlan)
[12:10:48] (AddVxlan) COMMAND net-dvs -s "com.vmware.net.vxlan.id=16777216" -p VXLAN TEST_DVS
[12:10:54] (AddVxlan) RESPONSE: []
[12:10:54] (AddVxlan)
[12:10:54] (AddVxlan) COMMAND net-dvs -s "com.vmware.net.vxlan.mcastip=16777455" -p VXLAN TEST_DVS
[12:10:59] (AddVxlan) RESPONSE: []
Second, the ‘esxcli’ command for verifying VXLAN traffic statistics has been removed from ESXi 6.0. The ‘vxlan’ namespace appears to have been removed from 6.0. However, it might be possible the ‘vxlan’ namespace gets installed with the VXLAN 'vdl2' module.
[root@dut1088:~] esxcli network vswitch dvs vmware vxlan stats list --vds-name TEST_DVS
Error: Unknown command or namespace network vswitch dvs vmware vxlan stats list
So, my second question here is what command has been added to verify VXLAN statistics if the ‘vxlan’ namespace has been removed?