I was seeing a ‘Misconfiguration Detected’ error after logging into the the Web client, selecting a cluster, Manage tab, Settings subtab, Virtual SAN->General. This went away after enabling VSAN traffic on management vmk0 interface. I found that, per vSphere 6.0 Documentation Center, "Network Misconfiguration Status in a Virtual SAN Cluster" this was likely due to misconfiguration of multicast.
I talked to my network admin, who made sure IGMP Snooping was disabled, and I turned VSAN off then on again, and the error disappeared temporarily. However, when choosing a specific host withing a cluster, I still see ‘Host cannot communicate with all other nodes in the Virtual SAN enabled cluster’. The "Misconfiguration detected" error has come back.
1. Does a multicast group need to be configured for this to work? I'm referring to vSphere 6.0 Documentation Center, "Multicast Filtering Modes", where it says:
In basic multicast filtering mode, a vSphere Standard Switch or vSphere Distributed Switch forwards multicast traffic for virtual machines according to the destination MAC address of the multicast group. When joining a multicast group, the guest operating system pushes the multicast MAC address of the group down to the network through the switch. The switch saves the mapping between the port and the destination multicast MAC address in a local forwarding table.
I'm assuming this doesn't apply to the VLAN that I have dedicated for VSAN multicast traffic, where IGMP Snooping should be disabled?
2. Can’t configure port 4095 to allow all trunked VLANs – is this needed?
I had thought that when configuring the network on the ESXi server, I had thought that I’d have to specify 4095 for the VLAN to allow all trunked 802.1q VLANs through.
As:
- the documentation I found only goes to 4094 and
- in the ESXi console I entered 4095 for the VLAN ID (which failed),
is the VLAN ID of the management network is fine? Or, could it be that traffic for this VLAN is not even getting passed through to the server because of this?
3. Below is what I understand VMware in terms of this error:
Cluster will not form – nodes won’t be able to communicate – Misconfiguration detected
–Option 1– Disabling IGMP Snooping => allows all multicast traffic through
–Option 2 - Configure IGMP Snooping Querier => if there is other multicast traffic and you are concerned that multicast traffic might flood network
I also got this link for what's required from the Cisco side, but it doesn't work (gives 404 error). Can anyone give instructions that work?
4. In comparing output from two servers, they can't ping each other (the 10.27.98.7* addresses) on vmk1. My network engineer says the switchports are all configured the same. Any thoughts on why they can't ping?
[root@host05:~] esxcli network ip interface ipv4 get
Name IPv4 Address IPv4 Netmask IPv4 Broadcast Address Type DHCP DNS
---- ------------ --------------- -------------- ------------ --------
vmk0 10.27.98.199 255.255.255.192 10.27.98.255 STATIC false
vmk1 10.27.98.71 255.255.255.192 10.27.98.127 STATIC false
vmk2 10.27.98.136 255.255.255.192 10.27.98.191 STATIC false
[root@host05:~] vmkping -I vmk1 10.27.98.71
PING 10.27.98.71 (10.27.98.71): 56 data bytes
64 bytes from 10.27.98.71: icmp_seq=0 ttl=64 time=0.096 ms
64 bytes from 10.27.98.71: icmp_seq=1 ttl=64 time=0.084 ms
--- 10.27.98.71 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.084/0.090/0.096 ms
[root@host05:~] vmkping -I vmk1 10.27.98.70
PING 10.27.98.70 (10.27.98.70): 56 data bytes
--- 10.27.98.70 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss
[root@host05:~] vmkping -I vmk1 10.27.98.71
PING 10.27.98.71 (10.27.98.71): 56 data bytes
64 bytes from 10.27.98.71: icmp_seq=0 ttl=64 time=0.088 ms
64 bytes from 10.27.98.71: icmp_seq=1 ttl=64 time=0.074 ms
64 bytes from 10.27.98.71: icmp_seq=2 ttl=64 time=0.081 ms
--- 10.27.98.71 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.074/0.081/0.088 ms
[root@host04:~] esxcli network ip interface ipv4 get
Name IPv4 Address IPv4 Netmask IPv4 Broadcast Address Type DHCP DNS
---- ------------ --------------- -------------- ------------ --------
vmk0 10.27.98.198 255.255.255.192 10.27.98.255 STATIC false
vmk1 10.27.98.70 255.255.255.192 10.27.98.127 STATIC false
vmk2 10.27.98.135 255.255.255.192 10.27.98.191 STATIC false
--- 10.27.98.70 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.072/0.080/0.089 ms
[root@host04:~] vmkping -I vmk1 10.27.98.71
PING 10.27.98.71 (10.27.98.71): 56 data bytes
64 bytes from 10.27.98.71: icmp_seq=0 ttl=64 time=0.726 ms
64 bytes from 10.27.98.71: icmp_seq=1 ttl=64 time=0.362 ms
64 bytes from 10.27.98.71: icmp_seq=2 ttl=64 time=0.561 ms
--- 10.27.98.71 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.362/0.550/0.726 ms