Recently, I had to run binding VMware ESXi NFS traffic to a custom vSphere TCP/IP stack for migration purposes.
In my case, the NFS server and the ESXi vmkernel port used for NFS traffic do not belong to the same VLAN. Therefore, I have to use Layer 3 communication for NFS connectivity.
The first step consists to create a custom TCP/IP stack for your ESXi host followed by the creation of a new vmkernel port binded to the custom TCP/IP stack:
esxcli network ip netstack add -N NFS
esxcli network ip interface add -s vDS_NFS -P 78 -i vmk5 -N NFS
esxcli network ip interface ipv4 set -i vmk5 -I 192.168.0.11 -N 255.255.255.0 -t static
In vSphere, the best practice is to create a dedicated (preferably non-routable) VLAN for NFS traffic. When not, NFS traffic is mapped by default to the vmkernel port where the ESXi Management Network is enabled.
Since I have 2 different VLANs for ESXi NFS vmkernel (vmk5 in this example) and NFS Server, I need to create a static route to pin the NFS traffic to the correct vmkernel port:
esxcli network ip route ipv4 add -g 192.168.0.1 -n 10.200.100.0/24
After completing all the steps above, I was confident enough to think that my ESXi could reach the NFS server…but no!
On the firewall, I could still see the NFS traffic going through the vmk0!
Luckily, I found the following VMware KB which helped to solve my issue:
NFS fails when using custom TCP/IP stack (50112854) (vmware.com)
Thanks to the commands below from the KB, I was able to:
- Stop the NFS service
- Unload the NFS modules
- Bind sunrpc with my custom TCP/IP stack for NFS
- Reload the NFS modules
- Start the NFS Service
/etc/init.d/nfsgssd stop
vmkload_mod -u nfs41client
vmkload_mod -u nfsclient
vmkload_mod -u sunrpc
localcli system module parameters set -m sunrpc -p netstackName=NFS
localcli system module load -m sunrpc
localcli system module load -m nfsclient
localcli system module load -m nfs41client
/etc/init.d/nfsgssd start
After running these commands, binding VMware ESXi NFS traffic to the custom vSphere TCP/IP stack worked like a charm 🙂
Update 21/03/2023:
With vSphere 8 Update 1, you have now the possibility to bind your NFSv3 to the vmknic of your choice directly from the UI (What’s New with vSphere 8 Core Storage | VMware)
