FTF 010: How to change IPs on infrastructure when NSX is installed?

By | November 4, 2019
0 Comment

Yesterday we needed to re-address VMware environment - Management, vMotion and VXLAN. This is not a complex task until you have NSX installed in your datacenter.

NSX VXLAN addresses reorganization

When you need to change addresses of VXLAN the following steps can be useful:

Prerequisites

  1. Create a new vSphere cluster and add to correct Transport Zone(s).
  2. Create a new NSX IP Pool for VXLAN.
  3. Create a new vDS used for VXLAN (Logical Switches).

Steps to properly readdress NSX VXLAN:

  1. Put a host into maintenance mode.
  2. Move the host outside (any) vSphere cluster. This step uninstalls NSX modules and removes old VXLAN vmkernel(s).
  3. Remove the host from current vDS used for NSX.
  4. Add host to a new vDS.
  5. Add host to a new Cluster.
  6. Enable NSX on a new cluster. Configure new VXLAN settings (with new VLAN and IP Pool).
  7. Repeat steps 1-5. Step nr 6 will be done automatically on the rest hosts.

Additional info:

  • In any cluster (and also host), only one VDS can be prepared for NSX. Different clusters may use a different vDS that has been prepared for logical networking.
  • VMware supports only when all vSphere clusters connected to the same vDS use the same VLAN id for VXLAN.

    VXLAN configuration in VMware NSX for vSphere 6.x fails with the error: Configure VXLAN networking VLAN xxx cannot be used. Other VLAN IDs are in use on the specified VDS (2091374)

Changing address for management vmkernel when vDS is used

When you need to change addresses and VLAN id for management vmkernel connected to vDS, the following steps can be useful:

  1. Put a host into maintenance mode.
  2. Disconnect (not remove) a host from vCenter.
  3. Replace/add a hostname for the host in vCenter's hosts file to point to vMotion's vmkernel. Confirm that vCenter resolves the host to a new address.
  4. Connect to ESXi host via SSH and add a static route to vCenter via gateway reachable via vMotion interface.
    esxcli network ip route ipv4 add --gateway IPv4_address_of_router --network IP_of_vCenter/32
  5. Connect the host again to vCenter.
  6. Repeat 1-5 for the rest hosts.
  7. When step 6 is done, change VLAN ID of port group for management vmkernel on vDS and then IP address on the interface on each host.
  8. Replace DNS records for hosts.
  9. Disconnect (not remove) a host from vCenter.
  10. Remove records added on the step 3 from the vCenter's hosts file.
  11. Remove a static route from each host.
    esxcli network ip route ipv4 remove -n network_ip/mask -g gateway_ip
  12. Connect host to vCenter.

Useful resources