VMWare ESX and changing IP address of iSCSI SAN

During a recent subnet change project, I ran into a very unexpected problem. I was going to a new IP range and subnet based upon some expansion going on at a data center and had my Dell Equallogic PS6500 connected to two ESX servers to host my VMWare disk images as well as iSCSI connections inside of those VM's. Once I made the change on the SAN to the new IP address after shutting down the ESX servers and then started to boot the ESX boxes, I noticed it was taking an extremely long time. Further investigation revealed that the box was was looking for the iSCSI LUN's and could not find them, so it was stuck in the middle of a boot. This was obviously not a good thing.

I was able to get the box to boot into single user mode and then was able to get into it. Once there, I attempted to remove the iSCSI connections to the old host but was unable to do so with the following command:

#esxcfg-swiscsi –d
#esxcfg-swiscsi –k

Both did not work. I then found in the following directory:

/etc/vmware/vmkiscsid/vmkiscsid.db

This file contains a list of all of the iSCSI connections that the box is using. I simply renamed this file to vmkiscsid.db.bak and then was able to start my iSCSI config from scratch. Obviously the LUN's were okay as they were on the SAN and all I had to do was point my iSCSI connection back to the LUN's on the new IP range.

Once this was done, everything was back to normal and I was able to get the connections back to the SAN.

In hindsight, I should have disabled the iSCSI connections BEFORE I rebooted the ESX hosts so I ended up making it more difficult than it should have been.