I’ve implemented a new five-node VMware vSphere cluster for a customer. The hardware we’re using are HP ProLiant DL360 G7’s. They’ve got the new Broadcom 5709c Gigabit Ethernet Adapters, which present themselves as both standard Ethernet network cards as well as storage controllers (‘Broadcom iSCSI Adapters’ in the Storage Adapter section of the host’s Configuration Tab). The main advantage of these dependent hardware iSCSI HBAs is the offloading of iSCSI traffic to the TCP/IP Offload Engine on the NICs, which reduces the amount of processor power needed for the iSCSI traffic. VMware describes dependent hardware iSCSI as “Depends on VMware networking, and iSCSI configuration and management interfaces provided by VMware. This type of adapter can be a card that presents a standard network adapter and iSCSI offload functionality for the same port. The iSCSI offload functionality depends on the host’s network configuration to obtain the IP, MAC, and other parameters used for iSCSI sessions.”

As they are dependent hardware iSCSI HBAs, you’ll need to use standard VMkernel components to make them work. In a nutshell, these are the steps you need to go through:

  • There is a static relationship between a NIC port (vmnic#) and a dependent iSCSI HBA (vmhba##). Use ‘esxcli swiscsi vmnic list -d vmhba##’ to view which vmnic belongs to this vmhba.
    In my case, vmnic0 corresponds to vmhba32. vmnic1 corresponds to vmhba33.
  • Create a VMkernel port on a vSwitch with a single uplink (choose one of the vmnics which have a corresponding vmhba).
    I’ve created two vSwitch Port Groups: iSCSI0 and iSCSI1. iSCSI0 has vmk0 with uplink vmnic0, iSCSI1 has vmk1 with uplink vmnic1.
  • Bind the vmk# to a vmhba in such a way that the uplink on the vSwitch of the VMkernel port corresponds with the vmhba##. Use ‘esxcli swiscsi nic add -n vmk# -d vmhba##’.
    I’ve bound vmk0 to vmhba32 and vmk1 to vmhba33.
  • Verify that all is well with ‘esxcli swiscsi nic list -d vmhba##’

Some points to take into account:

  • Make sure that each VMkernel port  only has a single vmnic uplink. Make sure that the vmhba corresponding to the vmnic is the same one you’ll configure with esxli swiscsi
  • Use 1:1 mapping of vmk# interfaces to vmnic#/vmhba##
  • You can use iSCSI multipathing, as long as you use multiple VMkernel ports (each with it’s own vmnic/vmhba pair)
  • You can put all the VMkernel ports (vmk#) on the same vSwitch or you can create a separate vSwitch for each. Your choice!
  • Broadcom iSCSI adapters do not support IPv6 and Jumbo Frames. You should enable flow control though, especially if the network is congested.
  • The vmk/vmnic/vmhba pair you use for dependent hardware iSCSI (and software iSCSI for that matter) cannot establish a session to an iSCSI Target that is in a different subnet / VLAN. Use a second  vmk/vmnic/vmhba pair in that different subnet if you need to connect a single ESX-host to multiple iSCSI subnets / VLANs.
  • Make sure that each vmhba has a unique IQN.
  • Add the iSCSI Target IP address to the Dynamic Discovery tab of each of the vmhba adapters you want to use.
  • Performance Reporting for iSCSI Traffic is unavailable, because the traffic bypasses the regular network stack, where the performance reporting does is monitoring.