Cisco UCS Service Profile Template Config Generator


org	leave blank for root
Template Name	service profile template to build
UCS domain	unique number for this UCS domain -- 0-255 or 00-ff
Max hosts	1 - 160
vmknic	Enabled	NIC redundancy	Preferred Fabric	VLAN Name	VLAN ID
Mgmt	yes		A B n/a A
Mgmt	yes		B
vMotion			A B n/a A
vMotion			B
iSCSI			A B n/a A
iSCSI			B
NFS			A B n/a A
NFS			B
FT logging			A B n/a A
FT logging			B
VTEP			A B n/a A
VTEP			B

guest vmnics			VLAN list (optional)
Guest DVS1	yes
Guest DVS1	yes		B
Guest DVS2
Guest DVS2			B
Guest DVS3
Guest DVS3			B

vhbas				VSAN NAME	VSAN ID
FC vHBA 1		Multipathing	A B n/a A
FC vHBA 1		Multipathing	B
FC vHBA 2		Multipathing	A B n/a A
FC vHBA 2		Multipathing	B

Description

This tool generates Cisco UCS Manager CLI config for an VMware ESXi service-profile template given your chosen NIC/HBA configuration, including best practice QoS, pools, vnic/vhba templates, etc. It also builds a matching partial ESXi CLI config.

Pools

Each adapter uses a different MAC/WWPN pool. I do this because it's easier to verify UCS vNIC to ESXi vmnic mapping (which can be butchered by PCIe/NIC ordering). The pool formula is:

UCS domain ww use a unique UCS domain for each UCS FI pair to avoid MAC/WWPN pool overlap
Fabric x (A or B)
vNIC pair y
ESXi host z

For example, UCS domain 5, host 7, ethernet nic pair 2 would get

and fibrechannel hba pair 1 would get

NIC Redundancy (adapter teaming)

With ESXi teaming or Individual vmknics, adapters are created on both fabrics and teaming/vmknic config is left to the ESXi administrator.
With Fabric Failover, a single adapter is created on one fabric, with the other fabric used for backup. ESXi teaming should not be used.

QoS

The standard QoS configuration is:

CoS	CoS Name	Weight	Enabled	No Drop?	QoS Policy	Usage	Notes
0	best-effort	5 (50%)	yes	no	QoS_BestEffort	Guest VM's	all guest VM traffic
1	bronze	1 (10%)	yes	no	QoS_Scavenger	vMotion, VTEP	"scavenger" is more accurately called "less-than-best effort"
2	silver	1 (10%)	yes	no	QoS_Mgmt	Management vmk0	Without management in a separate class, I've seen vCenter lose hosts during multi-vNIC vmotion.
3	fc	1 (10%)	yes	yes	built-in	FC and FCoE	I recommend you leave this on, even if you don't use FC/FCoE
4	gold	2 (20%)	yes	no	QoS_Priority	iSCSI, NFS, FT Logging	A true high-volume queue for storage and fault tolerance sync traffic
5	platinum		no	no		unused	Available for future use; e.g., high I/O VM's or VTEP

QoS is not witchcraft:

The weights above are minimum guarantees. When there is unused bandwidth, any traffic class can use it.
Simply placing traffic types in different queues offers protection from other traffic types. E.g., With the above QoS, vMotion cannot cause packet loss for iSCSI or Guest or Mgmt traffic.
The single UCS QoS policy is enforced in five places: (1) the fabric interconnect uplinks, (2) the fabric interconnect server and appliance ports, (3) the lanes within the I/O modules of each B-Series chassis, (4) the VIC adapters within each server, and (5) the FEX is you're using one for C-Series connectivity.
Traffic coming from the upstream network into the fabric interconnects should carry matching CoS values. If the upstream network is out-of-your control, you may want to confine bulky traffic types like vMotion and iSCSI/NFS to a single UCS fabric. This is easiest to do with Fabric Failover, but can also be achieved with careful ESXi teaming config.
All classes are configured MTU 9216. ESXi NICs can be configured with any MTU from 1500 to 9216. I recommend 1500 for Mgmt, 9000 for vMotion/iSCSI/NFS/FT, 1600 for VTEP, and 1500 for guest VM's.

Other things

VMware KB 1031111 describes ESXi default behavior where vmk0 takes on the hardware MAC of the first NIC it sees. This is problematic on UCS systems that use "ESXi teaming" since the hardware MAC is tied to a certain fabric, and won't fully move to the alternate fabric during some types of failures. The solution is to rebuild VMK0 and ensure it's got a 00:50:56:xx:xx:xx MAC. See also CSCuw55695.
VMware KB 2007467 describes how to set up multi-vNIC vMotion.
The configs use pool "assignment-order sequential", a feature that requires UCS Manager 2.1(1a), UCS Central 1.1(2a), and later.
UCS and ESXi adapter ordering usually works fine, but google finds plenty of people who struggle -- especially when they have large number of adapters. By using different pools for each adapter, it's easy to match up esxcfg-nics -l with UCS service profile network config.
Questions or comments to craig (dot) weinhold (at) cdw (dot) com