If you need to put a host in maintenance mode and only have access through ESXi Tech Support Mode, either local from DCUI or remote with SSH, in the following quick post I’ll show you how to do it using vim-cmd command.
In this final post I will discuss Server Profiles, what are they and how to create. As in the rest of the series I’m using Virtual Connect 3.10.
So, what is a Server Profile? We can define a Virtual Connect server profile as a logical grouping of attributes related to server connectivity that can be assigned to a server blade. You can see it as the connectivity personality of the server.
The server profile includes:
Preboot Execution Environment (PXE) enablement.
Network connection setting for each NIC port and WWN.
SAN fabric connection.
SAN boot paramenter setting for each Fibre Channel HBA port.
Once the server profile is created you can apply it to any server within the VC Domain. There is a maximum of 64 fully populated VC Server Profiles in a VC Domain.
As we saw in the network and storage posts the VCM can be configured so that blade servers use their factory-default MACs/WWNs and serial numbers or Virtual Connect provided and administered ranges of MACs and WWNs. These MACs and WWNs will override the default MAC and WWN values when a server profile is applied to the server and appear to preboot environments and host operating systems as the hardware addresses.
When a server profile is assigned to a Device Bay the Virtual Connect Manager securely connects to the blade in the bay and configures the NIC ports with profile provided MAC addresses and PXE settings and the FC HBA ports with the appropriate WWNs and SAN boot settings. Additionally the VCM automatically connects the server to the specified networks and SAN fabrics.
This server profile can then be copied or reassigned to another server as needed without interrupting the server connectivity to the network and SAN.
Once a blade server has been assigned a server profile and as long as it remains in the same device it does not require further VC Manager configuration during server or enclosure power cycle. They boot and access the network and fabric as long as soon as the server and interconnect modules are ready. If a server is inserted into a device bay that has already been assigned a server profile VCM automatically updates the configuration of that server before it is allowed to power on and connect to the network.
If a blade server is moved from a Virtual Connect managed enclosure to a non VC managed one all the ports automatically returns to their original factory values and settings in order to prevent duplicate MAC and WWNs within the datacenter because a blade server redeployment.
In addition to the above information the following points must be considered when working with server profiles:
Blade server and card firmware revision must be at a revision that supports Virtual Connect profile assignment.
Before creating the first server profile select whether to use Virtual Connect administered MAC and WWN ranges or the local factory default values.
After an enclosure is imported into a VC Domain the blades will remain isolated from networks and SAN fabrics until a server profile is assigned.
When Virtual Connect administered MACs and/or WWNs or when changing Fibre Channel boot parameters the servers must be powered off in order to receive or relinquish a server profile.
Fibre Channel SAN connections will display in the profile server screen only if the VC-FC module in the enclosure managed by Virtual Connect. If there is no VC-FC module the FC option wouldn’t appear in the server profile screen until a module has been added.
Some server profile SAN boor settings, like the controller boot order, are applied only after the server has been booted with the final mezzanine card configuration.
If PXE or SAN boot settings are made outside of Virtual Connect, the settings defined by the server profile will be restored after the blade server completes the next boot cycle.
If you have worked in the past with the 2.x Virtual Connect Manager revisions I’m sure that you will remember the Server Profile Wizard. That wizard has been removed from the 3.x revisions of VCM.
To start the server profile creation you have now to go to the Virtual Connect Home and in the Server area click on Define Server Profile.
In the Define Server Profile screen first enter the name of the profile, ESX01 in the example, and choose if you want to use factory default MAC and WWN or the VC-predefined.
Then move to Ethernet Network Connections. Here you can select the networks to assign to the ports, the port speed between AUTO, PREFERRED and CUSTOM and the PXE settings (ENABLED, DISABLED or USE-BIOS). By default there are only two connections created, to add more connections just right-click the area and choose Add connection.
In Network Name if you choose Multiple Networks a new icon will appear that will allow you to edit this connection type. Click and a new section will show up, this section allows to select the Shared Uplink Set and the networks. There is also a checkbox to set if you want to force the same VLAN mappings as the Shared Uplink Set to the different networks.
The next area is the FC SAN Connections. Assign the modules in the bays to the correspondent fabric and set the port speed.
Also in this section you can define the SAN boot parameters, click on the checkbox, the page will dim and a pop-up will appear.There you can configure each FC connection as PRIMARY, SECONDARY, DISABLED or USE-BIOS and set the Target Port Name and the LUN.
Finally we can assign the profile to a server bay.
Click Apply and the new server profile will be done. You can always edit the existent server profiles from the Server Profiles screen in the VC administration interface.
And this is the end. This series is done, if you have follow the correct steps outlined in the four posts you will have a fully operation Virtual Connect Domain. Of course there are a some topics I’d like to write about like iSCSI, FlexFabric and the VCM command line but I believe it’s better to do it in their own dedicated posts, stay tuned :-)
Please take into account that iSCSI is supported with Virtual Connect since the version 3.10 of Virtual Connect Manager and only with the Flex-10 and FlexFabric modules. However I’m going to leave iSCSI configuration for a future post, since I didn’t have many opportunities to try it with VC, and write only about Fibre Channel.
Before we start with the wizard and all the setup task is important to explain the Virtual Connect storage fundamentals.
The first concept to understand are the several key Fibre Channel port types. There a three basic FC ports:
N_Port (Node Port) – An N_Port is a port within a node that provides Fibre Channel attachment like an HBA port. VC-FC module uplink ports are N_ports.
F_Port (Fabric Port) – This a port on a FC switch connected to an N_port and addressable by it. These are commonly used in Edge or Core switches. The VC-FC module’s downlink ports are F_ports in order to allow the HBAs to login into them.
E_Port (Expansion Port) – These are switch ports used for switch-to-switch connections known as Inter Switch Link or ISL.
Additionally there are two other ports, however these ports are not typically seen in Virtual Connect environments.
NL_Port (Node Loop Port) – An N_port capable of Arbitrated Loop function.
FL_Port (Fabric Loop Port) – An F_port capable of Arbitrated Loop function.
The next key concept to understand in N_Port ID Virtualization or NPIV. It’s a T11 FC standard than can be defined as a Fibre Channel facility that allows to assign multiple N_Port_IDs to a single N_Port, this is a physical N_port having multiple port WWNs. Of course the VC-FC module must be connected to a Fibre Channel switch that supports NPIV.
And how manages Virtual Connect all this port stuff? I believe that an image is worth a thousand words, so first I will show with the below diagrams illustrate how FC ports and SAN are managed with and without Virtual Connect.
As it can be seen the SAN switches, like the Cisco MDS 9124e, that can be used in any blade enclosure including the HP ones are part of the SAN Fabric, that means the enclosure itself is part of the Fabric. These switches are connected to the SAN Core via E_ports or ISL.
In this configuration the SAN boundary has been moved out of the enclosure. The VC-FC module includes an HBA Aggregator which is an NPIV device. It passes, transparently, the signals from multiple HBAs to a single switch port.
Here it is how the whole process would go:
VC-FC module uplink port issue a Login Request, an FLOGI to the SAN and advertize itself as NPIV capable port.
Upon receiving an ACCept from the Fabric it would begin to process server requests.
Server HBAs would begin normal Fabric login process with the WWNs.
VC-FC module would translate FLOGI requests into an FDISC requests since a single N_Port can only receive one FLOGI request.
SAN switch would reply with an ACCept and provide HBAs with Fabric addresses.
The ACCept frames would reach uninterrupted the HBAs.
From then on all the traffic will be carried over the sane link for all HBA connections.
Now the the basic concepts are explained and, hopefully clear, it’s time to configure the storage.
We are going to use the Fibre Channel Setup Wizard to:
Identify the World Wide Names (WWNs) to be used by the servers.
Define the available SAN fabrics.
You can launch the wizard either from the Tools menu in the Virtual Connect page or right after finishing the Network Setup Wizard. From the welcome screen click Next and move into the World Wide Name (WWN) Settings page.
In this first page you can specify if you want to use the WWN settings that comes with the Fiber Channel HBA card or if the HP Virtual Connect supplied WWN settings.
Virtual Connect will assign both a port WWN and a node WWN to a Fibre Channel port, the node WWN will always be the same as the port WWN incremented by one.
There is key advantage when configuring Virtual Connect to assign the WWNs and is that, since it maintains a consistent storage identity, it allows blades to be replaced in case of failure without affecting the external SAN.
In the wizard select Virtual Connect assigned WWNs and click Next to move into the Assigned WWNs screen.
This screen is very similar to the MAC address range selection screen we saw in the previous post. Here you have to choose between an user defined WWNs range and an HP defined one. You must ensure that the selected range is unique within the environment.
Next we are going to define the Fabric, first you’ll be presented with a screen asking if you want to define the fabric.
After that we have to enter the Fabric name, assign the uplink ports and configure the speed.
After applying the configuration the wizard will move to the next screen where it will ask if you want to create more Fabric, for the example purposes I decided to create a another one named fabric_prod2.
When you are done with the second fabric finish the wizard and the storage setup will be done. You can review and modify the configuration from the Virtual Connect main interface.
The next post will be the last of the series and I will discuss about Virtual Connect Server Profiles. As always any feedback would be welcome :-)
In the first post of the series I introduced to you HP Virtual Connect and showed how to use the Domain Wizard Setup to initially configure a VC domain. In the following article I will outline the use of the Network Setup Wizard and explain Virtual Connect networking concepts.
Before we begin to setup the network it would be very useful to clarify the Virtual Connect port terminology.
External port – The Ethernet connectors SFP+ modules (either 1GB or 10GB), 10GBASE-CX4 and RJ-45 on the faceplate of the Ethernet module.
Stacking port – These are Ethernet external ports used to connect within a Virtual Connect Domain the VC Ethernet modules. The Ethernet modules automatically identify the stacking modules.
Uplink port – An external port configured within a Domain for use as a connection to the external networking equipment. These ports are defined within Virtual Connect by the enclosure name, interconnect bay that contains the module and the port number.
Uplink port set – A set of uplinks ports trunked together in order to provide improved throughput and availability.
Shared uplink port – This is an Ethernet uplink port that carries the traffic for multiple networks. The associated networks are mapped to a specific VLAN on the external connection, the appropiate VLAN tags are removed or added as Ethernet packets enter or leave the VC Domain.
Shared uplink port set – This is a group of Ethernet uplinks trunked to provide improved throughput and availability to VC Shared Uplink Set.
The Virtual Connect Network Setup Wizard will help to establish external Ethernet connectivity for the enclosure. With this wizard you will be able to:
Identify the MAC addresses to be used by the servers within the VC Domain.
Configure Server VLAN tagging.
Set up connections from the c-Class enclosure to the external networks.
The network connections can be:
Dedicated uplink to a specific Ethernet network.
Shared uplink sets.
The first screen of the wizard is the MAC Address Settings. As every server in the market the HP Blades come with factory-default MAC addresses already assigned to their network cards. However Virtual Connect can override these values while the server remains in the enclosure.
Virtual Connect access the NICs through the Onboard Administrator and the server iLO to manage the MAC addresses. It provides 64 predefined and reserved MAC address ranges. The wizard will give you the option to use either an HP predefined range or an user defined one. HP recommends to use the predefined ranges.
Once you have chosen the address range and click next the wizard will ask for confirmation before continue.
The next screen is Server VLAN Tagging Support. Here the wizard gives you two possible options:
Tunnel VLAN Tags
Map VLAN Tags
The first one, Tunnel VLAN Tags, supports only VLAN tagging on networks with dedicated uplinks where all VLAN tags passed through the VC Domain without modification and ports connected to networks using shared uplinks can only send and receive untagged frames.
On the other hand Map VLAN Tags allow you to add more than one network to an Ethernet server port and specify the VLAN mapping between server tags and VC-Enet networks. Also, the VLAN tunneling will be disabled for VC Ethernet networks with dedicated uplinks.
There is also a checkbox in the page to , if this option is enabled the server ports connected to multiple VC Ethernet networks are forced to use the same VLAN mappings as those used for the corresponding Shared Uplink Set and the network connections can only be selected from a single Shared Uplink Set. When this option is not checked server network connections can be selected from any VC network and the external VLAN ID mappings can be manually edited. In the example of the screenshots I decided to check it.
Below are another two optional settings for link speed control when using mapped VLAN tags. These settings are:
Set a Custom value for Preferred Link Connection Speed. This value applies to server profiles with a Multiple Networks connection defined and the Port Speed Setting set to Preferred.
Set a Custom value for Maximum Link Connection Speed. This value limits the maximum port speed for multi-network connections when a Custom port speed is specified.
In our example we’re not going to check neither of them . Click next to move into the Define Network Connection screen.
Choose the network type you want to define and click next. I choose .
The Define Single Network shows up. First define the network name (prod_net_01 in my example). There are three configurable values.
Smart Link – With this option enabled Virtual Connect will drop the Ethernet link on every server connected to that network if the link to the external switches is lost.
Private Network – This option is intended to provide extra network security by isolating all server ports from each other within the VC Domain. All packets will be sent through the VC Domain and out the uplinks ports so the communication between the severs will go through an external L3 router that will redirect the traffic back to the Domain.
Enable VLAN Tunneling.
Click in the Advanced button to configure Advanced Network Settings. Set the network link speeds that best suites your configuration.
Again from the Define Single Network page we are going to assign a port to our network. Click on Add Port and select an uplink port.
Set the Connection Mode to Auto if the ports are trunked and to Failver if not.
Click Apply and move onto the next screen. From this screen you can create as many additional networks as you need.
Now we are going to create a network using VLAN tagging. Click Next an move again into the Define Network Connection page, select Connection with uplink(s) carrying multiple networks (using VLAN tagging) and click Next. The Define Shared Uplink Port Set page will be displayed.
A shared uplink is the way Virtual Connect has to identify which uplinks carry multiple networks over the same cable. On shared uplinks the VLAN tags are added when packets leave the enclosure and added when leave. The external switch and the Virtual Connect Manager must be configured with the same VLAN tag ID for each network on the shared uplinks. The uplinks enables multiple ports to be added in order to support port aggregation and link failover, with a consistent set of VLAN tags. Virtual Connect has no restriction on which VLAN IDs can be used so the VLANs already used in the external infrastructure can be used here.
Since the VLAN tags are removed or added as soon as the packect enter or leave VC Ethernet Module shared uplink they have no relevance after the packet enter the enclosure. By identifying an associated network as the native VLAN will cause all untagged incoming packets to be placed onto this network, just one network can be designated as the native VLAN.
To finish the network creation assign a name (up to 64 characters with no spaces), add a port using the drop-down menu like in the single network process described above and add the networks you want to associate to the uplink. Finally click Apply.
In the final screen you will see now the three networks associated to a Shared Uplink Set. You can check this also from the Virtual Connect Manager page in the Ethernet Networks area.
And we are done with the Network Setup, in the next post I will show the storage part. As always any feedback would be welcome :-)
A friend asked me last week if I could produce a document for him explaining the initial basic setup of Virtual Connect, I decided that instead of that it would be better and more helpful to write in a series of blog posts, here it is the first of them for you to enjoy.
Virtual Connect is a technology developed by Hewlett-Packard for the HP BladeSystem c-Class enclosures. Provides server-edge and I/O virtualization in order to simplify the setup, maintenance and administration of server connections. It comprises a set of interconnect modules, both Ethernet and Fibre Channel, and a software known as Virtual Connect Manager.
Virtual Connect Manager, or VCM, is the single point administration interface for Virtual Connect. Under the hoods VCM is a software embedded into the VC Ethernet module, it can be accessed through a web-based interface or command line either with a serial connection to the Ethernet module or through a SSH connection to the module.
From the VCM only a single domain, with up to four enclosures, can be managed.
For large-scale infrastructures there is a more scalable version of VCM known as Virtual Connect Enterprise Manager, or VCEM. Unlike VCM, Virtual Connect Enterprise Manager is not embedded into the VC-Enet module, is a separate software that must be installed in another server. VCEM extends the VC management capabilities up to 250 domains and hundreds of blade servers.
Current series of articles will focus only on the Virtual Connect Manager GUI. Please take into account that I’m using Virtual Connect 3.10 version in the whole series and there some differences with the VC 2.x revisions.
When you login into the VCM for the first time a series of wizards will show up to help you with the initial setup of the domain. This article will cover the first of those wizards, the Domain Setup Wizard.
This wizard will allow you to:
Import enclosure configuration and communication settings
Name the domain
Set the IP address of the Virtual Connect Manager
Set up the local user accounts and its permissions and privileges
Confirm that the stacking links provide connectivity and redundancy
After the informative screen the first step will display. Here you have to provide the enclosure Onboard Administrator IP address and credentials, these credentials must have administrative level. Click next when finish.
Now VC Domain Wizard will import all the servers and VC interconnect modules within the enclosure.
In the next screen select the enclosure to import and click next.
A pop-up will show up to inform that the networking of all the blades within the enclosure will be disabled until VC Networking is properly configured. of course it will ask for confirmation.
After finishing the import the wizard will go the General Settings part. The Domain Setup Wizard automatically assigns a domain name based on the enclosure name, you can change the name when running the setup wizard or at any time later from the Domain Settings screen. The Virtual Connect domain name should be unique and can be up to 31 characters without spaces or special characters.
Next step is to configure the local user accounts.
By default the only local account is Administrator, this account cannot be deleted nor have domain privileges removed. You can also add up to 32 accounts with a combination of up to four levels of access. The available levels are:
Virtual Connect Domain
There is also an Advanced area for each account where you can set Strong Passwords requirement and the minimum password length.
With this the Domain Setup Wizard is done. In the next article I will write about the network setup of the enclosure using the Network Setup Wizard.