Archive

Posts Tagged ‘networking’

DNS configuration with esxcli

October 17, 2011 3 comments

With release of ESXi 5.0 the esxcli command has been also vastly improved. One of this new capabilities is the possibility to manage the DNS configuration of the server.

The basic syntax for dns is:

~# esxcli network ip dns

This gives you two namespaces to work with:

  • search
  • server

esxcli_dns1

With the first one you can manage the suffixes for DNS search and the second is for the DNS server to be used by the ESXi.

  • Server operations
    List the servers configured:

image

Add a new server:

image

Remove a configured server:

image

  • Domain search operations

List configured domain suffixes:

image

Add a new domain:

image

Remove a configured domain:

image

Juanma.

Mapping VLAN to ports in HPVM

January 17, 2011 Leave a comment

Long time since my last post about HP Integrity Virtual Machines, well you know I’ve been very occupied with vSphere and Linux but that doesn’t mean that I completely eliminate HP-UX from my life, on the contrary… HP-UX ROCKS! :-D

This is just a quick post on how to map a specific port of virtual switch to a specific VLAN. First retrieve the configuration of the vswitch.

[root@hpvmhost] ~ # hpvmnet -S devlan12
Name     Number State   Mode      NamePPA  MAC Address    IP Address
======== ====== ======= ========= ======== ============== ===============
devlan12      3 Up      Shared    lan4     0x000cfc0046b9 10.1.1.99    

[Port Configuration Details]
Port    Port         Port     Untagged Number of    Active VM
Number  State        Adaptor  VLANID   Reserved VMs
======= ============ ======== ======== ============ ============
1       Active       lan      none     1            oradev01
2       Active       lan      none     1            oradev02
3       Active       lan      none     1            oradev03
4       Active       lan      none     1            oradev04
5       Active       lan      none     1            nfstest01
6       Active       lan      none     1            linuxvm1
7       Active       lan      none     1            linuxvm2 

[root@hpvmhost] ~ #

We are going to map the port 5 to the VLAN 120 in order to isolate the traffic of that NFS server from the other virtual machines that aren’t on the same VLAN. Again the command to use is hpmvnet.

[root@hpvmhost] ~ # hpvmnet -S devlan12 -u portid:5:vlanid:120

If you display again the HPVM network configuration for the devlan12 vswitch the change will appear under the Untagged VLANID column.

[root@hpvmhost] ~ # hpvmnet -S devlan12
Name     Number State   Mode      NamePPA  MAC Address    IP Address
======== ====== ======= ========= ======== ============== ===============
devlan12      3 Up      Shared    lan4     0x000cfc0046b9 10.1.1.99    

[Port Configuration Details]
Port    Port         Port     Untagged Number of    Active VM
Number  State        Adaptor  VLANID   Reserved VMs
======= ============ ======== ======== ============ ============
1       Active       lan      none     1            oradev01
2       Active       lan      none     1            oradev02
3       Active       lan      none     1            oradev03
4       Active       lan      none     1            oradev04
5       Active       lan      120      1            nfstest01
6       Active       lan      none     1            linuxvm1
7       Active       lan      none     1            linuxvm2 

[root@hpvmhost] ~ #

Juanma.

HP Virtual Connect Domain Setup – Part 4: Server Profiles

January 7, 2011 7 comments

This is the fourth and last part of this series of posts about Virtual Connect, the first three were:

In this final post I will discuss Server Profiles, what are they and how to create. As in the rest of the series I’m using Virtual Connect 3.10.

So, what is a Server Profile? We can define a Virtual Connect server profile as a logical grouping of attributes related to server connectivity that can be assigned to a server blade. You can see it as the connectivity personality of the server.

The server profile includes:

  • MAC address.
  • Preboot Execution Environment (PXE) enablement.
  • Network connection setting for each NIC port and WWN.
  • SAN fabric connection.
  • SAN boot paramenter setting for each Fibre Channel HBA port.

Once the server profile is created you can apply it to any server within the VC Domain. There is a maximum of 64 fully populated VC Server Profiles in a VC Domain.

As we saw in the network and storage posts the VCM can be configured so that blade servers use their factory-default MACs/WWNs and serial numbers or Virtual Connect provided and administered ranges of MACs and WWNs. These MACs and WWNs will override the default MAC and WWN values when a server profile is applied to the server and appear to preboot environments and host operating systems as the hardware addresses.

When a server profile is assigned to a Device Bay the Virtual Connect Manager securely connects to the blade in the bay and configures the NIC ports with profile provided MAC addresses and PXE settings and the FC HBA ports with the appropriate WWNs and SAN boot settings. Additionally the VCM automatically connects the server to the specified networks and SAN fabrics.

This server profile can then be copied or reassigned to another server as needed without interrupting the server connectivity to the network and SAN.

Once a blade server has been assigned a server profile and as long as it remains in the same device it does not require further VC Manager configuration during server or enclosure power cycle. They boot and access the network and fabric as long as soon as the server and interconnect modules are ready. If a server is inserted into a device bay that has already been assigned a server profile VCM automatically updates the configuration of that server before it is allowed to power on and connect to the network.

If a blade server is moved from a Virtual Connect managed enclosure to a non VC managed one all the ports automatically returns to their original factory values and settings in order to prevent duplicate MAC and WWNs within the datacenter because a blade server redeployment.

In addition to the above information the following points must be considered when working with server profiles:

  • Blade server and card firmware revision must be at a revision that supports Virtual Connect profile assignment.
  • Before creating the first server profile select whether to use Virtual Connect administered MAC and WWN ranges or the local factory default values.
  • After an enclosure is imported into a VC Domain the blades will remain isolated from networks and SAN fabrics until a server profile is assigned.
  • When Virtual Connect administered MACs and/or WWNs or when changing Fibre Channel boot parameters the servers must be powered off in order to receive or relinquish a server profile.
  • Fibre Channel SAN connections will display in the profile server screen only if the VC-FC module in the enclosure managed by Virtual Connect. If there is no VC-FC module the FC option wouldn’t appear in the server profile screen until a module has been added.
  • Some server profile SAN boor settings, like the controller boot order, are applied only after the server has been booted with the final mezzanine card configuration.
  • If PXE or SAN boot settings are made outside of Virtual Connect, the settings defined by the server profile will be restored after the blade server completes the next boot cycle.

If you have worked in the past with the 2.x Virtual Connect Manager revisions I’m sure that you will remember the Server Profile Wizard. That wizard has been removed from the  3.x revisions of VCM.

To start the server profile creation you have now to go to the Virtual Connect Home and in the Server area click on Define Server Profile.

In the Define Server Profile screen first enter the name of the profile, ESX01 in the example, and choose if you want to use factory default MAC and WWN or the VC-predefined.

Then move to Ethernet Network Connections. Here you can select the networks to assign to the ports, the port speed between AUTO, PREFERRED and CUSTOM and the PXE settings (ENABLED, DISABLED or USE-BIOS). By default there are only two connections created, to add more connections just right-click the area and choose Add connection.

In Network Name if you choose Multiple Networks a new icon will appear that will allow you to edit this connection type. Click and a new section will show up, this section allows to select the Shared Uplink Set and the networks. There is also a checkbox to set if you want to force the same VLAN mappings as the Shared Uplink Set to the different networks.

The next area is the FC SAN Connections. Assign the modules in the bays to the correspondent fabric and set the port speed.

Also in this section you can define the SAN boot parameters, click on the checkbox, the page will dim and a pop-up will appear.There you can configure each FC connection as PRIMARY, SECONDARY, DISABLED or USE-BIOS and set the Target Port Name and the LUN.

Finally we can assign the profile to a server bay.

Click Apply and the new server profile will be done. You can always edit the existent server profiles from the Server Profiles screen in the VC administration interface.

And this is the end. This series is done, if you have follow the correct steps outlined in the four posts you will have a fully operation Virtual Connect Domain. Of course there are a some topics I’d like to write about like iSCSI, FlexFabric and the VCM command line but I believe it’s better to do it in their own dedicated posts, stay tuned :-)

Juanma.

HP Virtual Connect Domain Setup – Part 2: Network Setup

December 21, 2010 23 comments

In the first post of the series I introduced to you HP Virtual Connect and showed how to use the Domain Wizard Setup to initially configure a VC domain. In the following article I will outline the use of the Network Setup Wizard and explain Virtual Connect networking concepts.

Before we begin to setup the network it would be very useful to clarify the Virtual Connect port terminology.

  • External port – The Ethernet connectors SFP+ modules (either 1GB or 10GB), 10GBASE-CX4 and RJ-45 on the faceplate of the Ethernet module.
  • Stacking port – These are Ethernet external ports used to connect within a Virtual Connect Domain the VC Ethernet modules. The Ethernet modules automatically identify the stacking modules.
  • Uplink port – An external port configured within a Domain for use as a connection to the external networking equipment. These ports are defined within Virtual Connect by the enclosure name, interconnect bay that contains the module and the port number.
  • Uplink port set – A set of uplinks ports trunked together in order to provide improved throughput and availability.
  • Shared uplink port – This is an Ethernet uplink port that carries the traffic for multiple networks. The associated networks are mapped to a specific VLAN on the external connection, the appropiate VLAN tags are removed or added as Ethernet packets enter or leave the VC Domain.
  • Shared uplink port set – This is a group of Ethernet uplinks trunked to provide improved throughput and availability to VC Shared Uplink Set.

The Virtual Connect Network Setup Wizard will help to establish external Ethernet connectivity for the enclosure. With this wizard you will be able to:

  • Identify the MAC addresses to be used by the servers within the VC Domain.
  • Configure Server VLAN tagging.
  • Set up connections from the c-Class enclosure to the external networks.

The network connections can be:

  • Dedicated uplink to a specific Ethernet network.
  • Shared uplink sets.

The first screen of the wizard is the MAC Address Settings. As every server in the market the HP Blades come with factory-default MAC addresses already assigned to their network cards. However Virtual Connect can override these values while the server remains in the enclosure.

Virtual Connect access the NICs through the Onboard Administrator and the server iLO to manage the MAC addresses. It provides 64 predefined and reserved MAC address ranges. The wizard will give you the option to use either an HP predefined range or an user defined one. HP recommends to use the predefined ranges.

Once you have chosen the address range and click next the wizard will ask for confirmation before continue.

The next screen is Server VLAN Tagging Support. Here the wizard gives you two possible options:

  • Tunnel VLAN Tags
  • Map VLAN Tags

The first one, Tunnel VLAN Tags, supports only VLAN tagging on networks with dedicated uplinks where all VLAN tags passed through the VC Domain without modification and ports connected to networks using shared uplinks can only send and receive untagged frames.

On the other hand Map VLAN Tags allow you to add more than one network to an Ethernet server port and specify the VLAN mapping between server tags and VC-Enet networks. Also, the VLAN tunneling will be disabled for VC Ethernet networks with dedicated uplinks.

There is also a checkbox in the page to , if this option is enabled the server ports connected to multiple VC Ethernet networks are forced to use the same VLAN mappings as those used for the corresponding Shared Uplink Set and the network connections can only be selected from a single Shared Uplink Set. When this option is not checked server network connections can be selected from any VC network and the external VLAN ID mappings can be manually edited. In the example of the screenshots I decided to check it.

Below are another two optional settings for link speed control when using mapped VLAN tags. These settings are:

  • Set a Custom value for Preferred Link Connection Speed. This value applies to server profiles with a Multiple Networks connection defined and the Port Speed Setting set to Preferred.
  • Set a Custom value for Maximum Link Connection Speed. This value limits the maximum port speed for multi-network connections when a Custom port speed is specified.

In our example we’re not going to check neither of them . Click next to move into the Define Network Connection screen.

Choose the network type you want to define and click next. I choose .

The Define Single Network shows up. First define the network name (prod_net_01 in my example). There are three configurable values.

  • Smart Link – With this option enabled Virtual Connect will drop the Ethernet link on every server connected to that network if the link to the external switches is lost.
  • Private Network – This option is intended to provide extra network security by isolating all server ports from each other within the VC Domain. All packets will be sent through the VC Domain and out the uplinks ports so the communication between the severs will go through an external L3 router that will redirect the traffic back to the Domain.
  • Enable VLAN Tunneling.

Click in the Advanced button to configure Advanced Network Settings. Set the network link speeds that best suites your configuration.

Again from the Define Single Network page we are going to assign a port to our network. Click on Add Port and select an uplink port.

Set the Connection Mode to Auto if the ports are trunked and to Failver if not.

Click Apply and move onto the next screen. From this screen you can create as many additional networks as you need.

Now we are going to create a network using VLAN tagging. Click Next an move again into the Define Network Connection page, select Connection with uplink(s) carrying multiple networks (using VLAN tagging) and click Next. The Define Shared Uplink Port Set page will be displayed.

A shared uplink is the way Virtual Connect has to identify which uplinks carry multiple networks over the same cable. On shared uplinks the VLAN tags are added when packets leave the enclosure and added when leave. The external switch and the Virtual Connect Manager must be configured with the same VLAN tag ID for each network  on the shared uplinks. The uplinks enables multiple ports to be added in order to support port aggregation and link failover, with a consistent set of VLAN tags. Virtual Connect has no restriction on which VLAN IDs can be used so the VLANs already used in the external infrastructure can be used here.

Since the VLAN tags are removed or added as soon as the packect enter or leave VC Ethernet Module shared uplink they have no relevance after the packet enter the enclosure. By identifying an associated network as the native VLAN will cause all untagged incoming packets to be placed onto this network, just one network can be designated as the native VLAN.

To finish the network creation assign a name (up to 64 characters with no spaces), add a port using the drop-down menu like in the single network process described above and add the networks you want to associate to the uplink. Finally click Apply.

In the final screen you will see now the three networks associated to a Shared Uplink Set. You can check this also from the Virtual Connect Manager page in the Ethernet Networks area.

And we are done with the Network Setup, in the next post I will show the storage part. As always any feedback would be welcome :-)

Juanma.

HP Virtual Connect Domain Setup – Part 1: Domain Setup Wizard

December 20, 2010 10 comments

A friend asked me last week if I could produce a document for him explaining the initial basic setup of Virtual Connect, I decided that instead of  that it would be better and more helpful to write in a series of blog posts, here it is the first of them for you to enjoy.

Virtual Connect is a technology developed by Hewlett-Packard for the HP BladeSystem c-Class enclosures. Provides server-edge and I/O virtualization in order to simplify the setup, maintenance and administration of server connections. It comprises a set of interconnect modules, both Ethernet and Fibre Channel, and a software known as Virtual Connect Manager.

Virtual Connect Manager, or VCM, is the single point administration interface for Virtual Connect. Under the hoods VCM is a software embedded into the VC Ethernet module, it can be accessed through a web-based interface or command line either with a serial connection to the Ethernet module or through a SSH connection to the module.

From the VCM only a single domain, with up to four enclosures, can be managed.

For large-scale infrastructures there is a more scalable version of VCM known as Virtual Connect Enterprise Manager, or VCEM. Unlike VCM, Virtual Connect Enterprise Manager is not embedded into the VC-Enet module, is a separate software that must be installed in another server. VCEM extends the VC management capabilities up to 250 domains and hundreds of blade servers.

Current series of articles will focus only on the Virtual Connect Manager GUI. Please take into account that I’m using Virtual Connect 3.10 version in the whole series and there some differences with the VC 2.x revisions.

When you login into the VCM for the first time a series of wizards will show up to help you with the initial setup of the domain. This article will cover the first of those wizards, the Domain Setup Wizard.

This wizard will allow you to:

  • Import enclosure configuration and communication settings
  • Name the domain
  • Set the IP address of the Virtual Connect Manager
  • Set up the local user accounts and its permissions and privileges
  • Confirm that the stacking links provide connectivity and redundancy

After the informative screen the first step will display. Here you have to provide the enclosure Onboard Administrator IP address and credentials, these credentials must have administrative level. Click next when finish.

Now VC Domain Wizard will import all the servers and VC interconnect modules within the enclosure.

In the next screen select the enclosure to import and click next.

A pop-up will show up to inform that the networking of all the blades within the enclosure will be disabled until VC Networking is properly configured. of course it will ask for confirmation.

After finishing the import the wizard will go the General Settings part. The Domain Setup Wizard automatically assigns a domain name based on the enclosure name, you can change the name when running the setup wizard or at any time later from the Domain Settings screen. The Virtual Connect domain name should be unique and can be up to 31 characters without spaces or special characters.

Next step is to configure the local user accounts.

By default the only local account is Administrator, this account cannot be deleted nor have domain privileges removed. You can also add up to 32 accounts with a combination of up to four levels of access. The available levels are:

  • Virtual Connect Domain
  • Server
  • Networking
  • Storage

There is also an Advanced area for each account where you can set Strong Passwords requirement and the minimum password length.

With this the Domain Setup Wizard is done. In the next article I will write about the network setup of the enclosure using the Network Setup Wizard.

Juanma.

OpenBSD network interface trunking

August 11, 2010 2 comments

Network interface trunking is the OpenBSD equivalent of HP-UX Auto-Port Aggregation feature. It allows to combine two or more physical interfaces into a virtual one that will send the outgoing traffic through the physical ports with an algorithm that depends on the trunking protocol configured.

The trunk driver has been available since OpenBSD 3.8, then it only supported the roundrobin protocol, and in the current version, OpenBSD 4.7,  it supports the following protocols:

  • Broadcast: Sends frames to all ports of teh trunk and equally receives frames from any port.
  • Roundrobin: Distributes outgoing traffic through all active ports and accepts incoming traffic from any active port.
  • Failover: Sends and receives traffic only through the master port. If the master port becomes unavailable, the next active port is used. The first interface added is the master port; any interfaces added after that are used as failover devices.
  • Loadbalance: The Loadbalance protocol balances the outgoing traffic across the active ports based on hashed protocol header information and accepts incoming traffic from any active port. The hash includes the Ethernet source and destination address, and, if available, the VLAN tag, and the IP source and destination address.
  • LACP: Used to provide redundancy and increase link speed, it uses the IEEE 802.3ad Link Aggregation Control Protocol (LACP) and the Marketer Protocol. It requires a LACP capable switch.
  • None: This protocol disables any traffic without disabling the trunk interface itself.

Lets create a failover trunk interface as an example. First we are going to activate the physical interfaces and add them to the trunk0 interface.

[obsd47]/# ifconfig em1 up
[obsd47]/# ifconfig em2 up
[obsd47]/# ifconfig trunk0 trunkport em1
[obsd47]/# ifconfig trunk0 trunkport em2
[obsd47]/# ifconfig trunk0
trunk0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0c:29:24:b7:c6
        priority: 0
        trunk: trunkproto roundrobin
                trunkport em2 active
                trunkport em1 master,active
        groups: trunk
        media: Ethernet autoselect
        status: active
[obsd47]/#

Secondly configure the trunking protocol and the IP address of the interface.

[obsd47]/# ifconfig trunk0 trunkproto failover 192.168.126.5 netmask 255.255.255.0 up [obsd47]/# ifconfig lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33200 priority: 0 groups: lo inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 lladdr 00:0c:29:24:b7:bc priority: 0 media: Ethernet autoselect (1000baseT full-duplex,master) status: active inet 192.168.126.4 netmask 0xffffff00 broadcast 192.168.126.255 inet6 fe80::20c:29ff:fe24:b7bc%em0 prefixlen 64 scopeid 0x1 em1: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500 lladdr 00:0c:29:24:b7:c6 priority: 0 trunk: trunkdev trunk0 media: Ethernet autoselect (1000baseT full-duplex,master) status: active inet6 fe80::20c:29ff:fe24:b7c6%em1 prefixlen 64 scopeid 0x2 em2: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500 lladdr 00:0c:29:24:b7:c6 priority: 0 trunk: trunkdev trunk0 media: Ethernet autoselect (1000baseT full-duplex,master) status: active inet6 fe80::20c:29ff:fe24:b7d0%em2 prefixlen 64 scopeid 0x3 enc0: flags=0<> mtu 1536 priority: 0 trunk0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 lladdr 00:0c:29:24:b7:c6 priority: 0 trunk: trunkproto failover trunkport em2 trunkport em1 master,active groups: trunk media: Ethernet autoselect status: active inet 192.168.126.5 netmask 0xffffff00 broadcast 192.168.126.255 inet6 fe80::20c:29ff:fe24:b7c6%trunk0 prefixlen 64 scopeid 0x6 pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33200 priority: 0 groups: pflog [obsd47]/#

At this point we have a configured trunk interface in failover, but of course we want to make these changes persistent through a reboot of the server. We need to create a configuration file for each of the physical interfaces and another one for the trunk interface.

[obsd47]/# echo "up" >hostname.em1
[obsd47]/# echo "up" >hostname.em2
[obsd47]/# echo "trunkproto failover trunkport em1 trunkport em2 192.168.126.5 netmask 255.255.255.0" > hostname.trunk0
[obsd47]/#
[obsd47]/# cat hostname.trunk0
trunkproto failover trunkport em1 trunkport em2 192.168.126.5 netmask 255.255.255.0
[obsd47]/#

Now reboot and check that everything went well and the trunk0 interface is up and running. Of course the same procedure can be used to create a trunk interface for any of the supported protocols.

Juanma.

Categories: BSD Tags: , , ,

Configure AVIO Lan in HPVM Linux guests

The  AVIO Lan drivers for Linux HPVM guests are supported since HPVM4.0 but as you will see enabling it is a little more complicated than in HP-UX guests.

The first prerequisite is to have installed the HPVM management software, once you have this package installed look for a RPM package called hpvm_lgssn in /opt/hpvm/guest-images/linux/DRIVERS.

root@hpvm-host:/opt/hpvm/guest-images/linux/DRIVERS # ll
total 584
 0 drwxr-xr-x 2 bin bin     96 Apr 13 18:47 ./
 0 drwxr-xr-x 5 bin bin     96 Apr 13 18:48 ../
 8 -r--r--r-- 1 bin bin   7020 Mar 27  2009 README
576 -rw-r--r-- 1 bin bin 587294 Mar 27  2009 hpvm_lgssn-4.1.0-3.ia64.rpm
root@hpvm-host:/opt/hpvm/guest-images/linux/DRIVERS #

Copy the package to the virtual machine with your favorite method and install it.

[sles10]:/var/tmp # rpm -ivh hpvm_lgssn-4.1.0-3.ia64.rpm
Preparing...                ########################################### [100%]
Installing...               ########################################### [100%]

[sles10]:/var/tmp #

Check the installation of the package.

[sles10]:~ # rpm -qa | grep hpvm
hpvm-4.1.0-1
hpvmprovider-4.1.0-1
hpvm_lgssn-4.1.0-3
[sles10]:~ #
[sles10]:~ # rpm -ql hpvm_lgssn
/opt/hpvm_drivers
/opt/hpvm_drivers/lgssn
/opt/hpvm_drivers/lgssn/LICENSE
/opt/hpvm_drivers/lgssn/Makefile
/opt/hpvm_drivers/lgssn/README
/opt/hpvm_drivers/lgssn/hpvm_guest.h
/opt/hpvm_drivers/lgssn/lgssn.h
/opt/hpvm_drivers/lgssn/lgssn_ethtool.c
/opt/hpvm_drivers/lgssn/lgssn_main.c
/opt/hpvm_drivers/lgssn/lgssn_recv.c
/opt/hpvm_drivers/lgssn/lgssn_recv.h
/opt/hpvm_drivers/lgssn/lgssn_send.c
/opt/hpvm_drivers/lgssn/lgssn_send.h
/opt/hpvm_drivers/lgssn/lgssn_trace.h
/opt/hpvm_drivers/lgssn/rh4
/opt/hpvm_drivers/lgssn/rh4/u5
/opt/hpvm_drivers/lgssn/rh4/u5/lgssn.ko
/opt/hpvm_drivers/lgssn/rh4/u6
/opt/hpvm_drivers/lgssn/rh4/u6/lgssn.ko
/opt/hpvm_drivers/lgssn/sles10
/opt/hpvm_drivers/lgssn/sles10/SP1
/opt/hpvm_drivers/lgssn/sles10/SP1/lgssn.ko
/opt/hpvm_drivers/lgssn/sles10/SP2
/opt/hpvm_drivers/lgssn/sles10/SP2/lgssn.ko
[sles10]:~ #

There are two ways to install the driver, compile it or use one of the pre-compiled modules. These pre-compiled modules are for the following distributions and kernels:

  • Red Hat 4 release 5 (2.6.9-55.EL)
  • Red Hat 4 release 6 (2.6.9-67.EL)
  • SLES10 SP1 (2.6.16.46-0.12)
  • SLES10 SP2 (2.6.16.60-0.21)

For other kernels you must compile the driver. In the Linux box of the example I had a supported kernels and distro (SLES10 SP2) but instead of using the pre-compiled one I decided to go through the whole process.

Go the path /opt/hpvm_drivers/lgssn, there you will find the sources of the driver. To compile and install execute a simple make install.

[sles10]:/opt/hpvm_drivers/lgssn # make install
make -C /lib/modules/2.6.16.60-0.21-default/build SUBDIRS=/opt/hpvm_drivers/lgssn modules
make[1]: Entering directory `/usr/src/linux-2.6.16.60-0.21-obj/ia64/default'
make -C ../../../linux-2.6.16.60-0.21 O=../linux-2.6.16.60-0.21-obj/ia64/default modules
 CC [M]  /opt/hpvm_drivers/lgssn/lgssn_main.o
 CC [M]  /opt/hpvm_drivers/lgssn/lgssn_send.o
 CC [M]  /opt/hpvm_drivers/lgssn/lgssn_recv.o
 CC [M]  /opt/hpvm_drivers/lgssn/lgssn_ethtool.o
 LD [M]  /opt/hpvm_drivers/lgssn/lgssn.o
 Building modules, stage 2.
 MODPOST
 CC      /opt/hpvm_drivers/lgssn/lgssn.mod.o
 LD [M]  /opt/hpvm_drivers/lgssn/lgssn.ko
make[1]: Leaving directory `/usr/src/linux-2.6.16.60-0.21-obj/ia64/default'
find /lib/modules/2.6.16.60-0.21-default -name lgssn.ko -exec rm -f {} \; || true
find /lib/modules/2.6.16.60-0.21-default -name lgssn.ko.gz -exec rm -f {} \; || true
install -D -m 644 lgssn.ko /lib/modules/2.6.16.60-0.21-default/kernel/drivers/net/lgssn/lgssn.ko
/sbin/depmod -a || true
[sles10]:/opt/hpvm_drivers/lgssn #

This will copy the driver to /lib/module/<KERNEL_VERSION>/kernel/drivers/net/lgssn/.

To ensure that the new driver will loaded during the startup of the operative system first add the following line to /etc/modprobe.conf, one line for each interface configured for AVIO Lan.

alias eth1 lgssn

The HPVM 4.2 manual said you have to issue the command depmod -a in order to inform the kernel about the change but if you look the above log will see that the last command executed by the make install is a depmod -a. Look into the modules.dep file to check that the corresponding line for the lgssn driver has been added.

[sles10]:~ # grep lgssn /lib/modules/2.6.16.60-0.21-default/modules.dep
/lib/modules/2.6.16.60-0.21-default/kernel/drivers/net/lgssn/lgssn.ko:
[sles10]:~ #

At this point and if you have previously reconfigured the virtual machine, load the module and restart the network services.

[sles10]:/opt/hpvm_drivers/lgssn # insmod /lib/modules/2.6.16.60-0.21-default/kernel/drivers/net/lgssn/lgssn.ko
[sles10]:/opt/hpvm_drivers/lgssn # lsmod |grep lgssn
lgssn                 576136  0
[sles10]:/opt/hpvm_drivers/lgssn #
[sles10]:/opt/hpvm_drivers/lgssn # service network restart
Shutting down network interfaces:
    eth0      device: Intel Corporation 82540EM Gigabit Ethernet Controller
    eth0      configuration: eth-id-2a:87:14:5c:f9:ed
    eth0                                                              done
    eth1      device: Hewlett-Packard Company Unknown device 1338
    eth1      configuration: eth-id-66:f3:f8:4e:37:d5
    eth1                                                              done
    eth2      device: Intel Corporation 82540EM Gigabit Ethernet Controller
    eth2      configuration: eth-id-0a:dc:fd:cb:2c:62
    eth2                                                              done
Shutting down service network  .  .  .  .  .  .  .  .  .  .  .  .  .  done
Hint: you may set mandatory devices in /etc/sysconfig/network/config
Setting up network interfaces:
    lo        
    lo       
              IP address: 127.0.0.1/8   
              IP address: 127.0.0.2/8   
Checking for network time protocol daemon (NTPD):                     running
    lo                                                                done
    eth0      device: Intel Corporation 82540EM Gigabit Ethernet Controller
    eth0      configuration: eth-id-2a:87:14:5c:f9:ed
Warning: Could not set up default route via interface
 Command ip route replace to default via 10.31.12.1 returned:
 . RTNETLINK answers: Network is unreachable
 Configuration line: default 10.31.12.1 - -
 This needs NOT to be AN ERROR if you set up multiple interfaces.
 See man 5 routes how to avoid this warning.

Checking for network time protocol daemon (NTPD):                     running
    eth0                                                              done
    eth1      device: Hewlett-Packard Company Unknown device 1338
    eth1      configuration: eth-id-66:f3:f8:4e:37:d5
    eth1      IP address: 10.31.4.16/24   
Warning: Could not set up default route via interface
 Command ip route replace to default via 10.31.12.1 returned:
 . RTNETLINK answers: Network is unreachable
 Configuration line: default 10.31.12.1 - -
 This needs NOT to be AN ERROR if you set up multiple interfaces.
 See man 5 routes how to avoid this warning.

Checking for network time protocol daemon (NTPD):                     running
    eth1                                                              done
    eth2      device: Intel Corporation 82540EM Gigabit Ethernet Controller
    eth2      configuration: eth-id-0a:dc:fd:cb:2c:62
    eth2      IP address: 10.31.12.11/24   
Checking for network time protocol daemon (NTPD):                     running
    eth2                                                              done
Setting up service network  .  .  .  .  .  .  .  .  .  .  .  .  .  .  done
[sles10]:/opt/hpvm_drivers/lgssn #

If you have not configured the networking interface of the virtual machine shutdown the virtual machine and from the host modify each virtual NIC of the guest. Take into account that AVIO Lan drivers are not supported with localnet virtual switches.

root@hpvm-host:~ # hpvmmodify -P sles10 -m network:avio_lan:0,2:vswitch:vlan2:portid:4
root@hpvm-host:~ # hpvmstatus -P sles10 -d
[Virtual Machine Devices]
...
[Network Interface Details]
network:lan:0,0,0x2A87145CF9ED:vswitch:localnet:portid:4
network:avio_lan:0,1,0x66F3F84E37D5:vswitch:vlan1:portid:4
network:avio_lan:0,2,0x0ADCFDCB2C62:vswitch:vlan2:portid:4
...
root@hpvm-host:~ #

Finally start the virtual machine and check that everything went well and the drivers have been loaded.

Juanma

Moving vNICs between vSwitches

March 3, 2010 1 comment

Following with my re-learning HPVM process today I’ve been playing around with my virtual switches and a question had arise.

How can I move a vNic from one vSwitch to another?

I discovered is not a difficult task, just one important question to take into account, the virtual machine must be powered off. This kind of changes can’t be done if the IVM is online, at least with HPVM 3.5. I never used 4.0 or 4.1 releases of HPVM and I didn’t find anything in the documentation that suggest a different behavior.

To perform the operation we’re going to use, as usual ;-),  hpvmmodify. It comes with the -m switch to modify the I/O resources of an already existing virtual machine, but you have to specify the hardware address of the device. To identify the address of the network card  launch hpvmstatus with -d, this options shows the output with the format used on the command line.

[root@hpvmhost] ~ # hpvmstatus -P ivm1 -d
[Virtual Machine Devices]
...
[Network Interface Details]
network:lan:0,0,0x56E9E3096A22:vswitch:vlan02
network:lan:0,1,0xAED6F7FA4E3E:vswitch:localnet
...
[root@hpvmhost] ~ #

As it can be seen in the Networking Interface Details the third field shows, separated by commas,  the lan bus, the device number and the MAC address of the vNic. We only need the first two values, that is the lan bus and device number, “0,0″ in our the example.

Now we can proceed.

[root@hpvmhost] ~ # hpvmmodify -P ivm2 -m network:lan:0,0:vswitch:vlan03   
[root@hpvmhost] ~ #
[root@hpvmhost] ~ # hpvmstatus -P ivm1
[Virtual Machine Details]
Virtual Machine Name VM #  OS Type State
==================== ===== ======= ========
ivm1                     9 HPUX    On (OS)   
...
[Network Interface Details]
Interface Adaptor    Name/Num   PortNum Bus Dev Ftn Mac Address
========= ========== ========== ======= === === === =================
vswitch   lan        vlan03     9         0   0   0 56-e9-e3-09-6a-22
vswitch   lan        localnet   9         0   1   0 ae-d6-f7-fa-4e-3e
...
[root@hpvmhost] ~ #

And we are done.

I will write a few additional posts covering  more HPVM tips, small ones and big ones, at the same time I’m practicing them on my lab server.

Juanma.

Playing with lanadmin & lanscan

February 9, 2010 3 comments

Current release of HP-UX, 11.31, has the handy nwmgr to handle networking tasks, but for years instead of nwmgr we’ve been playing with lanscan and lanadmin (linkloop as well) to perform many networking tasks on the 11.23 release and previous ones. And surely some of you, just like myself, still have 11iv2 systems up and running. Following is a small list of tips and tasks for lanadmin from one of my “how-to-do” files.

  • Lanscan: Basically is used to get information about the LAN interfaces.
root@sap01:~# lanscan
Hardware Station        Crd Hdw   Net-Interface  NM  MAC       HP-DLPI DLPI
Path     Address        In# State NamePPA        ID  Type      Support Mjr#
0/0/0/1/0 0x001A4B07F002 0   UP    lan0 snap0     1   ETHER     Yes     119
0/0/0/1/1 0x001A4B07F003 1   UP    lan1 snap1     2   ETHER     Yes     119
0/0/1/1/0 0x0018FE2D7EE7 2   UP    lan2 snap2     3   ETHER     Yes     119
0/0/9/1/0 0x0018FE2D7EF4 3   UP    lan3 snap3     4   ETHER     Yes     119
0/0/10/1/0 0x000CFC0046B9 4   UP    lan4 snap4     5   ETHER     Yes     119
0/0/12/1/0 0x000CFC004672 5   UP    lan5 snap5     6   ETHER     Yes     119
LinkAgg0 0x000000000000 900 DOWN  lan900 snap900 9   ETHER     Yes     119
LinkAgg1 0x000000000000 901 DOWN  lan901 snap901 10  ETHER     Yes     119
LinkAgg2 0x000000000000 902 DOWN  lan902 snap902 11  ETHER     Yes     119
LinkAgg3 0x000000000000 903 DOWN  lan903 snap903 12  ETHER     Yes     119
LinkAgg4 0x000000000000 904 DOWN  lan904 snap904 13  ETHER     Yes     119
root@sap01:~#

A verbose version can be obtained with the -v switch, but for me this switch has a glitch since you can’t query for a single LAN card:

root@sap01:~# lanscan -v
-------------------------------------------------------------------------------
Hardware Station        Crd Hdw   Net-Interface  NM  MAC       HP-DLPI DLPI
Path     Address        In# State NamePPA        ID  Type      Support Mjr#
0/0/0/1/0 0x001A4B07F002 0   UP    lan0 snap0     1   ETHER     Yes     119

Extended Station                           LLC Encapsulation
Address                                    Methods
0x001A4B07F002                             IEEE HPEXTIEEE SNAP ETHER NOVELL 

Driver Specific Information
iether
-------------------------------------------------------------------------------
...
root@sap01:~#

There are other options for lanscan that can be used to obtain more simple info in a “script friendly” list format:

root@sap01:~# lanscan -a
0x001A4B07F002
0x001A4B07F003
0x0018FE2D7EE7
0x0018FE2D7EF4
0x000CFC0046B9
0x000CFC004672
0x000000000000
0x000000000000
0x000000000000
0x000000000000
0x000000000000
root@sap01:~# lanscan -i
lan0 snap0
lan1 snap1
lan2 snap2
lan3 snap3
lan4 snap4
lan5 snap5
lan900 snap900
lan901 snap901
lan902 snap902
lan903 snap903
lan904 snap904
root@sap01:~#
  • Lanadmin: The lanadmin, acording to its man page, allow you to:

+  Display and change the station address.
+  Display and change the 802.5 Source Routing options (RIF).
+  Display and change the maximum transmission unit (MTU).
+  Display and change the speed setting.
+  Clear the network statistics registers to zero.
+  Display the interface statistics.
+  Display the interface usage information.
+  Reset the interface card, thus executing its self-test.
+  Configure VLANs on the cards that support VLAN.

It can be used in two ways, if invoked with no options from the shell it will present a menu style interface where different tasks can be performed. Following is am example to illustrate.

          LOCAL AREA NETWORK ONLINE ADMINISTRATION, Version 1.0
                       Tue , Feb 9,2010  14:22:27

               Copyright 1994 Hewlett Packard Company.
                       All rights are reserved.

Test Selection mode.

        lan      = LAN Interface Administration
        menu     = Display this menu
        quit     = Terminate the Administration
        terse    = Do not display command menu
        verbose  = Display command menu

Enter command: lan

LAN Interface test mode. LAN Interface PPA Number = 0

        clear    = Clear statistics registers
        display  = Display LAN Interface status and statistics registers
        end      = End LAN Interface Administration, return to Test Selection
        menu     = Display this menu
        ppa      = PPA Number of the LAN Interface
        quit     = Terminate the Administration, return to shell
        reset    = Reset LAN Interface to execute its selftest
        specific = Go to Driver specific menu

Enter command: display

                      LAN INTERFACE STATUS DISPLAY
                       Tue , Feb 9,2010  14:22:31

PPA Number                      = 0
Description                     = lan0 HP PCI Core I/O 1000Base-T Release B.11.23.0712.01
Type (value)                    = ethernet-csmacd(6)
MTU Size                        = 1500
Speed                           = 1000000000
Station Address                 = 0x14c2650091
Administration Status (value)   = up(1)
Operation Status (value)        = up(1)
Last Change                     = 419
Inbound Octets                  = 120454615
Inbound Unicast Packets         = 863761
Inbound Non-Unicast Packets     = 4327
Inbound Discards                = 0
Inbound Errors                  = 0
Inbound Unknown Protocols       = 12
Outbound Octets                 = 145033817
Outbound Unicast Packets        = 1285500
Outbound Non-Unicast Packets    = 221
Outbound Discards               = 0
Outbound Errors                 = 0
Outbound Queue Length           = 0
Specific                        = 655367

Press  to continue

Ethernet-like Statistics Group

Index                           = 1
Alignment Errors                = 0
FCS Errors                      = 0
Single Collision Frames         = 0
Multiple Collision Frames       = 0
Deferred Transmissions          = 0
Late Collisions                 = 0
Excessive Collisions            = 0
Internal MAC Transmit Errors    = 0
Carrier Sense Errors            = 0
Frames Too Long                 = 0
Internal MAC Receive Errors     = 0

LAN Interface test mode. LAN Interface PPA Number = 0

        clear    = Clear statistics registers
        display  = Display LAN Interface status and statistics registers
        end      = End LAN Interface Administration, return to Test Selection
        menu     = Display this menu
        ppa      = PPA Number of the LAN Interface
        quit     = Terminate the Administration, return to shell
        reset    = Reset LAN Interface to execute its selftest
        specific = Go to Driver specific menu

Enter command:

When used with options from the command line lanadmin can perform the same tasks as as in the menu interface on each LAN card. Here are some of the most common features I’ve been using for years:

· Display interface info:

root@sap01:~# lanadmin -x card_info  1
*********** Version Information **********
Driver version: B.11.23.0712
Firmware version: N/A
Chip version: 0x3
PCI Sub-System ID: 0x12a6
PCI Sub-Vendor ID: 0x103c
Board Revision: D4503807
Software Key: 0
Engineering Date Code: A-4731

********** Card Setting ***********
Driver State: IETHER_ONLINE
Auto Negotiation: On
Flow Control: On
Send Max Buf Descriptors: 1
Recv Max Buf Descriptors: 1
Send Coalesced Ticks: 150
Recv Coalesced Ticks: 0
root@sap01:~#

· Display Auto-Port Aggregation status:

root@sap01:~# lanadmin -x -v 900
Link Aggregate PPA #      : 900
Number of Ports           : 2
Ports PPA                 : 0 1
Link Aggregation State    : LINKAGG MANUAL
Load Balance Mode         : Hot Standby (LB_HOT_STANDBY)

root@sap01:~#

· Show speed settings:

root@sap01:/# lanadmin -x 1
Speed = 1000 Full-Duplex.
Autonegotiation = On.

root@sap01:/#

· Creating an Aggregation link:

roo@sap01:/# lanadmin -X -a 1 2 900

· Show load balancing algorithm in APA:

root@sap02:/etc# lanadmin -x -l 900
Load Balancing = Hot Standby (LB_HOT_STANDBY)
root@sap02:/etc#

· Display MAC address:

root@sap01:~# lanadmin -a 1  
Station Address                 = 0x001a4b07f003
root@sap01:~#

· Display driver and adapter statistics:

root@sap01:/# lanadmin -x stats drv 1
****** Driver Statistics ******        
In Packet Error                                        0
Out Packet Error                                       0
Loopback packets                                      44
Link Down events                                       0

****** Host Command Statistics ******  
nicCmdsDelMCastAddr                                    0
nicCmdsSetMACAddr                                      0
nicCmdsSetPromiscMode                                  0
nicCmdsSetMulticastMode                                0
nicCmdsClearStats                                      1

****** NIC Events Statistics ******    
nicEventsFirmwareOperational                           0
nicEventsStatsUpdated                                  0
nicEventsLinkStateChanged                              1
nicEventsMCastListUpdated                              0

****** Interface Statistics ******     
ifIndex                                                2
ifType                                                 6
ifMtu                                               1500
ifSpeed                                       1000000000
ifAdminStatus                                          1
ifOperStatus                                           1
ifLastChange                                          36
ifInDiscards                                           0
ifInErrors                                             0
ifInUnknownProtos                                  87777
ifOutDiscards                                          0
ifOutErrors                                            0
ifOutQLen                                              0
ifInOctets_low                                1205643914
ifInOctets_high                                        0
ifInUcastPkts_low                                8695826
ifInUcastPkts_high                                     0
ifInMulticastPkts_low                                  0
ifInMulticastPkts_high                                 0
ifInBroadcastPkts_low                              87777
ifInBroadcastPkts_high                                 0
ifOutOctets_low                               1200310422
ifOutOctets_high                                       0
ifOutUcastPkts_low                               8696015
ifOutUcastPkts_high                                    0
ifOutMulticastPkts_low                                 0
ifOutMulticastPkts_high                                0
ifOutBroadcastPkts_low                                 0
ifOutBroadcastPkts_high                                0
root@sap01:/#

· Show Vital Product Data, a really funny name ;-) don’t you think?

root@sap01:/# lanadmin -x vpd 0
*********** Vital Product Data **********
Product Description: PCI/PCI-X 10/100/1000BT Dual Ethernet Adapter
Part Number: A7012-60601
Engineering Date Code: A-4731
Part Serial Number: 001A4B07F002
Misc. Information: 7.5W
Mfd. Date: 4749
Checksum: 0xb
EFI Version: 03048
ROM Firmware Version: N/A
Asset Tag: N/A
root@sap01:/#

· Show card type:

root@sap01:~# lanadmin -x type 1     
1000Base-T

root@sap01:~#

And we are finished. Probably I’m forgetting a more interesting uses of lanadmin if you have other everyday use please comment :-)

Juanma.

SSH foolishness

February 8, 2010 Leave a comment

Today it’s been one of those days that you wish to hide very deep under your desk.

For a couple of days I’ve been battling with a 11.31 server over its sshd configuration, somehow this was the only node which refused the public-key ssh authentication method from my management server and always asked for root password, that means I could’t run scripts remotely with a convenient for loop over the sever list, no remote tests, no cron tasks from the management node against it, etc; and that’s unacceptable for me.

I almost wiped out the config of the target server and recreated it from scratch using the sshd_config file of a working 11.31 node as my starting point and still the damn server asked for a password. I was desperate, looking through the config file one time after another, checked file permissions, running the same test from other servers with same result.

Finally I ask a colleague if he could review my sshd_config file, at first look he found nothing wrong and then he performed some test and MAGIC!! it worked. I asked him about his “guru trick” and he said ‘Dude, your root home had 777 permissions’ ‘WHAT?!?!?!?’

I was terribly embarrassed, one of the first tasks I do after install any HP-UX is to move the root home from / to /root and in this almost newly deployed 11.31 I didn’t change the permissions.  Of course I quickly checked all my servers just in case and fortunately for me none of them had a misconfigured sshd.

Two days completely lost finghting against my foolishness, got out of bed on the wrong side this morning.

Oh I almost forgot… thanks Javi, you’re the best :-)

Juanma.

Categories: HP-UX Tags: , , ,
Follow

Get every new post delivered to your Inbox.

Join 197 other followers

%d bloggers like this: