Objective: Having a baremetal install of Redhat Enterprise on a blade within the IBM Bladecenter, configure the OS to successfully mount a LUN on the SAN (Storwise v7000) as usable storage.
Hardware:
– Blades: IBM Bladecenter 88524TG
– SAN storage: FiberChannel Storwise V7000
– HBAs: Qlogic
Method:
I assume all cabling and SAN zoning has already been done in this article.
1. In this example we install redhat enterprise 5.8 directly on the blade unit (baremetal – not using a hypervisor such as VMWare). Simply download the appropriate ISO, boot from the CD and install RedHat as per usual installation procedure. Customize installation as appropriate for your enviornment
2. Reboot the blade. When asked, press F1 to enter BIOS setup. Login to the V7000 and add a new host (hosts > hosts > add new host > add port definition). Add the blade’s two HBAs WWN to the newly created host.
3. Back on the blade, exit the BIOS and continue booting the RedHat OS. The IBM storwise will mark the host as being “offline” but this is simply due to us not having yet presented any LUNs to the RedHat OS. (Thanks to Isaac Zarb for pointing this out)
4. Once Redhat is booted, make sure the Qlogic drivers (qla2xxx at the time of writing) are present and the QLogic HBAs are detected properly. You may use the following:
– lspci: within the output, you should see the QLogic HBAs listed. While they may appear here, this does not necessarily mean they can be used (i.e. functional drivers), but simply that the hardware has been correctly detected and identified by redhat
– dmesg | grep -i qla2xxx: this should return some output. If so, then the drivers and kernel module have been correctly loaded and the HBAs are ready to use. If no outpu is present here, search google using “qlogic linux super installer“. This should direct you the QLogic support site, which offers an archive with an all-in-one installer that automatically detects, installs and configures the appropriate drivers.
– lsmod | grep -i qla: like the above, this should also return some output meaning the modules have been correctly loaded. If not, install the drivers as mentioned in the previous point, and reboot the server.
5. Enable multipathing. At the time of writing, multipathing and persistent naming are properly implemented in the qlogic drivers. However, redhat by default disabled multipathing. In most IBM blade to storwise setups, there will be 4 paths to any given LUN on the SAN. So, multipathing should be enabled for increased throughput and tolerance. To do so, open your favored text editor and edit the /etc/multipath.conf file. By default, RedHat blacklists all devices from multipathing by using the following syntax:
blacklist {
devnode “*”
}
Change the above to:
blacklist {
devnode “sda”
}
Where “sda” is you local drive is such exists.
6. Ensure the multipath daemon is set to startup on every reboot. Use the chkconfig multipathd –level 3 on command for this (given that your normal runlevel is “3”)
7. Back on the storwise V7000, create a volume, and assign it to the redhat host which we created previously. The host will probably still be marked as “offline” but the IBM Storwise still allows you to map volumes to the offline host
8. Restart redhat by issuing a shutdown -r now. After the reboot, issue the command multipath -ll. This should show the 4 paths mentioned earlier. Change directory to /dev/mapper/ where you should see several device files named along the lines of /dev/mapper/multipath0p1. The number of device files present depends on the amount of volumes (LUNs) presented to the blade from the Storwise. Please note these device files will only be present if the directive user_friendly_names yes is present in the multipath.conf file.
9. Use the /dev/mapper/multipath0p1 or similar device files as you would any other harddisk device file. You can use familiar commands such as fdisk, mkfs, mount, du and so on.
Notes:
Since there a 4 multipaths to a given SAN volume, once all the above is done you can observe that other than sda (the local hardisk) there are another 4 logical drives eg: sdb, sdc, sdd,sde. All these represent a single LUN / volume on the SAN. If you use fdisk to create a partition on the /dev/mapper/multipath device file, this will be reflected in the above since each drive is assigned a partition number as well, i.e. sdb1, sdc1, sdd1, sde1. And similarly for all other device files.
References:
David Vassallo's Blog 