Linux LVM

From Tecniq

Warning, this page is a Work In Progress. As such, the information contained within this page is incomplete. Dylan Swift, Moderator, http://tecniq.info

Home · Help · Wikimedia Help

Linux LVM (Logical Volume Manager)

A module, or set of modules that allows for fine grained allocation of physical disk space to various parts of the file system. Also supports snapshots & mirroring

Contents 1 Overview 2 Terms 3 Concepts 4 Commands 5 Backup, Backup, Backup 6 How To 6.1 How to Scan 6.1.1 VMware disk 6.1.2 SAN 6.1.2.1 VMware 6.1.2.2 VM Guests 6.1.2.3 VM Guest OS 6.1.2.3.1 RedHat Enterprise Linux (RHEL) 6.2 How to Increase the Storage 6.2.1 Physical Volumes 6.2.1.1 Partitioning 6.2.1.2 LVM Physical Volume Creation 6.2.2 Volume Groups 6.2.3 Logical Volumes 6.3 How to Increase the Filesystem

Overview

Terms

• Disk Subsystem = Storage Array
• FS = File system
• Guest = VMware guest operating system
• LV = Logical Volume
• LVM = Logical Volume Manager
• PV = Physical Volume
• RHEL = RedHat Enterprise Linux - a popular north American commercial Linux distribution
• SAN = Storage Area Network
• VG = Volume Group
• VMware = Virtualisation Platform. Can be an application (VMware Server, VMware Workstation) or an OS (GSX, ESX Server).
• VMware Guest = a standard operating system (save for some minor driver files) running on virtual hardware. The OS should not be able to tell that it is running on virtual hardware (though there are ways for it to tell).
• VMware Host = a physical server running an instance of a VMware Server operating system, capable of providing a virtualised platform for a VMware Guest.

Concepts

Filesystem = OS organisation of disk (physical or logical) that allows for useful work to be performed LV = subset of a VG VG = collection of one or more PV's PV = Physical Volume - directly maps to one local disk, local disk partition, LUN, iSCSI LUN

<<<TBD>>>Insert a diagram of the LVM hierarchy<<<TBD>>>

Commands

Backup, Backup, Backup

Need I say any more?

Yes, oh well...

Don't forget to backup any important data on the drives / system you are about to make changes on. While all of the following command *can* be executed on a running system with no negative effects, it is always worth mitigating the effects of:

1. finger trouble (typing the wrong command)
2. bad code (the application / system commands are broken!)
3. co-incidental changes elsewhere on related systems (eg someone rebooting the SAN while you are working) that result in a partial or complete loss of data
4. conspiracy (in the best traditions of the BOFH, someone is out to get you!)

It is also worth noting that the manual pages for the following commands will also tell you to back up!

I have tested (almost) all of the following on live systems without any loss of data.

How To

How to Scan

VMware disk

VMware (ESX) disks are all presented to the guest as SCSI devices/LUNs regardless of whether they are real disks, container files emulating disks, fibre attached LUNs, NFS mounts or iSCSI LUNs. If you choose to add a new disk using space in the VMware file system then the only additional steps needed once you have added the disks are to force the guest operating system to recognise the new device. See the section 'VM Guests' <<<TBD>>>create link to relevant header section<<<TBD>>> (Note that for Windows users the disk management console should show you any new devices)

SAN

Creating a new LUN for use as a new disk by a VMware guest is much the same as creating a LUN for any other client, and instructions are specific to the disk subsystem and its tools. When mapping the LUN, make sure that the LUN is mapped to the correct VMware hosts / VMware cluster.

VMware

If you have created a new LUN on a SAN then you will need to get the VMware servers to 'recognise' the new LUN. This is a VMware administrators task, and is performed in the VMware server management console by selecting the relevant section under configuration and performing a re-scan operation.

To confirm the new LUN is seen by the VMware hosts (remember to re-scan all the hosts in a cluster - you don't want a VMware guest migrating to a VMware host that does not have access to all its disks) - by reading the LUN number. This number should be the same in the disk subsystem and will be the same when adding the disk to the VMware guest from the pool of available LUNs. Once added to the VMware guest hardware it will be given a new SCSI id which is the id seen by the guest operating system.

DISK Subsystem	->	VMware host	->	VM guest configuration	->	Guest Operating System
LUN x	->	LUN x	->	LUN x mapped as virtual node y	->	SCSI ID y

VM Guests

In VMware (again a VMware administration task), add a new disk to a VMware guest (this is the only type of hardware you can add to a running VMware guest in ESX 3.0 & 3.5). If this is a LUN, then choose Raw Device Mapping (RDM) and select the correct LUN ((IBM definitely, others unknown) note the LUN number will match the number used in the disk subsystem when the mapping was created). Once this configuration change has been made (observe the status window at the bottom of the management console for confirmation of if/when this is complete) you can move on to the next step.

VM Guest OS

RedHat Enterprise Linux (RHEL)

(CentOS, Fedora, WhiteBox, Scientific Linux, Oracle (Unbreakable) Linux, others derived from RHEL)

Scan the devices

one of the following 2 commands should work to instruct the kernel to re-scan the SCSI bus to identify any new devices:

1. echo "0 1 0 " > /sys/class/scsi_host/host0/scan

   where the 0 1 0 indicates that the LUN has been presented to the VMware guest as SCSI bus 0, Disk 1. The exact values can be determined from the virtual hardware settings for the VMware guest in the VMware management console.

2. echo "scsi add-single-device 0:0:1:0" > /proc/scsi/scsi

Now issue the command partprobe to force the OS to scan the new disk for any pre-existing partitions (this will ensure that you don't accidentally overwrite a LUN mounted on your VMware guest in error)

If you want to see the before and after change, then the following 2 commands will show up the differences if the new disk is recognised:

echo /proc/scsi/scsi

will show all the scsi devices attached to the system. and

ls -l /dev/sd*

will show all the disks and partitions recognised by the system.

How to Increase the Storage

Physical Volumes

You now have 2 choices, either add the whole disk as the LVM Physical Volume (PV), or add a partition of the disk (possibly a partition covering the whole disk) as the PV. I recommend the latter, as during the partitioning phase you can identify the partition as a LVM PV using the partition type field. This helps indicate later when remounting the partition, why it doesn't contain a file system & is not mountable directly.

Partitioning

Assuming that you are using fdisk to partition your disk, and that the disk is identified by your system as sdb (the second scsi device), then the following steps should allow you to create the necessary partition:

1. fdisk /dev/sdb

   this will launch the fdisk application. As with all usage of the fdisk command, I must remind you that any changes you make here will remove access to data you have on the disk you pass as the parameter to this command.
2. type n for new
3. type p for primary partition
4. type 1 for the first partition
5. accept the defaults for the start and end blocks (ie maximum size)
6. type t to change the type
7. enter the code 8e for Linux LVM
8. enter w to write the changes to disk and exit the application

LVM Physical Volume Creation

Whether or not you chose to partition the new disk, you will need to add it to the LVM configuration as a PV. To do this issue the command

pvcreate /dev/sdb1

(or pvcreate /dev/sdb if not partitioned)

If you want to see the before and after effects issue the command pvscan and pvdisplay before and after the pvcreate command shown above

Volume Groups

use of this command depends on whether or not you wish to add the new PV to an existing LVM Volume Group (VG) or a new VG.

To create a new VG: <<<TBD>>>(notes about different VG's in the same LV)<<<TBD>>>

vgcreate VolGroupXX /dev/sdb1

(or /dev/sdb if not partitioned), where the parameter VolGroupXX is the usual format for Volume Group named in RedHat, but any name meaningful to you as the system administrator can be used, with the XX being a 2 digit number.

To extend an existing VG:

vgextend VolGroupXX /dev/sdb1

(or /dev/sdb if not partitioned), where the paremeter VolGroupXX is the name of the existing Volume Group.

as above, before and after changes can be observed by running the vgscan and vgdisplay commands before and after the create or extend commands

Logical Volumes

To increase the size of the (or create a new) LVM Logical Volume (LV)

To create a new LV:

lvcreate -l 100%FREE -n /dev/VolGroupXX/LogVolYY /dev/VolGroup00

where the following parameters are used:

-l 100%FREE - uses all the available space in the Volume Group. Can be changed for -L nn[KMGTPE] for nn Kilo, Mega, Giga, Tera, Peta, Exa bytes -n /dev/VolGroupXX/LogVolYY - this is the name of the Logical Volume expressed as a path. With XX & YY as 2 digit numbers. This naming convention can be changed to suit your own needs but follows the default RedHat configuration. /dev/VolGroupXX - is the VG to take extents from for this LV.

To extend an existing LV:

lvextend -l +100%FREE /dev/VolGroupXX/LogVolYY /dev/sdb1

where the following parameters are used:

-l +100%FREE - extends the LV by all the available space in the VG. Can be changed as before for absolute values using -L. The preceding + means extend the current value by this much. Omitting the + means set the value to this much - if you pick the wrong value, you can actually shrink the LV!, which would be fatal if the file system has not yet been resized. /dev/VolGroupXX/LogVolYY - this is the LV you wish to extend /dev/sdb1 - this is teh PV you wish to use to extend the LV.

How to Increase the Filesystem

<<<TBD>>> Notes about increasing file system size. Include links to other documentation sites. <<<TBD>>>