Overview
There are many articles and tutorials on the use of the Linux Logical Volume Manager (LVM). I have two goals when I write up these sorts of docs. One to create a document that is appropriate for the environment and two to help others who need the information in the wee hours during an incident or event.
Background
One of the issues with disk management in Linux is being able to extend a file system when needed. Since file systems are generally split up into different uses; root, usr, var, home, and so on, being able to extend that file system when not using LVM means the new space must be contiguous with the file system you want to extend. For this reason, generally var is the last file system created in a system as that’s the file system more likely to need to be extended. The logs get bigger, applications such as mysql are in /var, etc. If a different file system needs to be extended, you have to create a brand new one of the new size, move all the files to the new file system, unmount the old one and mount the new one. This also leaves a gap in the file system usage. As such, when creating file systems admins tend to err on the side of caution and give a bit of extra space.
When using LVM, you add the new disk to the volume group and extend the volume to the necessary size. It does add a layer between the application and the physical disk so there is a minor performance hit. And with the new virtual technology, an admin can add a new disk at the touch of a button.
As a note, the boot disk is a physical slice and not brought under LVM control. This is due to the kernel being located in this file system and loaded prior to the LVM modules. Make sure this file system is large enough to support multiple kernels, about 500 gigabytes.
Commands
There are three groups of commands. Physical Volumes, Volume Groups, and Logical Volumes. Physical Volume commands manage the physical disks. Adding or removing them from LVM and making them available to the system. Volume Groups are collections of one or more Physical Volumes. And Logical Volumes are slices of the Volume Groups.
I’m not going to dig into every command. Feel free to check the references at the end of this article and on Linux server, use the lvm command to see a list of all possible commands on a server.
To view summary information, the pvs (physical volume), vgs (volume group), and lvs (logical volume) commands are used. The pvdisplay, vgdisplay, and lvdisplay provide extended information.
To create LVM volumes, the pvcreate (add a physical disk to lvm), vgcreate (create a volume group), and lvcreate (create a logical volume) commands are used.
To extend LVM volumes, the pvextend, vgextend, and lvextend commands are used. If you’re given a new disk, you would use pvcreate and then vgextend and lvextend. If an existing physical disk is extended though, you’d use pvextend and then lvextend.
Physical Volume Management
You are given a disk to be used for a server and need to add it. First off, you need to add it to LVM. This can be the entire disk or a partition on the disk. A partition will end in a number.
# pvcreate /dev/sda2
# pvcreate /dev/sdb
To view the information, you can use the pvs or pvdisplay command.
# pvs
PV VG Fmt Attr PSize PFree
/dev/sda2 vg00 lvm2 a-- 37.79g 4.00m
/dev/sdb vg01 lvm2 a-- <100.00g 0
# pvdisplay
--- Physical volume ---
PV Name /dev/sda2
VG Name vg00
PV Size <37.80 GiB / not usable 4.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 9675
Free PE 1
Allocated PE 9674
PV UUID UCPvsu-xHVc-hZ12-CQGW-y3Ss-vfUp-x1zCdS
--- Physical volume ---
PV Name /dev/sdb
VG Name vg01
PV Size 100.00 GiB / not usable 4.00 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 25599
Free PE 0
Allocated PE 25599
PV UUID eipsjW-KysF-VK4o-mUcV-D4j7-FsOu-uqra6l
Note that the pvdisplay command gives you the number of available or Free Physical Extents. This is the number you’d use when creating a logical volume.
If an existing physical disk is extended, you will need to delete the partition table and recreate it to the new size.
I’ve broken the example below up into sections to make it easier to parse.
# fdisk /dev/sda
Welcome to fdisk (util-linux 2.23.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Command (m for help): p
Disk /dev/sda: 85.9 GB, 85899345920 bytes, 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000f08ea
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 1026047 512000 83 Linux
/dev/sda2 1026048 80478207 39726080 8e Linux LVM
Command (m for help): d
Partition number (1,2, default 2): 2
Partition 2 is deleted
Note that the Linux LVM partition is 80k sectors. This equates to about 40 gigabytes. As noted here, you’ve been given 40 gigabytes more space but extending the original disk not as a second disk. You’ll delete the second partition (sda2) and add it back in at the new size.
Again, this is a single session. Do not save until the end.
Command (m for help): n
Partition type:
p primary (1 primary, 0 extended, 3 free)
e extended
Select (default p): p
Partition number (2-4, default 2): 2
First sector (1026048-167772159, default 1026048):
Using default value 1026048
Last sector, +sectors or +size{K,M,G} (1026048-167772159, default 167772159):
Using default value 167772159
Partition 2 of type Linux and of size 79.5 GiB is set
Command (m for help): p
Disk /dev/sda: 85.9 GB, 85899345920 bytes, 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000f08ea
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 1026047 512000 83 Linux
/dev/sda2 1026048 167772159 83373056 83 Linux
We’ve created a new partition using all the available sectors to extend it to 80 gigabytes. There is a slight issue though. Did you notice it?
Command (m for help): t
Partition number (1,2, default 2): 2
Hex code (type L to list all codes): 8e
Changed type of partition 'Linux' to 'Linux LVM'
Command (m for help): p
Disk /dev/sda: 85.9 GB, 85899345920 bytes, 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000f08ea
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 1026047 512000 83 Linux
/dev/sda2 1026048 167772159 83373056 8e Linux LVM
It’s not a Linux LVM partition. You’ll need to change the partition type from Linux to 8e aka Linux LVM.
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
And save your changes. Once done, use the partprobe command to have the kernel recognize the new partition table.
# partprobe
And finally you extend the physical volume in LVM.
# pvdisplay
--- Physical volume ---
PV Name /dev/sda2
VG Name vg00
PV Size 37.89 GiB / not usable 3.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 9698
Free PE 2
Allocated PE 9696
PV UUID tsFRnn-bQNH-qm0Q-CY7O-6GTl-93P2-4lbbig
# pvresize /dev/sda2
Physical volume "/dev/sda2" changed
1 physical volume(s) resized / 0 physical volume(s) not resized
# pvdisplay
--- Physical volume ---
PV Name /dev/sda2
VG Name vg00
PV Size 79.51 GiB / not usable 2.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 20354
Free PE 10658
Allocated PE 9696
PV UUID tsFRnn-bQNH-qm0Q-CY7O-6GTl-93P2-4lbbig
Volume Group Management
Now that you have physical volumes prepared, you need to either add a new volume group or extend an existing one.
# vgextend vg00 /dev/sdb
# vgcreate vg01 /dev/sdc
In order to view a volume group, you can use the vgs or vgdisplay commands.
# vgs
VG #PV #LV #SN Attr VSize VFree
vg00 1 6 0 wz--n- 37.79g 4.00m
vg01 1 1 0 wz--n- <100.00g 0
# vgdisplay
--- Volume group ---
VG Name vg00
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 7
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 6
Open LV 6
Max PV 0
Cur PV 1
Act PV 1
VG Size 37.79 GiB
PE Size 4.00 MiB
Total PE 9675
Alloc PE / Size 9674 / <37.79 GiB
Free PE / Size 1 / 4.00 MiB
VG UUID yTaVIr-8bLB-HvQ9-WSns-LB6O-FYXW-AkydB2
--- Volume group ---
VG Name vg01
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 2
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 1
Act PV 1
VG Size <100.00 GiB
PE Size 4.00 MiB
Total PE 25599
Alloc PE / Size 25599 / <100.00 GiB
Free PE / Size 0 / 0
VG UUID 5Llgfv-9fxg-IxzJ-dT4j-ef5Q-NB0Y-RHL5jS
Logical Volume Management
Now you want to use the available space in file systems. Either by creating a new one or extending an existing file system such as /var.
The logical volume command has two flags for identifying space. The -l (lower case L) flag which uses the physical extents as the option and the -L flag which uses disk sizes in Megabytes, Gigabytes, etc as options.
The -n flag names the file system.
For example, to create a 20 gigabyte file system called fs01 using the vg00 volume group:
# lvcreate -L20G -nfs01 vg00
You can also create one using physical extents. You’ll need to get the amount of Free PE from the output of the pvdisplay command and then calculate what 20 gigabytes might be. In looking at the above examples, 80 gigabytes is approximately 20,354 physical extents. divide by four and you get approximately 5,000 physical extents.
# lvcreate -l5000 -nfs01 vg00
Of course it’s not exactly 20G which is why you’d use the -L flag vs trying to figure out what 20 gigabytes is in physical extents.
Assuming you’re creating a file system, you’ll use the appropriate command to format it. In this example, I’m using a common EXT4 file system but others are available depending on your use case.
# mkfs -t ext4 /dev/mapper/vg00-fs01
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
4915200 inodes, 19659776 blocks
982988 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
600 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 27 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
Finally update /etc/fstab to add the mount point for the new file system.
And you’ll use the lvs or lvdisplay commands to view information about the logical volumes on the system.
# lvdisplay
--- Logical volume ---
LV Name /dev/vg00/lv00
VG Name vg00
LV UUID Zr3Axv-eoOJ-cdM8-Cuvo-t7Db-vlPF-u3wpB3
LV Write Access read/write
LV Status available
# open 1
LV Size 1.00 GB
Current LE 32
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:0
--- Logical volume ---
LV Name /dev/vg00/lv05
VG Name vg00
LV UUID 30VD3I-i90d-Efl1-PReu-0Wi3-V7ri-tyUJ4L
LV Write Access read/write
LV Status available
# open 1
LV Size 4.00 GB
Current LE 128
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:1
...
If you want to extend a logical volume once more space has been added, you’ll use the lvextend command. It works the same as the lvcreate commands with a few changes.
First, you’ll want to pass the -r flag as it extends the file system in addition to extending the logical volume. Much like when you delete a partition and add it back in, and needing to use resize2fs to extend the file system, the -r flag extends the file system.
Next up you can either use a plus to add more space to at logical volume or simply select a larger size than it current is. For example, if you extend a 40 gigabyte logical volume to 80 gigabytes you can either pass -L+40G or -L80G.
# lvextend -r -L+40G /dev/mapper/vg00-fs01
lvextend -- extending logical volume "/dev/vg00/fs01" to 80 GB
lvextend -- doing automatic backup of volume group "vg00"
lvextend -- logical volume "/dev/vg00/fs01" successfully extended
If you forget the -r flag, use the resize2fs command to resize the file system (or other command if not an EXT4 file system).
# resize2fs -p /dev/vg01/fs01
resize2fs 1.41.12 (17-May-2010)
Resizing the filesystem on /dev/vg00/fs01 to 104855552 (4k) blocks.
Begin pass 1 (max = 1600)
Extending the inode table XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
The filesystem on /dev/vg00/fs01 is now 104855552 blocks long.
Removing Volumes
In case you need to remove a logical volume, volume group, or physical volume, use the following commands.
To remove a logical volume, you have to unmount it first. Of course this means making sure no applications are using the file system or you won’t be able to unmount it.
# umount /dev/mapper/vg00-fs01
# lvremove /dev/mapper/vg00-fs01
lvremove -- do you really want to remove "/dev/mapper/vg00-fs01"? [y/n]: y
lvremove -- logical volume "fs01" successfully removed
Don’t forget to remove it from /etc/fstab.
To remove a volume group, run the vgremove command.
# vgremove vg01
If you just need to remove a physical volume from a volume group, use the vgreduce command.
# vgreduce vg01 /dev/sdb
Finally if you need to remove a physical volume from LVM control, use the pvremove command.
# pvremove /dev/sdb
References
A lot of this is experience but the LVM documentation got me started.
