Motorcycle Touring

Overview

There are many articles and tutorials on the use of the Linux Logical Volume Manager (LVM). I have two goals when I write up these sorts of docs. One to create a document that is appropriate for the environment and two to help others who need the information in the wee hours during an incident or event.

Background

One of the issues with disk management in Linux is being able to extend a file system when needed. Since file systems are generally split up into different uses; root, usr, var, home, and so on, being able to extend that file system when not using LVM means the new space must be contiguous with the file system you want to extend. For this reason, generally var is the last file system created in a system as that’s the file system more likely to need to be extended. The logs get bigger, applications such as mysql are in /var, etc. If a different file system needs to be extended, you have to create a brand new one of the new size, move all the files to the new file system, unmount the old one and mount the new one. This also leaves a gap in the file system usage. As such, when creating file systems admins tend to err on the side of caution and give a bit of extra space.

When using LVM, you add the new disk to the volume group and extend the volume to the necessary size. It does add a layer between the application and the physical disk so there is a minor performance hit. And with the new virtual technology, an admin can add a new disk at the touch of a button.

As a note, the boot disk is a physical slice and not brought under LVM control. This is due to the kernel being located in this file system and loaded prior to the LVM modules. Make sure this file system is large enough to support multiple kernels, about 500 gigabytes.

Commands

There are three groups of commands. Physical Volumes, Volume Groups, and Logical Volumes. Physical Volume commands manage the physical disks. Adding or removing them from LVM and making them available to the system. Volume Groups are collections of one or more Physical Volumes. And Logical Volumes are slices of the Volume Groups.

I’m not going to dig into every command. Feel free to check the references at the end of this article and on Linux server, use the lvm command to see a list of all possible commands on a server.

To view summary information, the pvs (physical volume), vgs (volume group), and lvs (logical volume) commands are used. The pvdisplay, vgdisplay, and lvdisplay provide extended information.

To create LVM volumes, the pvcreate (add a physical disk to lvm), vgcreate (create a volume group), and lvcreate (create a logical volume) commands are used.

To extend LVM volumes, the pvextend, vgextend, and lvextend commands are used. If you’re given a new disk, you would use pvcreate and then vgextend and lvextend. If an existing physical disk is extended though, you’d use pvextend and then lvextend.

Physical Volume Management

You are given a disk to be used for a server and need to add it. First off, you need to add it to LVM. This can be the entire disk or a partition on the disk. A partition will end in a number.

# pvcreate /dev/sda2
# pvcreate /dev/sdb

To view the information, you can use the pvs or pvdisplay command.

# pvs
  PV         VG   Fmt  Attr PSize    PFree
  /dev/sda2  vg00 lvm2 a--    37.79g 4.00m
  /dev/sdb   vg01 lvm2 a--  <100.00g    0
# pvdisplay
  --- Physical volume ---
  PV Name               /dev/sda2
  VG Name               vg00
  PV Size               <37.80 GiB / not usable 4.00 MiB
  Allocatable           yes
  PE Size               4.00 MiB
  Total PE              9675
  Free PE               1
  Allocated PE          9674
  PV UUID               UCPvsu-xHVc-hZ12-CQGW-y3Ss-vfUp-x1zCdS

  --- Physical volume ---
  PV Name               /dev/sdb
  VG Name               vg01
  PV Size               100.00 GiB / not usable 4.00 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              25599
  Free PE               0
  Allocated PE          25599
  PV UUID               eipsjW-KysF-VK4o-mUcV-D4j7-FsOu-uqra6l

Note that the pvdisplay command gives you the number of available or Free Physical Extents. This is the number you’d use when creating a logical volume.

If an existing physical disk is extended, you will need to delete the partition table and recreate it to the new size.

I’ve broken the example below up into sections to make it easier to parse.

# fdisk /dev/sda
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): p

Disk /dev/sda: 85.9 GB, 85899345920 bytes, 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000f08ea

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     1026047      512000   83  Linux
/dev/sda2         1026048    80478207    39726080   8e  Linux LVM

Command (m for help): d
Partition number (1,2, default 2): 2
Partition 2 is deleted

Note that the Linux LVM partition is 80k sectors. This equates to about 40 gigabytes. As noted here, you’ve been given 40 gigabytes more space but extending the original disk not as a second disk. You’ll delete the second partition (sda2) and add it back in at the new size.

Again, this is a single session. Do not save until the end.

Command (m for help): n
Partition type:
   p   primary (1 primary, 0 extended, 3 free)
   e   extended
Select (default p): p
Partition number (2-4, default 2): 2
First sector (1026048-167772159, default 1026048): 
Using default value 1026048
Last sector, +sectors or +size{K,M,G} (1026048-167772159, default 167772159): 
Using default value 167772159
Partition 2 of type Linux and of size 79.5 GiB is set

Command (m for help): p

Disk /dev/sda: 85.9 GB, 85899345920 bytes, 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000f08ea

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     1026047      512000   83  Linux
/dev/sda2         1026048   167772159    83373056   83  Linux

We’ve created a new partition using all the available sectors to extend it to 80 gigabytes. There is a slight issue though. Did you notice it?

Command (m for help): t
Partition number (1,2, default 2): 2
Hex code (type L to list all codes): 8e
Changed type of partition 'Linux' to 'Linux LVM'

Command (m for help): p

Disk /dev/sda: 85.9 GB, 85899345920 bytes, 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000f08ea

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     1026047      512000   83  Linux
/dev/sda2         1026048   167772159    83373056   8e  Linux LVM

It’s not a Linux LVM partition. You’ll need to change the partition type from Linux to 8e aka Linux LVM.

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.

And save your changes. Once done, use the partprobe command to have the kernel recognize the new partition table.

# partprobe

And finally you extend the physical volume in LVM.

# pvdisplay
  --- Physical volume ---
  PV Name               /dev/sda2
  VG Name               vg00
  PV Size               37.89 GiB / not usable 3.00 MiB
  Allocatable           yes 
  PE Size               4.00 MiB
  Total PE              9698
  Free PE               2
  Allocated PE          9696
  PV UUID               tsFRnn-bQNH-qm0Q-CY7O-6GTl-93P2-4lbbig
      
# pvresize /dev/sda2
  Physical volume "/dev/sda2" changed
  1 physical volume(s) resized / 0 physical volume(s) not resized
# pvdisplay
  --- Physical volume ---
  PV Name               /dev/sda2
  VG Name               vg00
  PV Size               79.51 GiB / not usable 2.00 MiB
  Allocatable           yes 
  PE Size               4.00 MiB
  Total PE              20354
  Free PE               10658
  Allocated PE          9696
  PV UUID               tsFRnn-bQNH-qm0Q-CY7O-6GTl-93P2-4lbbig

Volume Group Management

Now that you have physical volumes prepared, you need to either add a new volume group or extend an existing one.

# vgextend vg00 /dev/sdb
# vgcreate vg01 /dev/sdc 

In order to view a volume group, you can use the vgs or vgdisplay commands.

# vgs
  VG   #PV #LV #SN Attr   VSize    VFree
  vg00   1   6   0 wz--n-   37.79g 4.00m
  vg01   1   1   0 wz--n- <100.00g    0
# vgdisplay
  --- Volume group ---
  VG Name               vg00
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  7
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                6
  Open LV               6
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               37.79 GiB
  PE Size               4.00 MiB
  Total PE              9675
  Alloc PE / Size       9674 / <37.79 GiB
  Free  PE / Size       1 / 4.00 MiB
  VG UUID               yTaVIr-8bLB-HvQ9-WSns-LB6O-FYXW-AkydB2

  --- Volume group ---
  VG Name               vg01
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  2
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               1
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               <100.00 GiB
  PE Size               4.00 MiB
  Total PE              25599
  Alloc PE / Size       25599 / <100.00 GiB
  Free  PE / Size       0 / 0
  VG UUID               5Llgfv-9fxg-IxzJ-dT4j-ef5Q-NB0Y-RHL5jS

Logical Volume Management

Now you want to use the available space in file systems. Either by creating a new one or extending an existing file system such as /var.

The logical volume command has two flags for identifying space. The -l (lower case L) flag which uses the physical extents as the option and the -L flag which uses disk sizes in Megabytes, Gigabytes, etc as options.

The -n flag names the file system.

For example, to create a 20 gigabyte file system called fs01 using the vg00 volume group:

# lvcreate -L20G -nfs01 vg00

You can also create one using physical extents. You’ll need to get the amount of Free PE from the output of the pvdisplay command and then calculate what 20 gigabytes might be. In looking at the above examples, 80 gigabytes is approximately 20,354 physical extents. divide by four and you get approximately 5,000 physical extents.

# lvcreate -l5000 -nfs01 vg00

Of course it’s not exactly 20G which is why you’d use the -L flag vs trying to figure out what 20 gigabytes is in physical extents.

Assuming you’re creating a file system, you’ll use the appropriate command to format it. In this example, I’m using a common EXT4 file system but others are available depending on your use case.

# mkfs -t ext4 /dev/mapper/vg00-fs01
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
4915200 inodes, 19659776 blocks
982988 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
600 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
	4096000, 7962624, 11239424

Writing inode tables: done                            
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 27 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

Finally update /etc/fstab to add the mount point for the new file system.

And you’ll use the lvs or lvdisplay commands to view information about the logical volumes on the system.

# lvdisplay
  --- Logical volume ---
  LV Name                /dev/vg00/lv00
  VG Name                vg00
  LV UUID                Zr3Axv-eoOJ-cdM8-Cuvo-t7Db-vlPF-u3wpB3
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                1.00 GB
  Current LE             32
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:0
   
  --- Logical volume ---
  LV Name                /dev/vg00/lv05
  VG Name                vg00
  LV UUID                30VD3I-i90d-Efl1-PReu-0Wi3-V7ri-tyUJ4L
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                4.00 GB
  Current LE             128
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:1
  ...

If you want to extend a logical volume once more space has been added, you’ll use the lvextend command. It works the same as the lvcreate commands with a few changes.

First, you’ll want to pass the -r flag as it extends the file system in addition to extending the logical volume. Much like when you delete a partition and add it back in, and needing to use resize2fs to extend the file system, the -r flag extends the file system.

Next up you can either use a plus to add more space to at logical volume or simply select a larger size than it current is. For example, if you extend a 40 gigabyte logical volume to 80 gigabytes you can either pass -L+40G or -L80G.

# lvextend -r -L+40G /dev/mapper/vg00-fs01
lvextend -- extending logical volume "/dev/vg00/fs01" to 80 GB
lvextend -- doing automatic backup of volume group "vg00"
lvextend -- logical volume "/dev/vg00/fs01" successfully extended

If you forget the -r flag, use the resize2fs command to resize the file system (or other command if not an EXT4 file system).

# resize2fs -p /dev/vg01/fs01
resize2fs 1.41.12 (17-May-2010)
Resizing the filesystem on /dev/vg00/fs01 to 104855552 (4k) blocks.
Begin pass 1 (max = 1600)
Extending the inode table     XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
The filesystem on /dev/vg00/fs01 is now 104855552 blocks long.

Removing Volumes

In case you need to remove a logical volume, volume group, or physical volume, use the following commands.

To remove a logical volume, you have to unmount it first. Of course this means making sure no applications are using the file system or you won’t be able to unmount it.

# umount /dev/mapper/vg00-fs01
# lvremove /dev/mapper/vg00-fs01
lvremove -- do you really want to remove "/dev/mapper/vg00-fs01"? [y/n]: y
lvremove -- logical volume "fs01" successfully removed

Don’t forget to remove it from /etc/fstab.

To remove a volume group, run the vgremove command.

# vgremove vg01

If you just need to remove a physical volume from a volume group, use the vgreduce command.

# vgreduce vg01 /dev/sdb

Finally if you need to remove a physical volume from LVM control, use the pvremove command.

# pvremove /dev/sdb

References

A lot of this is experience but the LVM documentation got me started.

• LVM How To