LVM Notes

Table of Contents (TOC)

Adding a new disk and having the system see it.
Partitioning the new disk
Creating the LVM
Removing an LVM
Expanding an LVM
Reducing swap space

There are a lot of good LVM guides on the net. This one is written for myself, dealing with issues that I forget.


If you're reading this, you probably already know what an LVM is, and its advantages. It's very commonly used with iscsi servers connecting to network storage space, and with fibre channel connections such as an EMC SAN. It can expand partitions without taking them offline, including the root partition, and can create a partition that spans disks.

For the beginner, some of the disadvantages are that the syntax can be a little tricky, and the popular gparted doesn't (yet) work with it. Also, legacy grub, still used by many systems, including Fedora as of F15, will require a separate /boot partition. (Grub2 is able to boot from an LVM).

Assuming you have decided to use LVM, either during installation, or when adding disks afterwards, the procedure is relatively straightforward. One first creates a physical volume, then a volume group inside the physical volume. Next, one creates the logical volume inside the volume group. Lastly, one creates a filesystem for the logical volume.

Note that most of this will require root privilege.


Adding a new disk and having the system see it

For the home user, this usually means shutting the machine down, adding a disk, and restarting, in which case the system should see the disk. The beginner can almost certainly skip the rest of this section, and go to Partitioning. The rest of this section gives a rough overview of adding disks to SAN and NAS. Sometimes, one has a hot-swappable system, where the disk can be added without shutting off the machine.

In the workplace, this might mean adding a disk in a SAN or NAS. In any case where the disk has been added and the machine hasn't been restarted, it may not see the new space.

(For the newcomer, if any of the terms below are unfamiliar, don't worry. Briefly iscsi is a method of linking, by IP address, to storage devices. HBA stands for Host Bus Adapter, and is used with fibre channel storage.).

One can run fdisk -l to see if the new disk has been detected. If not, and it is an iscsi server then
iscsiadm -m node -R

will have the iscsi daemon rescan all sessions and should show the space. If this doesn't work, (though it should), you can also try
echo "- - -" > /sys/class/scsi_host/hostX/scan

The X in hostX refers to whatever host number you have in /sys/class/scsi_host, so the command might go
echo "- - -" > /sys/class/scsi_host/host1/scan

Also, note that there is a space between each dash.

With HBA's I usually use (if your browser doesn't show it, this should all be one line)
for name in $(ls /sys/class/scsi_device);do echo 1 >

I haven't done this often enough to really keep track of what works with what (that is SAN vs. NAS), but one of the three steps should show the new disk, and running fdisk -l should show a new, unpartitioned disk.

A couple of other things to note with NAS and SAN. The new disk might not show up where you expected it. In other words, if you have sda, sdb and sdc, you might find that you suddenly have sda, sdb, sdc, and sdd, but that the unpartitioned disk isn't, as you would expect, sdd, but is actually sdb with the former sdb now being sdc, etc.

When using HBA's, if there's more than one port, you might see that you suddenly have 5 new disks, all the same size. You can run the partitioning and volume creating commands on any of them, and it will work--that is, you will wind up with one disk, and suddenly, sdd, sde, sdf and so on will all show the results of your partitioning.


Partitioning the new disk

Now running fdisk -l has shown us that we have a new drive. In this example, it will be drive /dev/sdd. The first thing to do is create a partition. For this we can use fdisk.
fdisk /dev/sdd

I have an old page about fdisk that is, hopefully, still useful, explaining what we're doing here in a bit more detail. For purposes of this page, it's assumed that the user has some idea about what fdisk does. We'll create a new partition, make it primary, as it will be the only partition on the new disk, change its filesystem to LVM and then write it to disk. So first one types

For a new partition. It will give you a choice of e for extended or p for primary. Type

for primary. When asked for the partition number, type 1. It will then ask for the first cylinder. Just hitting enter accepts the default of the beginning of the drive. It will then ask for the last cylinder, again giving a default, this time the end of the drive. Hit enter again to accept it. Now type

to toggle the filesystem type. It will tell you that you can type L for a list of hex codes. We want 8e for LVM, so type that in. At this point, I like to type p as in print, to make sure that it's done what I wanted. Then type w as in write, and it will give a message that it's writing to disk. It may also say that the kernel is using the old partition table, rebooting is necessary.

However, one can avoid the reboot by running the command

In RH based systems at least, partprobe is provided by the parted package.
You should now see, if you do ls /dev/sd* that you have a /dev/sdd1.


Creating the LVM

Now it's time to create the logical volume. First, use pvcreate to make your physical space.
pvcreate /dev/sdd1

You should see a message that the volume was successfully created. (I'm using a CentOS 5.6 machine, with an extra 500 GB drive. You'll note that the sizes listed are somewhat smaller. The system reserves some disk space.)

If you run the command pvdisplay, you'll see something like
"dev /sdd1" is a new physical volume of "465.76 GB"
  PV Name               /dev/sdd1
  VG Name
  PV Size               465.76 GB
  Allocatable           NO
  PE Size (KByte)       0
  Total PE              0
  Free PE               0
  Allocated PE          0
  PV UUID               8t0c12-Q8p9-IzRs-2Zaj-i0QQ-G09V-ukR5J4

The next step is to make a volume group on the physical volume. You can run pvdisplay to see what you have--it should show a physical volume which is about the size of the disk. I like to give volumegroups, as well as LVMs, distinctive names--it will make it easier if I have to resize one, and I can be sure that I'm working on the right thing. In this case, I'll call it testvol
vgcreate testvol /dev/sdd1

Running pvdisplay now shows me
 --- Physical volume ---
  PV Name               /dev/sdc1
  VG Name               testvol
  PV Size               465.76 GB / not usable 1.50 MB
  Allocatable           yes
  PE Size (KByte)       4096
  Total PE              119234
  Free PE               119234
  Allocated PE          0

Running vgdisplay shows me
VG Name               testvol
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  1
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               465.76 GB
  PE Size               4.00 MB
  Total PE              119234
  Alloc PE / Size       0 / 0
  Free  PE / Size       119234 / 465.76 GB
  VG UUID               La9WTg-gZK4-pHA3-c9lH-YpVL-oM32-vpJSCB

The important thing is that I see how many free PE (physical Extents) that I have, for I'll use that in the next step, creating a logical volume on the this volume group. I'll call the logical volume test.

I'm going to use the number of extents. Although it also gives the size in GB, I've found that using that result almost always tells me there isn't enough free space, that it will be a few extents short. Since it's easy to see the number of free extents with the vgdisplay command, I might as well use them. One can also use a percentage--in this case, I am going to use the entire partition, so I can use percent instead of extents. To make the logical volume, named test, I run
lvcreate -n test -l 100%VG testvol

To do it with extents I would have used
lvcreate -n test -l 119324 testvol

So, the syntax is lvcreate -n for a name -l for the size in extents or percent and finally, the name of the volume group that you're using. The l is a lower case letter L, not the numeral one.

If you now look at your /dev directory, you'll see that there's an entry /dev/testvol/test which is a symbolic link to /dev/mapper/testvol-test.

The last step is to create a file system on the LVM. In this case, as I'm on CentOS 5.6, I'll use ext3.
mkfs.ext3 /dev/testvol/test

It will go through creating a file system on the logical volume. When done it will tell you that the drive will be checked every 39 mounts or 180 days, whichever comes first and that you can use tune2fs to change this. If you do want to change the behavior, then
tune2fs -c 0 /dev/testvol/test

Now that it has a file system, you can create a mount point and mount the new drive. For example
mkdir lvm_mount
mount /dev/testvol/test /lvm_mount

To make the mount permanent, add an entry to your /etc/fstab. Nothing special has to be done, the entry can be like any other. For example
/dev/testvol/test          /lvm_mount     ext3      defaults   0 0

Removing an LVM

To remove it one pretty much reverses the procedure. First, unmount the mounted LVM.
umount /lvm_mount

Note that the command is umount, NOT unmount.

Remove the logical volume.
lvremove /dev/testvol/test

You'll be asked if you're sure, hit y for yes.
Deactivate the volume group, then remove it.
vgchange -a n testvol
vgremove testvol

Now the physical volume
pvremove /dev/sdd1

Once this is done, you should then run fdisk and repartition the drive. If you don't know what you're planning to do with it, just remove the partition. In this case, it's simple, as there's only one partition.

The d is for delete--as there's only partition, it won't ask which partition to delete, then w for write, as we did before. Afterwards, one should run partprobe to make sure the system reads the new partition table.


Expanding an LVM

An existing LVM can be expanding onto a new partition or a new disk. In this example, we'll take a 20 GB partition on a hard disk and add another 20 GB. Once again, we'll use the name testvol with a logical volume named test.

So we have an LVM /dev/testvol/test that is 20GB in size. We now either add a new partition with fdisk, or perhaps expand the drive if it's an external SAN or NAS. In any case, one might have to follow the same steps given in adding a new disk to get the system to see it.

Once again, we'll use /dev/sdd. In this case, the steps are to add a new partition, give it a physical volume (I find this to be easier and less prone to my own errors than using pvresize), expand the volume group into it, then expand the logical volume. So first run fdisk as before, add another 20 GB partition. For example fdisk /dev/sdd, hit n for new, p for primary, 2 for partition number. Take the default starting cylinder and to add another 20GB use +20480M. (A GB is 1024 megabytes, so that's 1024*20). Use t to toggle it, choose partition 2, choose 8e as the partition type for LVM, hit w for write, and then if it says the system must be rebooted, run partprobe.

Now we create the new physical volume.
pvcreate /dev/sdd2

Running pvdisplay should now show us a new physical volume with something like
"/dev/sdb2" is a new physical volume of "20.91 GB"
  --- NEW Physical volume ---
  PV Name               /dev/sd22
  VG Name
  PV Size               20.91 GB
  Allocatable           NO
  PE Size (KByte)       0
  Total PE              0
  Free PE               0
  Allocated PE          0
  PV UUID               22ayk4-Ghhq-8yns-uNHh-nDdf-OGhc-p0G106
In theory, we can now extend the volumegroup testvol. In practice, it's always safer, if possible, to unmount the volume before working with it. If this was mounted on /lvm_mount
umount /lvm_mount. 

This is up to the individual--in production, I always unmount the volume first, at home, it depends. I have lost data once while extending a logical volume, but I've also frequently resized them without losing anything. Whether the volume is now mounted or not, the next steps are the same.

We will now extend the volumegroup testvol into the new physical volume.
vgextend testvol /dev/sdd2

If we now run vgdisplay testvol we'll see that we have 5353 extents free. We can, once again, either use -l 5353 or -l 100%VG.
lvextend -l 100%VG /dev/testvol/test

(I believe that using lvextend -lr will resize the file system, but, to be honest, haven't tried it. These days, I'm usually using zfs instead.)

Lastly we have to resize the filesystem as well. (But see above). One can unmount the partition, first, but usually, it's not necessary. If one runs df -H at this point, you'll still see the old size of 20GB. If using the RedHat and offshoots default of an XFS file system, use xfs_growfs, otherwise, use resize2fs.
resize2fs /dev/testvol/test

You may get a message like please run 'e2fsck -f /dev/testvol/test' first. I have never tried running e2fsck on a mounted logical volume, it's probably a bad idea. At any rate, run it. Hopefully, it completes without problems.

If the volume is mounted, you will see a message that it is performing an online resizing. If you run df -H again, you'll see that the /lvm_mount partition is now 20 GB larger.

Note that online resizing can be done with ext3fs but not with ext2fs. Although it is ext3 the command is still resize2fs.


Reducing swap space after install

RedHat based systems, e.g., CentOS, ScientificLinux and Fedora, can choose some odd defaults. For example, on a VirtualBox installation, if I make a 15 GB disk for ScientificLinux 6 (which is being used in this example), with 2GB of RAM, it will make a swap space of close to 5 GB. This is my quick and dirty way of fixing it. The only real difference from the typical way is that I completely delete the partition, as I find it easier to do that than to shrink space, then enlarge another partition.

In this default installation of ScientificLinux 6 (SL6), I use a 15GB disk and let it use a default partitioning scheme. Running pvdisplay afterwards shows me that 14.51 GB are used, with none free. Running lvdisplay shows me that the swap partition is 3.97 GB, more than I want to give it. Running ls /dev/mapper shows me that I have two logical volumes. Assuming I gave the machine a name like, the name will be something like /dev/mapper/vg_myhost-lv_root and vg_myhost-lv_swap.

First I want to completely delete the swap partition. I turn off swap first, (with any other sort of partition, one would unmount it before removing or reducing. With swap, one just turns it off.) Next I remove the logical volume. (NOT THE VOLUME GROUP). It will ask me if I'm sure, giving me a y/n option, and I will type y for yes.
swapoff /dev/mapper/vg_myhost-lv_swap
lvremove /dev/mapper/vg_myhost-lv_swap

Logical volumes are shown two ways in /dev. One is the one I use here, /dev/mapper/vg_myhost-lv_swap and also as /dev/vg_myhost/lv_swap. Use either one. Both are actually symlinks to /dev/dm-X, (starting with /dev/dm-0).

Now I want to see how much space I have.

This will give me various information, including Free PE / Size. PE is a physical extent, usually about 4MB, but size should give it, in this case, in GBs. In my case this line reads
Free PE / Size		1016 / 3.97 GiB

Out of this 3.97 GB, I decide that I want to extend the root partition 2.5 GB.
lvextend -L +2.5G /dev/mapper/vg_myhost-lv_root

-L is used when using sizes like G or M for gigabytes and megabytes, -l (lower case L) is used when using physical extents.

You should see a message that the volume was successfully resized.

Now to check how much space I have left, I run vgdisplay again. This time I see that I now I have 1.47 GB of free space. It will also show me the number of free extents. As the GB measurement usually isn't exact, it's easier to use extents. For example, if I actually use -L +1.47G I'll get a message that it's 4MB short. (The message will be something like Insufficient space, 377 PE required only 376 available.) So, as vgdisplay showed me that I have 376 PE left, I'll use that. I'll use the -n option to give it the same name that it had before.
lvcreate -l 376 -n lv_swap vg_myhost

In other words using extents, we use a lower case L, the -n is the name. This can confuse people. If you remember, the name was vg_myhost-lv_swap. However, if you put in that full name, you will have /dev/mapper/vg_myhost-vg_myhost-lv-swap. So just put in the last part of the name, in other words, whatever you would expect to see AFTER vg_myhost-. You can name it anything you wish, in this case, I'm staying with the original naming scheme.

Lastly, give the name of the volume group, the vg_myhost.

You should see that the logical volume was created. Now you will have to make it a swap partition and turn the swap on.
mkswap /dev/mapper/vg_myhost-lv_swap
swapon /dev/mapper/vg_myhost-lv_swap

Running the free command (just type the word "free", without quotes, at a command prompt), should now show you that swap is working.

Lastly, we have to resize the filesystem on the root logical volume. If you run df -H right now, you'll see that the / partition is still at the old size.
resize2fs /dev/mapper/vg_myhost-lv_root

You'll see a message that it is performing an online resize. Running df -H will now show you the new size for the / partition.

I just find this method a bit simpler than reducing the swap partition. The user may prefer the more common method of using lvreduce, but this, at least for me, is somewhat quicker.