Quick HOWTO : Ch27 : Expanding Disk Capacity

From Linux Home Networking
Jump to: navigation, search

Introduction

The lack of available disk storage frequently plagues Linux systems administrators. The most common reasons for this are expanding databases, increasing numbers of users, and the larger number of tasks your Linux server is expected to perform until a replacement is found.

This chapter explores how to add a disk to a Linux system in two ways. The first is by moving directories from a full partition to an empty one made available by the new disk and then linking the directory structures of the two disks together. The second is by merging the partitions together to create a combined partition using the Linux Logical Volume Manager (LVM.

Debian / Ubuntu Differences

This chapter focuses on Fedora / CentOS / RedHat for simplicity of explanation. Whenever there is a difference in the required commands for Debian / Ubuntu variations of Linux it will be noted.

The universal difference is that the commands shown are done by the Fedora / CentOS / RedHat root user. With Debian / Ubuntu you will either have to become root using the "sudo su –" command or you can temporarily increase your privilege level to root using the "sudo <command>" command.

Here is an example of how to permanently become root:

user@ubuntu:~$ sudo su -
[sudo] password for peter: 
root@ubuntu:~#

Here is an example of how to temporarily become root to run a specific command. The first attempt to get a directory listing fails due to insufficient privileges. The second attempt succeeds when the sudo keyword is inserted before the command.

user@ubuntu:~$  ls -l /var/lib/mysql/mysql
ls: cannot access /var/lib/mysql/mysql: Permission denied
user@ubuntu:~$ sudo ls -l /var/lib/mysql/mysql
[sudo] password for peter: 
total 964
-rw-rw---- 1 mysql mysql   8820 2010-12-19 23:09 columns_priv.frm
-rw-rw---- 1 mysql mysql      0 2010-12-19 23:09 columns_priv.MYD
-rw-rw---- 1 mysql mysql   4096 2010-12-19 23:09 columns_priv.MYI
-rw-rw---- 1 mysql mysql   9582 2010-12-19 23:09 db.frm
...
...
...
user@ubuntu:~$
 

Now that you have got this straight, let’s continue with the discussion.

Adding Disks To Linux

At some stage you'll be faced with the task of installing an additional hard drive into your Linux server. Perhaps an existing device failed, or maybe you ran out of available space. To provide more space, this section will cover adding a hard disk with only one partition and will then explain how to migrate data from the full disk to the new one.

Scenario

Things are getting crowded on bigboy: Even after you removed all unwanted data, the /var partition is full. You need to add a new hard drive to the system. You can verify this situation with the df -k command's output, which also shows that the other partitions are too full to accept any more data.

[root@bigboy tmp]# df -k
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/hda3               505636    118224    361307  25% /
/dev/hda1               101089     14281     81589  15% /boot
none                     63028         0     63028   0% /dev/shm
/dev/hda5               248895      6613    229432   3% /tmp
/dev/hda7              3304768   2720332    416560  87% /usr
/dev/hda2              3304768   3300536      4232  99% /var
[root@bigboy tmp]#

A new hard disk was added according to the manufacturer's instructions, but you now need to know how to proceed.

Determining The Disk Types

Linux stores the names of all known disk partitions in the /proc/partitions file. The entire hard disk is represented by an entry with a minor number of 0, and all the partitions on the drive are sequentially numbered after that. In the example, the system has two hard disks; disk /dev/hda has been partitioned, but the new disk (/dev/hdb) needs to be prepared to accept data.

[root@bigboy tmp]# cat /proc/partitions
major minor  #blocks  name
 
   3     0    7334145 hda
   3     1     104391 hda1
   3     2    1052257 hda2
   3     3    2040255 hda3
   3     4          1 hda4
   3     5    3582463 hda5
   3     6     554211 hda6
  22     0   78150744 hdb
[root@bigboy tmp]#

Note: Linux hard disk device names follow a specific standard. SCSI disks all start with sd and IDE disks with hd. After this comes a letter that identifies the unit number of the disk, so for example, the first disk would be a, the second would be b, the third would be c, and so on. Finally, a two-digit number defines the partition number. Using this convention the fifth partition on the fourth IDE drive would be /dev/hdd5.

Preparing Partitions on New Disks

Linux partition preparation is very similar to that in a Windows environment, because both operating systems share the fdisk partitioning utility. The steps are:


1) The first Linux step in adding a new disk is to partition it in preparation of adding a filesystem to it. Type the fdisk command followed by the name of the disk. You want to run fdisk on the /dev/hdb disk, so the command is:

[root@bigboy tmp]# fdisk /dev/hdb
 
The number of cylinders for this disk is set to 9729.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)
 
Command (m for help): 

2) Just to make sure you're on the correct device, issue the p command to print all the known partitions on the disk. In this case, there are none which is good.

Command (m for help): p
 
Disk /dev/hdb: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
 
   Device Boot      Start         End      Blocks   Id   System
 
Command (m for help):

3) The fdisk m command prints a small help manual of valid commands. You will see that n is the command to add a new partition. Add a new primary partition, number 1, and use the defaults to make the partition occupy the entire disk.

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)

Partition number (1-4): 1
First cylinder (1-9729, default 1):<RETURN>
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-9729, default 9729):

4) Run the print (p) command to confirm that you successfully created the partition partition.

Command (m for help): p
 
Disk /dev/hdb: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
 
   Device Boot      Start         End      Blocks   Id   System
/dev/hdb1               1        9726    78148161   83  Linux
 
Command (m for help):

Tip: If you make a mistake, you can use the d command to delete the partition and start over. The t command enables you to change partition type from the default of 83 for regular Linux partitions to something else, such as 82 for swap space. In most cases, this won't be necessary, the default value is sufficient.

Note: When you created the new partition, you may have noticed that fdisk queried you as to whether it was going to be a primary or secondary partition. Linux allows only four primary partitions; if you need more, you can convert one of the primary ones into an extended one. Here is an example of a partition table that includes an extended partition followed by two regular partitions within it.

Command (m for help): p
 
Disk /dev/hda: 7510 MB, 7510164480 bytes
255 heads, 63 sectors/track, 913 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
 
   Device Boot      Start         End      Blocks   Id   System
/dev/hda1   *           1          13      104391   83  Linux
/dev/hda2              14         144     1052257+  83  Linux
/dev/hda3             145         398     2040255   82  Linux swap
/dev/hda4             399         913     4136737+   5  Extended
/dev/hda5             399         844     3582463+  83  Linux
/dev/hda6             845         913      554211   83  Linux
 
Command (m for help):

Adding more partitions is just a question of repeating the previous steps the required number of times, while remembering that at some stage, you may need to add an extended partition.

5) Changes won't be made to the disk's partition table until you use the w command to write, or save, the changes. Do that now, and, when finished, exit with the q command.

Command (m for help): w
Command (m for help): q

After this is complete you'll need to verify your work and start migrating your data to the new disk. These steps will be covered next.

Verifying Your New Partition

You can take a look at the /proc/partitions file or use the fdisk -l command to see the changes to the disk partition structure of your system:

[root@bigboy tmp]# cat /proc/partitions
major minor  #blocks  name
...
...
...
  22     0   78150744 hdb
  22     1   78150744 hdb1
[root@bigboy tmp]#
 
 
[root@bigboy tmp]# fdisk -l
...
...
...
Disk /dev/hdb: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
 
   Device Boot      Start         End      Blocks   Id  System
/dev/hdb1              1         9729    76051710   83  Linux
[root@bigboy tmp]#

Putting A Directory Structure On Your New Partition

You now need to format the partition, giving it a new directory structure by using the mkfs command. The Fedora installation procedure defaults to an ext3 type, which is what you should use here.

[root@bigboy tmp]# mkfs -t ext3 /dev/hdb1

Next, you must create special mount point directory, to which the new partition will be attached. Create directory /mnt/hdb1 for this purpose.

[root@bigboy tmp]# mkdir /mnt/hdb1

When Linux boots, it searches the /etc/fstab file for a list of all partitions and their mounting characteristics, and then it mounts the partitions automatically. You'll have to add an entry for your new partition that looks like this:

#
# File: /etc/fstab
#
/dev/hdb1  /mnt/hdb1  ext3  defaults 1 2

The first entry is the name of the partition followed by the mount point directory and the filesystem type. The fourth entry defines the mounting options, which need be only default for most scenarios. The fifth entry governs whether the dump filesystem backup command can be used for the filesystem. A value of 0 means no, and 1 means yes. The final entry defines the order in which a filesystem check is done at boot time. The check is done twice. The root (or master) filesystem has a value of 1 and is checked on the first pass, all other filesystems should have a value of 2. If you are not familiar with the /etc/fstab file use the man fstab command to get a full explanation of its various options.

You don't have to wait for a reboot to mount your partition. You can use the mount command with the -a option to read the /etc/fstab file for new entries.

[root@bigboy tmp]# mount -a

You are now able to access your new partition as device /mnt/hdb1.

Migrating Data Over To your New Partition

As you remember from investigating with the df -k command, the /var partition is almost full.

[root@bigboy tmp]# df -k
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/hda3               505636    118224    361307  25% /
/dev/hda1               101089     14281     81589  15% /boot
none                     63028         0     63028   0% /dev/shm
/dev/hda5               248895      6613    229432   3% /tmp
/dev/hda7              3304768   2720332    416560  87% /usr
/dev/hda2              3304768   3300536      4232  99% /var
[root@bigboy tmp]#

The du -sk * command shows the disk usage of all subdirectories in a directory. You can recursively use the command by using the cd command to step down through all the subdirectories until you discover the one with the greatest file usage. In this case, you only had to go to the /var directory to see that the /var/transactions directory was the culprit.

[root@bigboy tmp]# cd /var
[root@bigboy var]# du -sk *
2036    cache
4       db
8       empty
...
...
133784  transactions
...
...
[root@bigboy var]#

As a solution, the /var partition will be expanded to the new /dev/hdb1 partition mounted on the /mnt/hdb1 directory mount point. To migrate the data, use these steps:

1) Back up the data on the partition you are about to work on.

2) Use the who command to see who's logged in. If other users are present, send a message with the wall command informing them that the system is about to shutdown:

[root@bigboy tmp]# who
root     pts/0        Nov  6 14:46 (192-168-1-242.my-site.com)
bob      pts/0        Nov  6 12:01 (192-168-1-248.my-site.com)
bunny    pts/0        Nov  6 16:25 (192-168-1-250.my-site.com)
[root@bigboy tmp]# wall The system is shutting down now!
 
Broadcast message from root (pts/0) (Sun Nov  7 15:04:27 2004):
 
The system is shutting down now!
[root@bigboy tmp]#

3) Log into the VGA console, and enter single-user mode.

[root@bigboy tmp]# init 1

4) Rename the /var/transactions directory /var/transactions-save to make sure you have an easy to restore backup of the data, not just the tapes.

sh-2.05b# mv /var/transactions /var/transactions-save

5) Create a new, empty /var/transactions directory; this will later act as a mount point.

sh-2.05b# mkdir /var/transactions

6) Copy the contents of the /var/transactions-save directory to the root directory of /dev/hdb1, which is actually /mnt/hdb1.

sh-2.05b# cp -a /var/transactions-save/* /mnt/hdb1

7) Unmount the new /dev/hdb1 partition.

sh-2.05b# umount /mnt/hdb1

8) Edit the /etc/fstab file, removing our previous entry for /dev/hdb1 replacing it with one using the new mount point.

#
# File: /etc/fstab
#
 
#/dev/hdb1  /mnt/hdb1  ext3  defaults 1 2
 
/dev/hdb1  /var/transactions  ext3  defaults 1 2

9) Remount /dev/hdb1 on the new mount point using the mount -a command, which reads /etc/fstab and automatically mounts any entries that are not mounted already.

sh-2.05b# mount -a

10) Test to make sure that the contents of the new /var/transactions directory is identical to /var/transactions-save.

11) Return to multi-user mode by typing exit. The system will return to its default runlevel.

sh-2.05b# exit

12) Make sure your applications are working correctly and delete both the /var/transactions-save directory and the /mnt/hdb1 mount point directory at some later date.

This exercise showed you how to migrate the entire contents of a subdirectory to a new disk. Linux also allows you to merge partitions together, to create a larger combined one. The reasons and steps for doing so will be explained next.

Expanding Partitions With LVM

The Logical Volume Manager (LVM) enables you to resize your partitions without having to modify the partition tables on your hard disk. This is most useful when you find yourself running out of space on a filesystem and want to expand into a new disk partition versus migrating all or a part of the filesystem to a new disk.

  • Physical Volume: A physical volume (PV) is another name for a regular physical disk partition that is used or will be used by LVM.
  • Volume Group: Any number of physical volumes (PVs) on different disk drives can be lumped together into a volume group (VG). Under LVM, volume groups are analogous to a virtual disk drive.
  • Logical Volumes: Volume groups must then be subdivided into logical volumes. Each logical volume can be individually formatted as if it were a regular Linux partition. A logical volume is, therefore, like a virtual partition on your virtual disk drive.
This may seem complicated, but it allows you to create new virtual partitions with sizes you can change from groups of real disk partitions whose sizes you probably cannot change. Another advantage of LVM is that this can all be done without disturbing other partitions on your hard disks.
  • Physical Extent: Real disk partitions are divided into chunks of data called physical extents (PEs) when you add them to a logical volume. PEs are important as you usually have to specify the size of your volume group not in gigabytes, but as a number of physical extents.

Make sure you understand these terms fully as they will be used repeatedly in many of the following sections. Lets go!

Configuring LVM Devices

It is probably best to learn about the features of LVM through a scenario. Suppose a small company needs to expand disk capacity, but there isn't the budget to purchase an adequately sized hard drive. The /home filesystem, which resides on /dev/hde5, has become too full. You just added a new hard drive /dev/hdf with 50% of the capacity of /dev/hde5 into which you want to expand /home. The device /dev/hdf has a single partition named /dev/hdf1 into which /dev/hde5 will be merged. Take a look at the required steps.

Backup Your Data

Use the tar command or some other method to backup your data in /home. The LVM process will destroy the data on all physical volumes.

Unmount your /home filesystem

As /home stores most users' data, you'll need to do some preparatory work before unmounting the filesystem.

1) Use the who command to see who's logged in. If other users are present, send a message with the wall command informing them that the system is about to shutdown.

[root@bigboy tmp]# who
root     pts/0        Nov  6 14:46 (192-168-1-242.my-site.com)
bob      pts/0        Nov  6 12:01 (192-168-1-248.my-site.com)
bunny    pts/0        Nov  6 16:25 (192-168-1-250.my-site.com)
[root@bigboy tmp]# wall The system is shutting down now!
 
Broadcast message from root (pts/0) (Sun Nov  7 15:04:27 2004):
 
The system is shutting down now!
[root@bigboy tmp]#

2) Log into the VGA console, and enter single-user mode.

[root@bigboy tmp]# init 1

3) Unmount the filesystem.

sh-2.05b# umount /home

Now we're ready to start modifying the partitions which is covered next.

Determine The Partition Types

You have to change each LVM partition used to be of type 8e (Linux LVM). You can test this with the fdisk -l command. Here is an example using /dev/hde that shows your target partitions are of the incorrect type.

sh-2.05b# fdisk -l /dev/hde
 
Disk /dev/hde: 4311 MB, 4311982080 bytes
16 heads, 63 sectors/track, 8355 cylinders
Units = cylinders of 1008 * 512 = 516096 bytes
 
   Device Boot    Start       End    Blocks   Id  System
/dev/hde1             1      4088   2060320+  fd  Linux raid autodetect
/dev/hde2          4089      5713    819000   83  Linux
/dev/hde3          5714      6607    450576   83  Linux
/dev/hde4          6608      8355    880992    5  Extended
/dev/hde5          6608      7500    450040+  83  Linux
sh-2.05b#

Start FDISK

You can change the partition type using fdisk with the disk name as its argument. Use it to modify both partitions /dev/hde5 and /dev/hdf1. The fdisk examples that follow are for /dev/hde5; repeat them for /dev/hdf1.

sh-2.05b# fdisk /dev/hde
 
The number of cylinders for this disk is set to 8355.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)
 
Command (m for help):

Set The ID Type To 8e

You now need to set the partition types to the LVM value of 8e. Partitions /dev/hde5 and /dev/hdf1 are the fifth and sixth partitions on disk /dev/hde. Modify their type using the t command, and then specify the partition number and type code. You can also use the L command to get a full listing of ID types in case you forget.

Command (m for help): t
Partition number (1-6): 5
Hex code (type L to list codes): 8e
Changed system type of partition 5 to 8e (Linux LVM)
 
Command (m for help): t
Partition number (1-6): 6
Hex code (type L to list codes): 8e
Changed system type of partition 6 to 8e (Linux LVM)
 
Command (m for help):

Make Sure The Change Occurred

Use the p command to get the new proposed partition table.

Command (m for help): p
 
Disk /dev/hde: 4311 MB, 4311982080 bytes
16 heads, 63 sectors/track, 8355 cylinders
Units = cylinders of 1008 * 512 = 516096 bytes
 
   Device Boot    Start       End    Blocks   Id  System
/dev/hde1             1      4088   2060320+  fd  Linux raid autodetect
/dev/hde2          4089      5713    819000   83  Linux
/dev/hde3          5714      6607    450576   83  Linux
/dev/hde4          6608      8355    880992    5  Extended
/dev/hde5          6608      7500    450040+  8e  Linux LVM
 
Command (m for help):

Save The Partition Changes

Use the w command to permanently save the changes to disk /dev/hde.

Command (m for help): w
The partition table has been altered!
 
Calling ioctl() to re-read partition table.
 
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
sh-2.05b#

The error above will occur if any of the other partitions on the disk is mounted. This shouldn't be grave as you are already in single user mode in which most of the system's processes that would be accessing the partition have been shutdown.

Define Each Physical Volume

After modifying the partition tables of /dev/hde and /dev/hdf, initialize the target partitions with the pvcreate command. This wipes out all the data on them in preparation for the next step. If you haven't backed up your data yet, do it now!

sh-2.05b# pvcreate /dev/hde5 /dev/hdf1
pvcreate -- physical volume "/dev/hde5" successfully created
pvcreate -- physical volume "/dev/hdf1" successfully created
sh-2.05b#

Create A Volume Group For the PVs

Use the vgcreate command to combine the two physical volumes into a single unit called a volume group. The LVM software effectively tricks the operating system into thinking the volume group is a new hard disk. In the example, the volume group is called lvm-hde.

sh-2.05b# vgcreate lvm-hde /dev/hdf1 /dev/hde5
Volume group "lvm-hde" successfully created
sh-2.05b#

Therefore, the vgcreate syntax uses the name of the volume group as the first argument followed by the partitions that it will be comprised of as all subsequent arguments.

Run VGscan

The next step is to verify that Linux can find your new LVM disk partitions. To do this, use the vgscan command.

sh-2.05b# vgscan
vgscan -- reading all physical volumes (this may take a while...)
Found volume group "lvm-hde" using metadata type lvm2
sh-2.05b#

Create A Logical Volume From The Volume Group

Now you're ready to partition the volume group into logical volumes with the lvcreate command. Like hard disks, which are divided into blocks of data, logical volumes are divided into units called physical extents (PEs).

You'll have to know the number of available PEs before creating the logical volume. This is done with the vgdisplay command using the new lvm-hde volume group as the argument.

sh-2.05b# vgdisplay lvm-hde
--- Volume group ---
VG Name               lvm-hde
VG Access             read/write
VG Status             available/resizable
VG #                  0
MAX LV                256
Cur LV                0
Open LV               0
MAX LV Size           255.99 GB
Max PV                256
Cur PV                2
Act PV                2
VG Size               848 MB
PE Size               4 MB
Total PE              212
Alloc PE / Size       0 / 0
Free  PE / Size       212 / 848 MB
VG UUID               W7bgLB-lAFW-wtKi-wZET-jDJF-8VYD-snUaSZ
 
sh-2.05b#

As you can see, 212 PEs are available as free. You can now use all 212 of them to create a logical volume named lvm0 from volume group lvm-hde.

sh-2.05b# lvcreate -l 212 lvm-hde -n lvm0
Logical volume "lvm0" created
sh-2.05b#

Note: You can also define percentages of the volume group to be used. The first example defines the use of 100% of the volume group's free space and the second example specifies using 50% of the total volume group.

sh-2.05b# lvcreate -l 100%FREE -n lvm0 lvm-hde
sh-2.05b# lvcreate -l 50%VG -n lvm0 lvm-hde

Format The Logical Volume

After the logical volume is created, you can format it as if it were a regular partition. In this case, use the -t switch to specify to the mkfs formatting program that you want a type ext3 partition.

sh-2.05b# mkfs -t ext3 /dev/lvm-hde/lvm0
mke2fs 1.32 (09-Nov-2002)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
108640 inodes, 217088 blocks
10854 blocks (5.00%) reserved for the super user
First data block=0
7 block groups
32768 blocks per group, 32768 fragments per group
15520 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840
 
Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done
 
This filesystem will be automatically checked every 38 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
sh-2.05b#

Create A Mount Point

When you formatted the /dev/hde5 partition, you lost the /home directory. Now you have to recreate /home on which you'll later mount your new logical volume.

sh-2.05b# mkdir /home

Update The /etc/fstab File

The /etc/fstab file lists all the partitions that need to be automatically mounted when the system boots. This snippet configures the newly labeled partition to be mounted on the /home mount point.

/dev/lvm-hde/lvm0   /home      ext3    defaults        1 2

The /dev/hde5 and /dev/hdf1 partitions are replaced by the combined /lvm0 logical volume. You, therefore, don't want the old partitions to be mounted again. Make sure that any reference to them in this file has either been commented a # character at the beginning of each line or deleted entirely.

#/dev/hde5       /data1        ext3    defaults        1 2
#/dev/hdf1       /data2        ext3    defaults        1 2

Mount The Volume

The mount -a command reads the /etc/fstab file and mounts all the devices that haven't been mounted already. After mounting, test the volume by listing its directory contents. It should just contain the lost+found directory

sh-2.05b# mount -a
sh-2.05b# ls /home
lost+found
sh-2.05b#

Restore Your Data

You can now restore your backed up data to /home.

Get Out Of Single User Mode

Return to your original run state by using either the init 3 or init 5 commands. The exit command will make you return to your default runlevel.

Conclusion

The demise of the hard disk has been predicted for many years. Faster, denser memory chips were supposed to eliminate their need, but hard disk technology has evolved, dramatically increasing their speed and capacity too. They will be around for a long time to come.

It seems as if when drives get bigger, so does the data they are intended to store. Expanding the existing disk capacity of your server may become an everyday occurrence and the tools described in this chapter should make the task easier.