Devin Reade

GNO Consortium | Home | GNO/ME

Public Projects | System Administration | General Computing | Parallel Computing | Compilers | Linux

Cryptography and Security | Physics Resources

Other Pages of Interest

Up


Encrypted Root and Swap with LUKS and LVM on CentOS 5 (5.1, 5.2, 5.3)

UPDATE: Note that as of CentOS 5.4, encrypted filesystems are now supported in the default install.

This document shows an example of how to set up a CentOS 5 machine with encrypted root and swap, in a fashion that allows suspend and hibernate to work correctly while still maintaining the cryptographic integrity of the system.

This configuration is ideal for laptops, but may be applied to any machine.

Before continuing, please read the introductory page, which also deals with a server-type configuration where only some of the filesystems are encrypted.

Contents


Pre-Installation Partitioning

Here we assume that software RAID is not available, and that we've instead got a single disk, /dev/hda, with which to work.

While we will be using the regular installation procedure, we don't want to use the default partitioning tools as they do not provide sufficient control over partition placement. Instead we will use fdisk.

You can get at fdisk either by booting the installation DVD in rescue mode ('linux rescue' at the prompt) or, when you get to the first CentOS installation screen (after the language and locale selection) type CTRL-ALT-F2 to get to a text prompt. If you use rescue mode, skip mounting of any partitions.

Using fdisk, create three partitions:

If you're reusing an old disk, clobber any filesystem header data that may be in place:

  dd if=/dev/zero of=/dev/hda2 bs=1024 count=2
  dd if=/dev/zero of=/dev/hda4 bs=1024 count=2

If you used the regular installer, you can now return to it via CTRL-ALT-F6. If you used the rescue disk, reboot to the installer.


OS Installation (Unencrypted)

Proceed with the installer until you get to the partitioning screen. Select "Create custom layout" and hit "Next".

You should see the three partitions you created before running the installer:

  1. /dev/hda1: Configure this partition to be of type ext3 and mounted as /boot.
  2. /dev/hda2: Leave this partition as-is for now.
  3. /dev/hda4: On this partition, place a volume group. Use a volume group name that will eventually be thrown away. We will assume the volume group name is vg2.
We will create volume group vg1 later for our encrypted data.

Next, create your filesystems as logical volumes under volume group vg2. Use whatever allocation scheme you want, but I prefer separate filesystems:

VolumeMount PointSize
swap(none)2 GB
root/1 GB
usr/usr8 GB
var/var2 GB
home/home512 MB

We'll make /home larger later. 2GB is a good size for /var as it allows lots of space for OS updates.

I'd suggest using a boot loader password. This should not be the same password that you're going to use for the disk encryption, later.

Select your sets or packages of software and perform the install. Reboot when prompted.

During the "final" configuration, you will be prompted for the SELinux setting, which is by default enabled. These instructions have only been tested with SELinux disabled. Given the history of SELinux, if you have it enabled you will probably have to do some additional configuration.

Reboot if the installation script requires you to do so.


Initial Configuration Changes

Before continuing, there are a couple of things that I'd recommend. The first is to disable any automatic updates while we're getting the encrypted setup working:

   service yum-updatesd stop
   chkconfig yum-updatesd off

The second is to use a text boot sequence so that we can see any interesting information go by. If you want to, you can reverse this later to get the default graphical boot sequence, but I leave it as-is. Note that even if you perform these steps, the system still boots into run level 5, meaning that you'll have your graphical window manager when it's time to log in:

  1. Modify /etc/sysconfig/init changing GRAPHICAL=yes to GRAPHICAL=no
  2. Modify /boot/grub/grub.conf (or the symlink /etc/grub.conf) thus:
    1. Comment out the splashimage line.
    2. Comment out the hiddenmenu line.
    3. Delete the word quiet from the kernel arguments line.

Update mkinitrd

At the time of this writing (CentOS 5.1), there is a mkinitrd bug that can cause kernel updates to fail. It is possible, by the time that you are reading this, that the bug has been fixed and propagated. UPDATE: This seems to have been fixed CentOS 5.3 circa early 2009.

Therefore, first update mkinitrd to its current version before we start:

The URL in this block seems to be dead; see the following comments

Next, make sure that your version of mkinitrd is at least 5.1.19.6-28.1. If it is not after performing the above yum update, then upgrade it manually. (You must also upgrade nash.) The following commands are for 32-bit installations; modify them appropriately for 64-bit installations:

  • wget http://people.redhat.com/pjones/mkinitrd/RHEL-5/i386/mkinitrd-5.1.19.6-28.1.i386.rpm
  • wget http://people.redhat.com/pjones/mkinitrd/RHEL-5/i386/nash-5.1.19.6-28.1.i386.rpm
  • rpm -Uvh mkinitrd-5.1.19.6-28.1.i386.rpm nash-5.1.19.6-28.1.i386.rpm

[17 Nov 2008] The above URL seems to be a dead link. If you need the fix and it is not yet in your mkinitrd, you can apply my mkinitrd-slaves patch.


Disk Preparation

This next step is very time consuming, but important. In order to avoid certain types of cryptanalysis on your soon-to-be-encrypted disk, it should be filled with random data. We'll do this in two parts; one now, one later.

[Optional] If your disk previously had data on it that is of a sensitive nature, you can also first shred it, which helps to protect you from having someone get your disk and read your old data off of it via hysteresis effects. In this context, shredding is an repeated overwrite procedure. See Data Remanence. There is no need to shred the disk if it is new or if the old contents are not sensitive (you still need to randomize it). Shredding may seem paranoid, but you're reading this, aren't you? ;)

The preparation consists of:

  1. [Optional] Shred the currently unused device, that will later hold the encrypted data.
  2. Randomize the same device

The specific commands are:

  1. shred -v /dev/hda2
  2. dd if=/dev/urandom of=/dev/hda2 bs=1024

Do not reverse the two steps above; the order is important, otherwise you nullify the effect of the second one.

Do not use /dev/random as you will empty your entropy pool quickly and possibly cause your system to hang. Use /dev/urandom instead.

The one or two commands above can take a LONG time. Having your computer run them overnight is a good idea. They might still be running when you wake up in the morning. If you want to run both, then you can chain them with a semicolon:

shred -v /dev/hda2; dd if=/dev/urandom of=/dev/hda2 bs=1024

My laptop (IBM Thinkpad T42) performs the dd step at about 3.4 MB/s, so my 72GB /dev/hda2 partition takes almost 6 hours to randomize. The time to perform shredding is comparable.

If you want to see the progress of dd, then you can (from another window or terminal) send it a SIGUSR1. If you don't know how to do this, look at the ps(1) and kill(1) man pages. If you don't know what a man page is, you probably shouldn't be doing this ...


Set Up Encrypted Volumes

Whew! Now that you have your target partition scrubbed and randomized, it's time to get to the meat of the issue. We will now set up the LVM configuration that will be used in production.

Naming Scheme

Since we are setting up a volume group over top of an encrypted device, I use the following naming scheme: The decrypted device base name uses the same numerical identifier as does the volume group name, and has all of its letters in upper case. Volume group names use lower case (and numbers and hyphens). For example, if I have a volume group vg1 which, when encrypted, resides on /dev/md1, then:

  1. The volume group name is vg1 and we will see it in the filesystem as /dev/vg1/....
  2. The volume group vg1 will use the "physical volume" /dev/mapper/PV1. (If this were a server with multiple physical volumes per volume group, then I'd use PV1A, PV1B, or vg01 with PV01A, PV01B, etc)
  3. The above "physical volume" is actually the unencrypted virtual device that is based on the encrypted device /dev/hda2.

Edit lvm.conf

Now you should edit the /etc/lvm/lvm.conf file so that pvscan and vgscan can pick up the soon-to-be-encrypted device. Edit it to have the following line (or apply this patch):

   types = [ "device-mapper", 16 ]
There's already a prototype line for types, so put it near that. Also in /etc/lvm/lvm.conf, you should to edit the filter line to be:
   filter = [ "a|^/dev/mapper/[A-Z].*|", "r|^/dev/mapper/.*|", "a/.*/" ]

Create the Encrypted Volume

Now we're ready to create the encrypted device. We use LUKS for this. Pick a strong passphrase, but one that you're going to remember, because you'll be typing it a lot -- every time you boot your computer:

   cryptsetup -c aes-cbc-essiv:sha256 luksFormat /dev/hda2
You will be prompted for the passphrase, twice.

Now before you go any further, make a copy of that passphrase somewhere secure. Write it on a piece of paper, seal it in an envelope, and lock it up in your safe. Make another copy of it, seal it, and put it in your safety deposit box. If you lose it, all your data is gone forever; there is no magical passphrase recovery mechanism, no back doors. (However you can have more than one passphrase that unlocks your data; see the LUKS documentation for details.)

Next, open the device as PV1, which will be the physical volume for volume group vg1:

   cryptsetup luksOpen /dev/hda2 PV1
We can then create the physical volume and volume group:
   pvcreate /dev/mapper/PV1
   vgcreate vg1 /dev/mapper/PV1
Followed by the logical volumes and file systems:
   lvcreate -L1G -n root vg1
   lvcreate -L10G -n usr vg1
   lvcreate -L10G -n home vg1
   lvcreate -L2G -n var vg1

   mke2fs -j /dev/vg1/root
   mke2fs -j /dev/vg1/usr
   mke2fs -j /dev/vg1/home
   mke2fs -j /dev/vg1/var
And then the swap:
   lvcreate -L2G -n swap vg1
   mkswap /dev/vg1/swap


Get a Backup Boot Configuration

Before going any further, it is time to get a backup boot configuration. This is a copy of your current initrd, booting the unencrypted partition.

  1. Make a copy of your existing initial ramdisk (initrd):
        cp -p /boot/initrd-`uname -r`.img /boot/initrd-`uname -r`.img.old
    
  2. Edit /boot/grub/grub.conf and copy the four or so lines of one boot entry. Change the title line to be different, and change the initrd filename to have a suffix of ".old". Also, for the first entry, change the root device from /dev/vg2/root to /dev/vg1/root. For example, change:
        title CentOS (2.6.18-53.el5)
            root (hd0,0)
            kernel /vmlinuz-2.6.18-53.el5 ro root=/dev/vg2/root rhgb
            initrd /initrd-2.6.18-53.el5.img
    
    to:
        title CentOS (2.6.18-53.el5)
            root (hd0,0)
            kernel /vmlinuz-2.6.18-53.el5 ro root=/dev/vg1/root rhgb
            initrd /initrd-2.6.18-53.el5.img
    
        title CentOS (2.6.18-53.el5) (OLD)
            root (hd0,0)
            kernel /vmlinuz-2.6.18-53.el5 ro root=/dev/vg2/root rhgb
            initrd /initrd-2.6.18-53.el5.img.old
    
    Keep the backup entry at the end of the file; the first (default) entry should be the one using the encrypted disk.

Customize mkinitrd Script

Here's the really only annoying part of this situation; it is necessary to customize the /sbin/mkinitrd script so that it unlocks the encrypted device soon enough in the boot sequence. A patch is available to do this, and is shown here:

--- mkinitrd.orig       2007-11-10 18:57:35.000000000 -0700
+++ mkinitrd    2008-01-09 23:31:15.000000000 -0700
@@ -1105,6 +1105,7 @@
 fi
 inst /sbin/nash "$MNTIMAGE/bin/nash"
 inst /sbin/insmod.static "$MNTIMAGE/bin/insmod"
+inst /sbin/cryptsetup "$MNTIMAGE/bin/cryptsetup"
 ln -s /sbin/nash $MNTIMAGE/sbin/modprobe

 for MODULE in $MODULES; do
@@ -1351,6 +1352,11 @@
     done
 fi

+if [ -n "$CRYPTO_ROOT_DEV" -a -n "$CRYPTO_ROOT_MAP" ]; then
+    emit "echo Decrypting root device"
+    emit "cryptsetup luksOpen '$CRYPTO_ROOT_DEV' '$CRYPTO_ROOT_MAP'"
+fi
+
 if [ -n "$vg_list" ]; then
     emit "echo Scanning logical volumes"
     emit "lvm vgscan --ignorelockingfailure"

To control this script, create the file /etc/sysconfig/mkinitrd/crypto-root and put the following in it, modified suitably for your system:

MODULES="$MODULES aes sha256 dm_crypt"
# Do NOT use the /dev/disk/by-* hierarchies here
CRYPTO_ROOT_DEV="/dev/hda2"
CRYPTO_ROOT_MAP="PV1"
And then set the executable bits. This is critical, otherwise the initrd will be unable to decrypt your root partition:
    chmod 755 /etc/sysconfig/mkinitrd/crypto-root

Some Technical Details

As described in the file comment above, one cannot use the /dev/disk/by-* symlinks to identify the device to decrypt. This is the piece that is sensitive to device renaming by the kernel, as described earlier in this document. I haven't seen a kernel rename a root device before, but I also don't know what the criteria are, although I suspect that it can only happen with non-root devices. Therefore this is an area that you should watch for when doing your initial tests.

Note earlier that we're putting all of our If we didn't include the swap volume in the same physical volume as the


Copy Filesystems and Create Boot Image

At this point in time you have your encrypted filesystems ready but empty, and your original installation on unencrypted filesystems. Here we copy all the files to the encrypted filesystems, and configure the system to boot from them.

  1. Reboot into single user mode using your backup configuration. (The default one is not currently valid after you edited the grub.conf file.)
  2. Unlock and activate the encrypted volume:
       cryptsetup luksOpen /dev/hda2 PV1
       vgchange -ay vg1
    
  3. Copy over the root filesystem:
       mount /dev/vg1/root /mnt
       cd /mnt
       dump -0af - / | restore -rf -
       rm restoresymtable
    
  4. Copy over the /usr filesystem:
       mount /dev/vg1/usr /mnt/usr
       cd /mnt/usr
       dump -0af - /usr | restore -rf -   
       rm restoresymtable
    
  5. Copy over the /var filesystem:
       mount /dev/vg1/var /mnt/var
       cd /mnt/var
       dump -0af - /var | restore -rf -   
       rm restoresymtable
    
  6. Copy over any remaining filesystems, such as /home:
       mount /dev/vg1/home /mnt/home
       cd /mnt/home
       dump -0af - /home | restore -rf -   
       rm restoresymtable
    
  7. Edit the /mnt/etc/fstab (NOT /etc/fstab) file and change all occurances of vg2 to vg1.
  8. Create the new initial ramdisk (initrd):
       rm /boot/initrd-`uname -r`.img
       mkinitrd --fstab=/mnt/etc/fstab /boot/initrd-`uname -r`.img `uname -r`
    
    This takes a few seconds to complete. Note the --fstab flag and argument. You're creating an initrd that boots your encrypted filesystems, so it's critical that you get the correct mount points.
  9. You can now unmount the encrypted filesystems:
       cd /
       umount /mnt/home
       umount /mnt/var
       umount /mnt/usr
       umount /mnt
    

Reenable Automatic Updates

After everything is sorted out, you should reenable automatic updates. While normally I would suggest doing the opposite of the disabling process, there was a BIG problem with updates when yum-updatesd was introduced. See the RedHat bug report for details. This was reported in FC6, but it also affected RHEL5, thus CentOS 5.0 and 5.1.

An alternative to reenabling the official (and broken) mechanism is to do what was done before yum-updatesd was introduced, namely invoke it as an overnight cron job. A script to do this is available here and can be installed like this:

    cd /etc/cron.daily
    wget ftp://ftp.gno.org/pub/tools/centos5.2/0yum-custom
    chmod 755 0yum-custom


[OPTIONAL] Move /tmp to tmpfs

In order to make access to /tmp faster and to keep it cleaned out between reboots, I like to put /tmp on tmpfs.

Regardless of whether or not you choose to perform this step, the data in /tmp (or the data that was in /tmp) will be encrypted whenever the machine is off, hibernating, or suspended. (That is, assuming that you have performed the other steps described in this document.)

  1. Reboot into single user mode.
  2. Add the following entry to /etc/fstab:
       tmpfs       /tmp       tmpfs     defaults     0 0
    
    If you want to limit the amount of space used by /tmp, you can specify it with the size option:
       tmpfs       /tmp       tmpfs     size=512m    0 0
    
  3. Delete all the files in /tmp, including hidden files (ls -a).
  4. Reboot.

[OPTIONAL] Disable Periodic Filesystem Checks

Since a journalling file system was used above, you may optionally choose to disable the periodic filesystem checks. There are differing opinions on whether this is a good or bad thing to do. Theoretically, a journalled filesystem shouldn't need a periodic check and disabling it can avoid the occasional long boot time while a check is conducted (especially annoying when running on battery). However, some people are uncomfortable with the idea of never running fsck on a filesystem unless it is marked as dirty. It's up to you.

Do disable the checks, you would use tune2fs:

   tune2fs -i0 -c0 /dev/vg0/root
   tune2fs -i0 -c0 /dev/vg0/usr
   tune2fs -i0 -c0 /dev/vg0/var
   tune2fs -i0 -c0 /dev/vg0/home


Testing

Reboot, using the default configuration. Verify that:

  1. You should see a Enter LUKS passphrase: prompt shortly after the the Waiting for driver initialization kernel message.
  2. After entering your passphrase the system should boot up normally.
  3. As root, do a 'df' and verify that all "regular" filesystems are mounted from the vg1 volume group, with the exception of /boot.
  4. As root, do a 'swapon -s' and verify that the swap in use is from vg1.
  5. Log in as a regular user. (If you didn't create one during the installation, you should create one now.) After your window manager comes up, you should verify that the following work as you expect: In KDE, these functions are available by right-clicking on the panel's power/battery icon.

Watch for Changes to mkinitrd

The biggest problem with this procedure is that we're using a modified mkinitrd script, which has the potential to be replaced during a system update (ie: automated patch installation). The new version of mkinitrd would of course not know anything about our special boot requirements.

I have written a check-mkinitrd script that you can download and place into /etc/cron.hourly. Assuming that your root mail is going somewhere where it's being read, this will notify you if there has been a change to mkinitrd. The script doesn't fix the problem, but it lets you know that you need to repatch mkinitrd.

If for some reason you didn't notice a mkinitrd update before you rebooted, try your older configuration (as left by mkinitrd when the kernel was updated). It may allow you to boot and fix the problem.


Recover Unencrypted Partition Space

We can now recover the disk space that we used to perform the initial installation. We need to do this in stages.

Remove Unencrypted Volume Group

First we remove the temporary volume group that we created on installation, vg2. However, before we do that, we should remove references to it in the boot mechanism:

  1. Remove the backup ramdisk image in /boot (the one with the .old suffix; don't get the wrong one!)
        rm /boot/initrd-`uname -r`.img.old
    
    or maybe:
        rm /boot/initrd*.old
    
  2. Edit /boot/grub/grub.conf and remove the backup entry (the one that references the file you just deleted).

Now you can remove the logical volumes from vg2 and vg2 itself:

    lvremove vg2
    vgremove vg2
    pvremove /dev/hda4

Randomize Remainder of Disk

Before we recover the space used during the initial installation, you should randomize (and optionally shred) the device as described under Disk Preparation, above. Using the example disk layout, this would be:

    shred -v /dev/hda4
    dd if=/dev/urandom of=/dev/hda4 bs=1024

Change Partition Table

I still need to write this up: Describe how to recover the space used by the unencrypted partition.

Grow vg1 With Recovered Space

I still need to write this up: Describe how to use the space from the partition table change previously described.


Contact Me

Up

Last Updated: 24 Jun 2011