Encrypt your /home directory using LUKS and a spare disk

Every year or two, I rotate a drive out of my NAS.  My most recent rotation yielded me with a spare 1TB SSD.  My main machine only had a 250GB SSD, so I figured that I'd just replace my /home directory with a mountpoint on that new disk, giving my lots of space for video editing and such, since I no longer had the room to deal with my GoPro footage.

My general thought process was as follows:

1. I don't want to mess too much with my system.
2. I don't want to clone my whole system onto the new drive.
3. I want to encrypt my personal data.
4. I don't really care about encrypting the entire OS.

I had originally looked into some other encryption options, such as encrypting each user's home directory separately, but even in the year 2019 there seemed to be too much drama dealing with that (anytime that I need to make a PAM change, it's a bad day).  Using LUKS, the disk (well, partition) is encrypted, so everything kind of comes for free after that.

If you register the partition in /etc/crypttab, your machine will prompt you for the decryption key when it boots (at least Kubuntu 18.04 does).

One other thing: dealing with encrypted data may be slow if your processor doesn't support AES encryption.  Do a quick check and make sure that "aes" is listed under "Flags":

lscpu;

If "aes" is there, then you're good to go.  If not, then maybe run some tests to see how much CPU overhead disk operations use on LUKS (you can follow this guide, but stop before "Home setup, phase 2", and see if your overhead is acceptable).

The plan

1. Luks Setup
1. Format the new disk with a single partition.
2. Set up LUKS on that partition.
3. Back up the LUKS header data.
2. Home setup, phase 1
1. Copy everything in /home to the new partition.
2. Update /etc/crypttab.
3. Update /etc/fstab using a test directory.
4. Reboot.
5. Test.
3. Home setup, phase 2
1. Update /etc/fstab using the /home directory.
2. Reboot.
3. Test.

LUKS setup

Wipe the new disk and make a single partition.  For the remainder of this post, I'll be assuming that the partition is /dev/sdx1.

Install "cryptsetup".

sudo apt install cryptsetup; 

Set up LUKS on the partition.  You'll need to give it a passphrase.  I recommend something that's easy to type, like a series of four random words, but you do you).  You'll have to type this passphrase every time that you boot your machine up.

sudo cryptsetup --verify-passphrase luksFormat /dev/sdx1;

Once that's done, you can give it some more (up to 8) passphrases.  This may be helpful if you want to have other people access the disk, or if you just want to have some backups, just in case.  If there are multiple passphrases, any one of them will work fine; you don't need to have multiple on hand.

sudo cryptsetup --verify-passphrase luksAddKey /dev/sdx1;

The next step is to "open" the partition.  The last argument ("encrypted-home") is the name to use for the partition that will appear under "/dev/mapper".

sudo cryptsetup luksOpen /dev/sdx1 encrypted-home;

At this point, everything is set up and ready ready.  Confirm that with the "status" command.

sudo cryptsetup status encrypted-home;

Back up the LUKS header data.  If this information gets corrupted on the disk, then there is no way to recover your data.  Note that if you recover data using the header backup, then the passphrases will be the ones in the header backup, not whatever was on the disk at the time of the recovery.

sudo cryptsetup luksHeaderBackup /dev/sdx1 --header-backup-file /root/luks.encrypted-home.header;

I put mine in the /root folder (which will not be on the encrypted home partition), and I also backed it up to Google Drive.  Remember, if you add, change, or delete passphrases, you'll want to do make another backup (otherwise, those changes won't be present during a restoration operation).

If you're really hardcore, fill up the partition with random data so that no part of it looks special.  Remember, the whole point of encryption is to make it so that whatever you wrote just ends up looking random, so writing a bunch of zeros with "dd" will do the trick:

sudo dd if=/dev/zero of=/dev/mapper/encrypted-home;

Before you can do anything with it, you'll need to format the partition.  I used EXT4 because everything else on this machine is EXT4.

sudo mkfs.ext4 /dev/mapper/encrypted-home;

Home setup, phase 1

Once the LUKS partition is all set up, the next set of steps is just a careful copy operation, tweaking a couple /etc files, and verifying that everything worked.

The safest thing to do would be to switch to a live CD here so that you're guaranteed to not be messing with your /home directory, but I just logged out of my window manager and did the next set of steps in the ctrl+alt+f2 terminal.  Again, you do you.

Mount the encrypted home directory somewhere where we can access it.

sudo mkdir /mnt/encrypted-home; 

sudo mount /dev/mapper-encrypted-home /mnt/encrypted-home;

Copy over everything in /home.  This could take a while.

sudo cp -a /home/. /mnt/encrypted-home/;

Make sure that /mnt/encrypted-home contains the home folders of your users.

Set up /etc/crypttab.  The format is:

${/dev/mapper name} UUID="${disk uuid}" none luks

In our case, the /dev/mapper name is going to be "encrypted-home".  To find the UUID, run:

sudo blkid /dev/sdx1;

So, in my particular case, /etc/crypttab looks like:

encrypted-home UUID="5e01cb97-ceed-40da-aec4-5f75b025ed4a" none luks

Finally, tell /etc/fstab to mount the partition to our /mnt/encrypted-home directory.  We don't want to clobber /home until we know that everything works.

Update /etc/fstab and add:

/dev/mapper/encrypted-home /mnt/encrypted-home ext4 defaults 0 0

Reboot your machine.

When it comes back up, it should ask you for the passphrase for the encrypted-home partition.  Give it one of the passphrases that you set up.

Log in and check /mnt/encrypted-home.  As long as everything's in there that's supposed to be in there (that is, all of your /home data), then phase 1 is complete.

Home setup, phase 2

Now that we know everything works, the next step is to clean up your actual /home directory and then tell /etc/fstab to mount /dev/mapper/encrypted-home at /home.

I didn't want to completely purge my /home directory; instead, I deleted everything large and/or personal in there (leaving my bash profile, some app settings, etc.).  This way, if my new disk failed or if I wanted to use my computer without it for some reason, then I'd at least have normal, functioning user accounts.  Again, you do you.  I've screwed up enough stuff in my time to like to have somewhat nice failback scenarios ready to go.

Update /etc/fstab and change /dev/mapper/encrypted-home line to mount to /home.

/dev/mapper/encrypted-home /home ext4 defaults 0 0

Reboot.

When it comes back up, it should ask you for the passphrase for the encrypted-home partition.  Give it one of the passphrases that you set up.

Log in.  You should now be using an encrypted home directory.  Yay.

To confirm, check your mountpoints:

mount | grep /home

You should see something like:

/dev/mapper/encrypted-home on /home type ext4 (rw,relatime,data=ordered)

Now that everything's working, you can get rid of "/mnt/encrypted-home"; we're not using it anymore.

sudo rmdir /mnt/encrypted-home;

Google App Engine cron jobs run as no user

Google App Engine (GAE) is a great platform to use when developing just about any new application.  I use it for a couple personal projects, as well as for some fire department apps (dispatch and inventory management).

Today I'd like to talk about two things: users and cron jobs.

Users

One of the really convenient things about GAE is that it has built-in support for Google authentication (no surprise there).  This means that you can let GAE take care of your sign-in system (and trust me, handling the single-sign-on 3-way handshake isn't all that fun).  With GAE, you easily go one of two routes:

1. Certain paths in your web application are automatically required to have a logged-in user.  If someone tries to access a path without being logged in, GAE will redirect him to a sign-in window and then bring him back when he's done.
2. Server-side, you can ask GAE for log-in and log-out URLs, and you can direct a user to these at any time to have him log in or out.

I personally prefer the second route because there's no strange redirection and all of my endpoints behave as I expect them to.  For example, if I have a JSON REST API, a user (who has not signed in yet) can access the endpoint and be given a normal 400-level error (in JSON) instead of being redirected to a Google sign-in page.  My REST clients much prefer this.

To see if a user is signed in, a Java Servlet can call the "getCurrentUser()" method of the "UserService" instance.  If the result is null, then no one is logged in.  Otherwise, you get a couple of helpful methods to tell you about the user:

1. "getEmail()"; this returns the user's e-mail address.
2. "getUserId()"; this returns a unique numeric ID for the user.
3. "isUserAdmin()"; this returns whether or not the user is an administrator of the GAE application.

For my apps, user authentication is whitelist style.  I check the "@" portion of the e-mail address to see if the user is in one of the domains that I care about (I typically build apps internal to an organization that uses Google's mail system), and I check the administrator status to grant administrator powers to my admins.

If someone tries to access a sensitive API endpoint without being logged in appropriately, I'll send back a 400-level error stating that the user is not signed in with an appropriate account.

Pretty easy stuff.

UserService userService = UserServiceFactory.getUserService();

User user = userService.getCurrentUser();

if( user == null ) {

   // There is no user logged in.

} else {

   // The user is logged in.

   System.out.println( "User is logged in: " + userService.isUserLoggedIn() );

   System.out.println( "User is administrator: " + userService.isUserAdmin() );

   System.out.println( "User:" );

   System.out.println( "   Auth Domain: " + user.getAuthDomain() );

   System.out.println( "   E-mail: " + user.getEmail() );

   System.out.println( "   Federated ID: " + user.getFederatedIdentity() );

   System.out.println( "   Nickname: " + user.getNickname() );

   System.out.println( "   User ID: " + user.getUserId() );
}

Warning: you cannot call "userService.isUserAdmin()" if the user is not already logged in.  If you try to, then it will throw an exception.

Cron Jobs

Another thing that GAE can do is schedule cron jobs.  Basically, these are page requests that are scheduled like normal Linux "cron" jobs.  So if you need to have some task performed regularly, create an endpoint for it and schedule a job to access that endpoint.

Cron jobs act as if an administrator is making the request, so they can access all paths with "admin" requirements.  Howerver, you cannot check this using "userService.isUserAdmin()" because cron jobs do not run as any particular user.

To determine if a request is coming from the cron scheduler, you have to check for the "X-Appengine-Cron" header.  This header cannot be faked (except by admins); if you try to set this header, GAE will quietly remove it by the time that it gets to your Servlet.

To detect a cron job, you have to check for the header and make sure that its value is "true".

String cronHeader = request.getHeader("X-Appengine-Cron");

if( cronHeader != null && cronHeader.compareTo("true") == 0 ) {

   log.info( "Cron service is making this request." );

}

Ultimately, if I'm checking to see whether a user is allowed to access a particular section, I go through these steps:

1. (Assume no access at all.)
2. No user is logged in.
1. Is the "X-Appengine-Cron" header set to "true"?  If so, then allow administrative access.
3. A user is logged in.
1. Is the user a GAE admin of the application?  If so, then allow administrative access.
2. Is the user's e-mail domain in the whitelist of basic user access?  If so, then allow basic access.