I am planning scheduled downtime on May 03, 2014 at 5:00 AM CST.

I’ve installed a new 4TB hard drive, and incorporated it into the RAID 1 mirror. Using the two older 2TB drives, I’ve striped together their main partition into a ~3.5 TiB block, onto which I mirrored the 4TB drive. This was all done live, transparently to my users, without data loss. However the RAID mirror size and filesystem for the home directory is still only 1.7TiB.

During the downtime, I plan to grow the RAID mirror and ext4 filesystem to fill the full 3.5 TiB. I will try to do this live first, however it is unlikely to work and I will have to momentarily dismount the array in order to make the necessary changes. I will disconnect the network during this time in order to delay the receipt of any email. I will also take the opportunity to run a filesystem integrity check, do a cleaning of the physical hardware, and reboot for software upgrades and to ensure service integrity.

Though I expect no serious problems, I also have a backup on hand of the system state before the 4TB drive was installed, just in case primary data becomes irrecoverable.

This change will double space on the server, and possibly slightly increase IOPS. User quotas will remain the same; I’ll be using the space for snapshot/backup and archival purposes.

 

** Update 1: I was able to conduct an online raid array and filesystem resize without any problems. I’ve decided to shut down the server anyway for the aforementioned maintenance tasks. If I configured the bootloader correctly, this should only take a few minutes.

The reboot will update the system to the latest kernel with enabled support for ACLs and certain iptables modules. ACLs will allow me to enhance user security by tightening internal restrictions that might otherwise have allowed users to read each others’ files, while still allowing access from critical system services. The iptables modules will allow me to apply block lists that will (further) hamper some of the more nefarious botnets in their attempts to get user passwords. The kernel has also been updated with some experimental filesystems, one of which I may implement at a later date.

 

**Update 2: Had some problems with the raid configuration not being correctly applied on boot. I am looking into this issue and will schedule future reboots for testing.