Re-partitioning Xubuntu

Note: if you prefer the bare-essential steps and code, please jump straight to the "The re-partition process" section.

Having used Xubuntu for a while now on my transition test rig, I’ve decided to re-partition the HDD to better suit my needs. Here’s why I’m doing this:

  • Data security – Not encryption, but rather to ensure the data is (most likely) safe on an OS re-installation
  • The default swap partition is not flexible – It always takes up space even when you don’t need it to
  • Makes backup easier – I can simply image the small OS partitions and back them up onto several DVDs

However, the main drawback (and my original reason for not having a separate data partition) is that your HDD’s capacity is effectively reduced (Data = HDD – OS – Swap), and not to mention when your OS partition runs out of your space, you’re boned (alright, I know what you’re thinking, but let’s not get carried away with hacks like creating symbolic links to the data partition…).

First thing first, let’s figure out what the setup should look like and how much space we need, then we can explore different options when we encounter them.

Partition, swap and hibernation config

As much I want to make a complete transition over to Linux, there are some situations where Windows is required (like certain applications and firmware updates), so we need a Windows partition.

The best way to judge how much space is required is to look at the existing space usage of "%SystemRoot%" and "%ProgramFiles%" – you can use a tool like "diruse" (from the Windows XP support tools) or folder properties in explorer.exe…

Windows: In a clean(ish) install of XP SP3, %SystemRoot% + %ProgramFiles% comes to about 2.7 GiB (say, 3GiB), and allowing 1 GiB each (yeah, I know it’s rather generous) for MS Office, IDEs, .NET, JDK/JRE and AutoRoute the minimum size so far comes to about 8 GiB. Adding a 15% defrag free space requirement, overhead for old (un)installation files, system restore files, pagefile.sys, hiberfil.sys, and some free space, the minimum size for the Windows partition comes to 12 GiB (which also fits nicely onto 3 DVDs when imaged).

It is worth point out that if you have a lot of RAM and want to hibernate your Windows session, you’d obviously want more space to accommodate hiberfil.sys. A safe suggestion would be the next multiple of 4, say, 16 GiB (if you have 3 GiB of RAM) (to fit onto 4 DVDs). As for page file, it’s up to your judgement / Windows’s to determine how much is needed – and if you have a second HDD, put the page file there instead.

Linux: As for the Linux partition, it’s a little bit more difficult to estimate (seeing I haven’t really used it for that long) – but base install rounds up to about 3 GiB. For now, I’ve assigned 16 GiB to it as I plan on installing some of the software above using WINE (like MS Office), OpenOffice, IDEs. Though I’d probably use a 12 GiB partition when I partition my main system – but I’ll just have to wait and see how it goes.

As per the Xubuntu default setup, the OS(es) should reside on a primary partition of its own, with the rest being on extended, logical partitions – you can only have up to 4 primary partitions on a volume.

As for swap and hibernation (suspend to disk), I’ve opted for a 512 MiB swap file and a 512 MiB swap partition dedicated for hibernation, for the 512 MiB of RAM (though there are usage problems with this setup – I’ll talk about that next).

The reason behind using a swap file is because it is more flexible (can be resized dynamically) with no reported performance difference under the 2.6 kernel comparing to a swap partition.

As for the swap partition, it is used because there doesn’t seem to be a way to configure Xubuntu’s in-kernel hibernation implementation to use a hibernation file. The main problem with this setup is that hibernation would fail if the entire swap file had been filled and overflowed onto the hibernation swap partition. Unfortunately, swapoff-ing the swap partition will result in a hibernation failure.


The only usable hack around the the filled swap file problem is to run swapoff after boot and resume from hibernation, and swapon just before hibernation. Unfortunately, I was not able to find a way to do swapoff after resuming from hibernation (I’d welcome any suggestions on this!) – I could use upstart on boot for swapoff and modify "/usr/lib/hal/scripts/linux/hal-system-power-hibernate-linux" to run swapon (this is used by GDM (and the logout GUI) to hibernate), but I still need to run swapoff on resume from hibernation… However, saying that, you could modify and use "/etc/acpi/hibernate.sh" to do the swapon and swapoff as the script stops at "echo -n "disk" >/sys/power/state" and resumes execution afterwards on wake.


The only command I haven’t had time to explore is "/usr/sbin/pmi" (called with parameters "action [hibernate|sleep]") – I found this in gdmsetup –> "Edit Commands…".

Update: 2008-05-27 @ 03:09
You can use "pmi" to put your system to sleep / hibernation and be able to do swapon and swapoff at the appropriate time – just do swapon before calling pmi, and swapoff after, since pmi doesn’t detach itself. I’ll post some scripts on how to do this later.

Hacks aside, there are actually solutions to enable proper hibernation to a file – such as TuxOnIce, but it requires patching and re-compilation of the kernel – which is something I don’t plan on doing in this project.

While we’re (kinda) still on the subject on partitions, there is something that can resolve the issues with partition sizes – LVM (Logical Volume Manager). In theory, pretty much everything (except "/boot") can be setup to use LVM, but I’ve decided against it because:

  • My inexperience to Linux – I think it’s best to keep it simple at this stage
  • Compatibility – LVM volumes cannot be read by the "Ext2 Installable File System for Windows" FS driver
  • Reliability – A disk failure or (somehow) a LVM Volume Group corruption would be disastrous
  • Performance – Since the data goes through another abstract layer, there may be issues there

For now, I’ll leave LVM alone until I need a heavy-duty file server with RAID (or equivalent) that runs only on Linux.

File system

So far, we’ve decided on the partition configuration and now, it’s time to choose the file systems.

Since the aim of this project was to re-partition the drive, I’ve decided to stick with Ext3 in Linux as I don’t see any benefits in using another FS for my needs. As for Windows (XP), it’s a no-brainer – NTFS (linux can mount that as ntfs-3g type).

I have chosen Ext3 as the file system for the data partition as I will be using Linux primarily, and Windows XP can use the "Ext2 Installable File System for Windows" to access that partition.

Pre-requisites and tools

We’re now almost at the point where we can start re-partitioning the drive – now that we know what the disk setup is going to be.

We obviously need a re-partitioning tool, but we’d also need something to backup and restore the drive (in case anything horrible were to happen), and somewhere to store the HDD images.

Not long ago, I came across the SystemRescueCd (possibly on Hak5 or this article on linux-mag.com (free subscription required) – can’t remember exactly which…) which demoed the imaging and bare-metal restore capability of the live CD.

As well as the partimage tool on the live CD, it also contains GParted (for re-partitioning) and samba (for smb-based network backups). But there are so many more tools on this CD it is definitely worth keeping in your toolbox.

The re-partition process

Here’s the target partition configurations (for a single HDD setup):

12GiB NTFS (primary):
	(Windows) C:

16GiB ext3 (primary):
	(Xubuntu) /

* (extended):
	[*]GiB ext3 (logical):
	/home/[user]/	# Replace [user] with the users

	[RAM]GiB swap (logical):
	hibernation "swap" partition == RAM size

The essential steps of this project are:

  1. Boot into SystemRescueCd
    1. ifconfig … – If you need network access to store the partition image
    2. mount … – Create a mount point to save the drive image
    3. partimage – Image the existing OS + Data partition and note down the partition sizes
    4. startx – We’ll need a GUI
      1. gparted – Re-size and create partitions

        • IMPORTANT: Note down the existing partition sizes! – in case you need to do a restore
        • IMPORTANT: Make sure you have a backup of the partitions! – go back to step 3 if you don’t
        • IMPORTANT: Keep actions to a minimum! – ALL actions will be replayed in the re-partitioning process!
        • IMPORTANT: Keep actions to a minimum! – Sorry, I cannot stress this enough – For I’ve been burnt by it and had to go through a lengthy restore process…
        • If you screw up, re-create the size of the original partitions and restore image, then try again
    5. shutdown -r 0 – Cross your fingers!
      (note the Xubuntu boot screen will exit with a segfault error, just ignore it for now, will fix later)
  2. login and gain root access
  3. dd if=/dev/zero of=/swap bs=1M count=512 – create a 512 MiB swap at /
  4. mkswap /swap – Configure the swap file – note we’ll edit fstab instead of using swapon to activate it on next boot
  5. mkdir /mnt/h – Make a new mount point
  6. fdisk -l – Note down which partition is for what. For instance:

    Disk /dev/sda: 60.0 GB, 60000000000 bytes
    255 heads, 63 sectors/track, 7294 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    Disk identifier: 0x########
    
       Device Boot      Start         End      Blocks   Id  System
    /dev/sda1               1        2089    16779861   83  Linux
    /dev/sda2            2090        7294    41809162+   5  Extended
    /dev/sda5            7231        7294      514080   82  Linux swap / Solaris
    /dev/sda6            2090        7230    41295019+  83  Linux
    
  7. cd /dev/disk/by-uuid/ && ls -lFa – Take note of uuids and the partitions they point to – we’ll need this next. For instance:

    6db5296a-0bf4-4773-a9f5-22eb1abbd066 -> ../../sda1
    bc3216cf-32f3-49ff-87af-6e2fdb7ce376 -> ../../sda6
    c9e7c3c8-6f49-44be-9cdf-1387ce320b27 -> ../../sda5
    
  8. cd /etc/ && cp fstab fstab.bak && vim /etc/fstab – Backup then edit fstab according to fdisk and ls output, don’t forget to change the "pass" number for disk check – see man fstab for more information. The edited part of fstab should look something like:
    # /etc/fstab: static file system information.
    #
    # <file system> <mount point>   <type>  <options>       <dump>  <pass>
    ...
    # -------- HDD mounts --------
    # /dev/sda1 - avoid mounting it as /c ...
    UUID=6db5296a-0bf4-4773-a9f5-22eb1abbd066       /       ext3    defaults,errors=remount-ro      0       1
    # /dev/sda6 - Home drive - symbolic links are then made from /mnt/h
    UUID=bc3216cf-32f3-49ff-87af-6e2fdb7ce376       /mnt/h  ext3    defaults,errors=remount-ro      0       2
    #
    #
    #
    # -------- Swap --------
    /swap           none            swap    sw,pri=-1       0       0
    # ---- Update: 2008-05-27 @ 03:09 ----
    # You can use "pmi" to do this - you can ignore the swap partition configuration below
    # /dev/sda5 - swap partition for hibernation... - hence pri=-2
    #UUID=c9e7c3c8-6f49-44be-9cdf-1387ce320b27       none    swap    sw,pri=-2       0       0
    

    (Note I’ve left the other stuff like "proc", "/dev/scd*" out in the example above – keep them in fstab!)

    As you can see, I’ve set the hibernation swap partition as "pri=-2" (i.e. lower than the swap file’s) in an attempt to not use it – as mentioned before, I still need a proper workaround for this… – Use "pmi" instead – see update description above.

  9. vim /etc/initramfs-tools/conf.d/resume – Change the UUID to that of the new "swap" partition, e.g.:

    RESUME=UUID=c9e7c3c8-6f49-44be-9cdf-1387ce320b27
    
  10. update-initramfs -u – Update (fixes) the boot screen
  11. shutdown -r 0 – Make sure everything still works and boot screen fixed
  12. Log back in as root
  13. swapon -s – Check the output to ensure there is 1 mounted swap:

    Filename    Type       Size    Used    Priority
    /swap       file       524280  38216   -1
    
  14. cp -r /home /mnt/h/home – Copy the existing home directory
  15. mv /home /home_old – Make way for the new home
  16. cd /mnt/ && chown -R [user]:[group] h – Makes the new user own everything in the data partition (since I’ll be the only one using the data partition for now), then tweak it to your liking…
  17. ln -s /mnt/h/home /home – Link the new home in – make sure you use the full path or you’ll get link errors.

That’s it! Simple eh? lol.


Some links to the interesting articles:

Advertisements

MPlayer resume script

MPlayer logo

I might as well share this – below are 2 scripts I’ve written for MPlayer to resume playback at a position saved in a text file.
(Note the scripts below also attempt to load the "global" profile – I use it for chaining profiles – i.e. I have other profiles that includes "global".)

It works by appending the "-ss" parameter to mplayer when a resume file, in the format "[mediaFileName.ext].txt" is detected.

An example usage (we’ll assume we’re trying to resume a file called "mediaFile.avi" at timecode 300, and the script is named "mp"):

echo 300>mediaFile.avi.txt
mp mediaFile.avi

You’ll probably want to enable the "statusline" display by adding "msglevel=statusline=9" into your mplayer config file (at "%ProgramFiles%/MPlayer/mplayer/config" | "~/.mplayer/config" | "/etc/mplayer/config"), and ensure "quiet=0".

Windows installation:
0 – Get a copy of "grep" and "head" somewhere – such as UnxUtils.
1 – Save script as mp.bat (or something simple to type)
2 – Put into a directory in your %PATH% env. variable.
3 – Play files using the "mp" command, and create resume file as specified in example above.

@echo off
setlocal ENABLEDELAYEDEXPANSION

set mplayerbin=%ProgramFiles%MPlayermplayer.exe
set cli=

FOR /F "usebackq delims==" %%c IN (`echo %*^|grep -c -i " -profile "`) DO set count=%%c
if !count! == 0 (
	set cli="!mplayerbin!" -profile global %*
) else (
	set cli="!mplayerbin!" %*
)

REM Resume stuff - this'd only work if filename is specified as first argument.
FOR /F "usebackq delims==" %%c IN (`echo %*^|grep -c -i "-ss"`) DO set count=%%c
set resumeFile=%1.txt
if exist "!resumeFile!" (
	if !count! == 0 (
		FOR /F "usebackq delims==" %%c IN (`head -n 1 "!resumeFile!"`) DO set pos=%%c
		set cli=!cli! -ss !pos!
	)
)

echo ^>^>^>^> !cli!
!cli!

endlocal

Linux installation:
1 – Save script as mp (or something simple to type)
2 – Put into a directory in your $PATH env. variable.
2 – chmod +x mp
3 – Play files using the "mp" command, and create resume file as specified in example above.

#!/bin/bash

# Script to launch mplayer and resume if [filename.ext].txt exists
# TODO - Integrate the new parser into the old windows script below

mplayerbin=mplayer

# Unfortunately, something simple like: for i in $*; do echo $i; done
# doesn't work with spaces (even when it's escaped) - it'd separate parameters out by space...
# This also assumes the first argument with a space is the filename
argc=$#
i=0
while ((i < argc)); do
	# escape spaces - can't use <code>echo $1|grep " "</code> or it'll display the params...
	if [ "$(echo $1|grep -cE "[ ()$]")" == 1 ]; then	# Any others?
		argv[$i]="$1"
	else
		argv[i]=$1
	fi
	shift; ((i++))
done


cli=${argv[*]}
# Old windows port of the script below
count=$(echo $cli|grep -c -i " -profile ")
if [ $count == 0 ]; then
	cli="$mplayerbin -profile global $cli"
else
	cli="$mplayerbin $cli"
fi

# Resume stuff - this'd only work if filename is specified as first argument.
count=$(echo $cli|grep -c -i "-ss")
resumeFile="${argv[0]}.txt"
# -f ensures the file (not directory) exist
if [ -f "$resumeFile" ]  && [ $count == 0 ]; then
	pos=$(head -n 1 "$resumeFile")
	cli="$cli -ss $pos"
fi
echo >>>> $cli
eval $cli

There is however a bug in the Windows version due to the use of delayed expansion – if you have a file with an exclamation mark, you’d need to escape it with "^^" (e.g. "^^!").

Also, it’d be a good idea to specify the filename as the first parameter – I haven’t done any extension check to figure out which parameter the filename is at.

You can find more information about the command-line parameters for MPlayer at:
http://www.mplayerhq.hu/DOCS/man/en/mplayer.1.html


Update: 2008-07-10 @ 03:54
I’ve made a new version – v2-alpha. The new post is here

XP SP3 is out!

Synaptic or Aptitude?

Short answer: Aptitude!

Long answer (and the story):
I was evaluating various video editors for Linux today when I noticed various files weren’t being removed after removing "kdenlive" using Synaptic.

Here’s a list of executables beginning with "k" before installation:

kbd_mode    kbdrate     kill        killall     killall5    klogd       koi8rxterm 

And here’s the list I got after specifying complete removal of "kdenlive" and "kdenlive-data":

kab2kabc                kde-config              kdostartupconfig        kinstalltheme           koi8rxterm
kaddprinterwizard       kded                    kfile                   kioexec                 kpac_dhcp_helper
kbd_mode                kdeinit                 kfmexec                 kio_http_cache_cleaner  ksendbugmail
kbdrate                 kdeinit_shutdown        kgrantpty               kioslave                kshell
kbuildsycoca            kdeinit_wrapper         khotnewstuff            kio_uiserver            kstartupconfig
kcmshell                kde-menu                kill                    klauncher               ktelnetservice
kconf_update            kdesu_stub              killall                 klogd                   ktradertest
kcookiejar              kdontchangethehostname  killall5                kmailservice            kwrapper

Turns out Synaptic forgot to remove the "kdelibs4c2a" and "kdelibs-data" package…

Even after installing "deborphan", as well as following this advice to filter for orphaned packages, Synaptic still didn’t present those 2 packages for removal.

Aptitude on the other hand took care of all the dependencies on installation and removes the 2 extra packages upon removal!

So next time when you want to install a package using Synaptic, or see a command like:

sudo apt-get update && sudo apt-get install some_package

Make sure you use Aptitude UI or the following command instead, so you can uninstall with Aptitude later:

sudo aptitude update && sudo aptitude install some_package

Note: This was tested on Xubuntu 8.04 so I’m not using an outdated version of Synaptic / apt-get.


Oh, just in case you are interested in my video editor of choice (for now at least…) – it’s Kdenlive!
If you want me to explain my decision, drop me a comment.