Debian Administration

Recovering from file system corruption using TestDisk

We've all been there. We press the wrong key, we do some silly mistake, and suddenly, one or more of our file systems refuse to work. Whenever this happens, the first thing we hear is "You should have made a backup", the dreaded sentence that we'll never listen to. Let's face it, we're stupid, and we don't backup.

Fortunately, we have several powerful, open source tools that can help us save our data. The first thing you should do after having such a problem - to quote Douglas Adams - Don't Panic!

In this article, we'll use the amazing 'TestDisk' program to try and recover our damaged/lost partitions. The 'testdisk' package is available in all Debian releases (From stable to unstable) and can be installed easily.

So, after getting over the panic phase, we need to buckle up and get going. I suggest the following steps:

1. Classify the type of hardware where data has been lost: Hard Drive, USB Storage Device, CD, DVD, Compact Flash, etc. This is important because we are assuming the corrupted file system is not the one where the operating system resides, so it's still up and running. If that's not the situation, then the best way is to use a Live distribution. Knoppix is recommended, since it's based on Debian, containes all the tools we're gonna talk about, and every Debian user should feel comfortable with it. If you don't have a copy of Knoppix yet, you should get one now (See external links).

Note that when dealing with CDs and DVDs, these could be scratched or fast erased discs.

2. Backup the damaged partition(s): You may raise an eyebrow, but this step is actually very important. After suffering a corruption, we can still save our file system. This step assumes that if we were stupid enough to corrupt a file system and not make a backup, we're stupid enough to make another mistake while trying to save it, and destroy our chances of saving it completely.

This step might be difficult, depending on the size of the partition, but do your best to avoid skipping it. Basically, you'd want to backup to a larger device, and always, ALWAYS, to a DIFFERENT device. We'll do this using the 'ddrescue' program, available in the 'gddrescue' package:

$ apt-get install gddrescue

Then, we can create our backup:

$ ddrescue -B -n /dev/old_disk /dev/new_disk rescued.log

$ ddrescue -B -r 1 /dev/old_disk /dev/new_disk rescued.log

Replace old_disk and new_disk with your source and destination devices, respectively (We're backing up the whole device, not just a partition). Note that the first command backs up error-free areas, while the other tries to backup damaged areas.

Unfortunately, stable doesn't have the 'gddrescue' package yet, so you will need to use 'dd' instead.

$ dd if=/dev/old_disk of=/dev/new_disk

Which is a pretty basic command. You might wanna try 'man dd' to learn about dd's arguments and use a command more suitable for your hardware.

If you prefer not to use 'dd', you can also try Kurt Garloff's 'dd_rescue'.

3. Install TestDisk: 'TestDisk' is a powerful program designed to recover damaged or lost partitions. To install it, simply type:

$ apt-get install testdisk

'TestDisk' works with Ext2/Ext3, ReiserFS 3.6, ReiserFS 4, XFS, JFS, VFAT, NTFS, Linux Swap, etc. It also comes bundled with another program, 'PhotoRec', which started as a program for recovering photographs from digital cameras, but developed into a general data recovery program from different storage devices, even hard disks. For a list of file systems 'TestDisk' can work with and file types 'PhotoRec' can work with, use apt-cache:

$ apt-cache show testdisk

4. Run TestDisk: We're now ready to run 'TestDisk':

$ testdisk

After a few seconds, 'TestDisk' will list the storage devices connected to your computer. Highlight your device and choose "Proceed". 'TestDisk' will then ask you the type of your partition. Usually, this would be Intel. After that, first thing to do is use "Analyze". 'TestDisk' will then analyze the device and try to find all of the partitions in it.

If you can't find your partition, try a more thorough search using "Search!". If you still can't find your partition, move to step 5.

If you have found your partition, highlight it and press Enter. Next, press 'p' to view the files in the partition. I call this step the "Phew, my files are still there step".

Right now, you might wanna try using "Write" to write the changes to the device. We haven't really done any changes, but 'TestDisk' might have done them itself, if it has identified a damaged boot sector.

After quitting 'TestDisk', you can try to access the partition. If you still can't do that, don't despair. Possibly, the boot sector is damaged. Run 'TestDisk' again, but this time use "Advanced". Select your partition, and choose "Boot".

If the NTFS or FAT boot sector has been found using the backup boot sector, 'TestDisk' will let you choose between "Rebuild BS" and "Backup BS". Choose "Backup BS" to replace the damaged boot sector by its backup.

Also, 'TestDisk' lets you dump the boot sector content ("Dump"). If it's a FAT partition, you can try to "Repair FAT". If it's an Ext2/Ext3 partition, you can try "Superblock".

Hopefully, 'TestDisk' has managed to repair the boot sector and the partition can now be mounted.

If 'TestDisk' found your partition, but doesn't know its type, use "Advanced" and select "Type". 'TestDisk' will then ask you for the fs type.

You are now ready to try mounting the partition again. If you can't do so, check if the disk is in use. If it is, you may have to reboot to be able to mount the partition. If you still can't mount, we might as well give up on trying to get the partition working, and try to recover the data inside it instead.

5. Run PhotoRec: 'PhotoRec' is your best choice when trying to recover data from damaged partitions. Basically, it doesn't care about the file system. It simply ignores it. To run 'PhotoRec', just type:

$ photorec

I will remind you that 'PhotoRec' has been automatically installed with 'TestDisk'.

The program's interface is just like 'TestDisk'. Select your device, the partition type (Intel, Mac, etc.), and use "Search". 'PhotoRec' will then make a thorough search of the partition to find files in it and recover them to your home directory (Into numbered "recup_dir" directories, like "recup_dir.1", "recup_dir.2", etc.). This may take some minutes, or hours, if the partition is very big.

Note that if you are recovering from an Ext2/Ext3 partition, you have to activate the ext2/ext3 mode in "Options".

After search has been completed, quit, take a look at your restored files, and use the command

$ hip hip hooray

which may or may not work.

If you need a more general program for recovering files, you can try the 'recover' and 'e2undel' packages, but these only work with ext2 partitions, and we are assuming that the partitions are not damaged. Basically, you can use these tools after you have used 'TestDisk' to repair a damaged partition or recover a lost one, but some of your files are still lost.

Remember: At any point of the above process, if somehow you make a mistake and damage your partition even more badly, just grab your backup device and start over.

I have managed to use 'TestDisk' to recover a damaged ext2 partition, back in the days when I was dual booting Debian and Windows. I used Partition Magic to resize some partitions, and it completely screwed them up.

I have also managed to use 'PhotoRec' to recover photographs from a CompactFlash card, after it failed mysteriously. I was taking pictures with my camera, when suddenly I got a "CF Error" message. I couldn't mount the card on my computer, and 'TestDisk' didn't manage to recover it. Fortunately, 'PhotoRec' restored all of the files with no problems, and I was able to format the card and start using it again.

Good Luck!

I would be happy to recieve any comments, suggestions, etc.

External Links:
1. Knoppix.
2. TestDisk - The program's home page, contains a Wiki with a lot of useful information and documentation. Be sure to check out Data recovery examples.
3. PhotoRec.
4. ddrescue.

Thanks to Christophe Grenier, the author of TestDisk, for helping out with the article!