Recovering from file system corruption using TestDisk

Posted by ido50 on Tue 18 Jul 2006 at 09:30

We've all been there. We press the wrong key, we do some silly mistake, and suddenly, one or more of our file systems refuse to work. Whenever this happens, the first thing we hear is "You should have made a backup", the dreaded sentence that we'll never listen to. Let's face it, we're stupid, and we don't backup.

Fortunately, we have several powerful, open source tools that can help us save our data. The first thing you should do after having such a problem - to quote Douglas Adams - Don't Panic!

In this article, we'll use the amazing 'TestDisk' program to try and recover our damaged/lost partitions. The 'testdisk' package is available in all Debian releases (From stable to unstable) and can be installed easily.

So, after getting over the panic phase, we need to buckle up and get going. I suggest the following steps:

1. Classify the type of hardware where data has been lost: Hard Drive, USB Storage Device, CD, DVD, Compact Flash, etc. This is important because we are assuming the corrupted file system is not the one where the operating system resides, so it's still up and running. If that's not the situation, then the best way is to use a Live distribution. Knoppix is recommended, since it's based on Debian, containes all the tools we're gonna talk about, and every Debian user should feel comfortable with it. If you don't have a copy of Knoppix yet, you should get one now (See external links).

Note that when dealing with CDs and DVDs, these could be scratched or fast erased discs.

2. Backup the damaged partition(s): You may raise an eyebrow, but this step is actually very important. After suffering a corruption, we can still save our file system. This step assumes that if we were stupid enough to corrupt a file system and not make a backup, we're stupid enough to make another mistake while trying to save it, and destroy our chances of saving it completely.

This step might be difficult, depending on the size of the partition, but do your best to avoid skipping it. Basically, you'd want to backup to a larger device, and always, ALWAYS, to a DIFFERENT device. We'll do this using the 'ddrescue' program, available in the 'gddrescue' package:

$ apt-get install gddrescue

Then, we can create our backup:

$ ddrescue -B -n /dev/old_disk /dev/new_disk rescued.log
$ ddrescue -B -r 1 /dev/old_disk /dev/new_disk rescued.log

Replace old_disk and new_disk with your source and destination devices, respectively (We're backing up the whole device, not just a partition). Note that the first command backs up error-free areas, while the other tries to backup damaged areas.

Unfortunately, stable doesn't have the 'gddrescue' package yet, so you will need to use 'dd' instead.

$ dd if=/dev/old_disk of=/dev/new_disk
Which is a pretty basic command. You might wanna try 'man dd' to learn about dd's arguments and use a command more suitable for your hardware.

If you prefer not to use 'dd', you can also try Kurt Garloff's 'dd_rescue'.

3. Install TestDisk: 'TestDisk' is a powerful program designed to recover damaged or lost partitions. To install it, simply type:

$ apt-get install testdisk

'TestDisk' works with Ext2/Ext3, ReiserFS 3.6, ReiserFS 4, XFS, JFS, VFAT, NTFS, Linux Swap, etc. It also comes bundled with another program, 'PhotoRec', which started as a program for recovering photographs from digital cameras, but developed into a general data recovery program from different storage devices, even hard disks. For a list of file systems 'TestDisk' can work with and file types 'PhotoRec' can work with, use apt-cache:

$ apt-cache show testdisk

4. Run TestDisk: We're now ready to run 'TestDisk':

$ testdisk

After a few seconds, 'TestDisk' will list the storage devices connected to your computer. Highlight your device and choose "Proceed". 'TestDisk' will then ask you the type of your partition. Usually, this would be Intel. After that, first thing to do is use "Analyze". 'TestDisk' will then analyze the device and try to find all of the partitions in it.

If you can't find your partition, try a more thorough search using "Search!". If you still can't find your partition, move to step 5.

If you have found your partition, highlight it and press Enter. Next, press 'p' to view the files in the partition. I call this step the "Phew, my files are still there step".

Right now, you might wanna try using "Write" to write the changes to the device. We haven't really done any changes, but 'TestDisk' might have done them itself, if it has identified a damaged boot sector.

After quitting 'TestDisk', you can try to access the partition. If you still can't do that, don't despair. Possibly, the boot sector is damaged. Run 'TestDisk' again, but this time use "Advanced". Select your partition, and choose "Boot".

If the NTFS or FAT boot sector has been found using the backup boot sector, 'TestDisk' will let you choose between "Rebuild BS" and "Backup BS". Choose "Backup BS" to replace the damaged boot sector by its backup.

Also, 'TestDisk' lets you dump the boot sector content ("Dump"). If it's a FAT partition, you can try to "Repair FAT". If it's an Ext2/Ext3 partition, you can try "Superblock".

Hopefully, 'TestDisk' has managed to repair the boot sector and the partition can now be mounted.

If 'TestDisk' found your partition, but doesn't know its type, use "Advanced" and select "Type". 'TestDisk' will then ask you for the fs type.

You are now ready to try mounting the partition again. If you can't do so, check if the disk is in use. If it is, you may have to reboot to be able to mount the partition. If you still can't mount, we might as well give up on trying to get the partition working, and try to recover the data inside it instead.

5. Run PhotoRec: 'PhotoRec' is your best choice when trying to recover data from damaged partitions. Basically, it doesn't care about the file system. It simply ignores it. To run 'PhotoRec', just type:

$ photorec

I will remind you that 'PhotoRec' has been automatically installed with 'TestDisk'.

The program's interface is just like 'TestDisk'. Select your device, the partition type (Intel, Mac, etc.), and use "Search". 'PhotoRec' will then make a thorough search of the partition to find files in it and recover them to your home directory (Into numbered "recup_dir" directories, like "recup_dir.1", "recup_dir.2", etc.). This may take some minutes, or hours, if the partition is very big.

Note that if you are recovering from an Ext2/Ext3 partition, you have to activate the ext2/ext3 mode in "Options".

After search has been completed, quit, take a look at your restored files, and use the command

$ hip hip hooray

which may or may not work.

If you need a more general program for recovering files, you can try the 'recover' and 'e2undel' packages, but these only work with ext2 partitions, and we are assuming that the partitions are not damaged. Basically, you can use these tools after you have used 'TestDisk' to repair a damaged partition or recover a lost one, but some of your files are still lost.

Remember: At any point of the above process, if somehow you make a mistake and damage your partition even more badly, just grab your backup device and start over.

I have managed to use 'TestDisk' to recover a damaged ext2 partition, back in the days when I was dual booting Debian and Windows. I used Partition Magic to resize some partitions, and it completely screwed them up.

I have also managed to use 'PhotoRec' to recover photographs from a CompactFlash card, after it failed mysteriously. I was taking pictures with my camera, when suddenly I got a "CF Error" message. I couldn't mount the card on my computer, and 'TestDisk' didn't manage to recover it. Fortunately, 'PhotoRec' restored all of the files with no problems, and I was able to format the card and start using it again.

Good Luck!

I would be happy to recieve any comments, suggestions, etc.

External Links:
1. Knoppix.
2. TestDisk - The program's home page, contains a Wiki with a lot of useful information and documentation. Be sure to check out Data recovery examples.
3. PhotoRec.
4. ddrescue.

Thanks to Christophe Grenier, the author of TestDisk, for helping out with the article!

 

 


Posted by Anonymous (193.69.xx.xx) on Tue 18 Jul 2006 at 11:07
I used TestDisk a week ago and it saved my day.

Thank you for this program. And thanks for making this tool known. I had been trying to use older less supported tools such as gpart and those were not as efficient (in my case).

Jerome Lacoste

[ Parent | Reply to this comment ]

Posted by Anonymous (193.41.xx.xx) on Tue 18 Jul 2006 at 11:09
Great UTIL! %)
i've rescue photos from my friend vocation %)

[ Parent | Reply to this comment ]

Posted by Anonymous (216.46.xx.xx) on Tue 18 Jul 2006 at 15:03
In case this article hasn't made this clear enough for everyone, TestDisk and PhotoRec are extremely useful tools in situations where partition meta-information may have been lost, but the data held on the partition has not yet been overwritten with new data.

This would be the case, for example, if you mistakenly used a 'format' command on a filesystem. Thus, for example, I was able to recover my mom's 'lost' pictures off a CF card that she had 'formatted' by mistake using her camera. The picture files did not show up with an ls command, but were nonetheless recovered in full after a PhotoRec run.

The important detail to remember here is that when you make this kind of mistake with a filesystem, you should unmount it right away and proceed to use Grenier's recovery tools immediately, so as not to allow the existing (hidden) data to be overwritten with new data.

Finally, it should be noted that the tools in question are also available for the Windows platform, so they are still useable even if you don't have access to a proper PC.

[ Parent | Reply to this comment ]

Posted by tong (69.158.xx.xx) on Tue 18 Jul 2006 at 21:12
[ Send Message | View Weblogs ]
I've been looking for such tool for a long long time and no one on debian-user mlist seems to know it by the times that I asked for them.

Thanks a lot for sharing!!!

A side question, would it work on CDR too?

I have a CDR full of tools that I collected. It was burnt ok, but I can't mount it any more (due to alignment problem of the original burnner?).

[ Parent | Reply to this comment ]

Posted by ido50 (89.1.xx.xx) on Tue 18 Jul 2006 at 21:19
[ Send Message | View Weblogs ]
Yes, it works with CD-R and CD-RW as well.

[ Parent | Reply to this comment ]

Posted by ido50 (89.1.xx.xx) on Tue 18 Jul 2006 at 21:21
[ Send Message | View Weblogs ]
P.S. More information in the program's wiki.

[ Parent | Reply to this comment ]

Posted by Anonymous (85.234.xx.xx) on Wed 19 Jul 2006 at 08:28
I found testdisk when it was needed by running "apt-cache search partition recovery". Hey, this is Debian! You should know how to use your distribution in a right way :)

[ Parent | Reply to this comment ]

Posted by Anonymous (87.100.xx.xx) on Wed 3 Jan 2007 at 16:34
I made this terrible mistake : willing to replace a drive in a quite complicated IDE drives setup (see bottom of post), I failed to remove the drives to be kept safe, launched a "disk to disk copy" and waited there. It was not until I relaunched the GUI to troubleshoot why the new disk would not have any data on it, that I found that I just overwrote my main 250 GB DATA disk with multiple SYSTEM partitions. Suffice it to say that the disk had 30% space free, no media -music or video- files on it... just plain, straight valuable data.

Ouch, that hurts ! No raid, no such thing as a 200 GB backup around, overwritten sectors ... plus I trained just last week with secure erasing. Hence the old data disk is wiped clean and no hope on this side. Luck hates me this new year.

Only hope: the 6-7 system partitions are rather small, compared with 200 GB worth of data; I could try and access the disk below file system, at sector level. Testdisk would restore the old partition table but would fail -no wonder- at opening a valid filesystem. Other tools would fail as well. Disk investigator inside windows, would show some content but is helpless at restoring data.

Trying the last testdisk release (2006-10), I came accross photorec, almost by accident. I read that it "doesn't care about a file system" : sounds just what I need. I setup the new 250 GB drive as free storage space, launch photorec from there and watch hundreds of thousands of files be found. Prompt claims it should be done in 10 hours.

Note that we are talking 500,000,000 (500 million) sectors here: 250 GB. This looks damn fast .

Not sure what will wait for me in 10 hours, but I guess it will be much better than what I have now : anger and frustration. With a little luck, either the backup archives or the original data will be in some kind of good shape and most of the lost data will be reinstallable programs or CD/DVD images.

Retrieving data without a file system if a good thing, as this example illustrates.

btw, Merci Gilles Grenier : tu rockes (et roules)

Roger DUBOST
follow ups in full article (french)

Setup:

      Asus P4C800-E DELUXE
      IDE 0 : PATA :system partitions
      IDE 1 : IDE DVD-Rom
      IDE 2 SATA : DataDisk
      IDE 2'SATA : mp3's
      IDE 3 PATA (Promise) : new disk
      IDE 3'PATA (Promise) : SAVEPART images on spare drive (system only)
      burner is SCSI

    "Fatal" copy operation with Ranish partition manager : ghost 2003 would fail at building a perfect copy, either direct cloning or with intermediate backup. Got fooled with bios disk numbering being different than that in windows.

    Tip : stay away from this so called hi-end mobo: P4C800-E Deluxe, even this brand : ASUS. This hardware violates every IDE standard ever. You know it when you use it IDE goodies and install linux. Part of the reason I got fooled. The rest is just me being an a$$.

[ Parent | Reply to this comment ]

Posted by Anonymous (24.193.xx.xx) on Sun 7 Jan 2007 at 05:49
Thanks Christophe ! you saved my day !!! I have been able to recover all my pictures that I lost when my compact flash crashed. Thanks a lot!

Definitely recommend this tool!

[ Parent | Reply to this comment ]

Posted by Anonymous (82.59.xx.xx) on Wed 28 Feb 2007 at 17:09
I'm trying it out if it works as it should and it looks like it will great prog thanks !!!!

[ Parent | Reply to this comment ]

Posted by summitwulf (70.95.xx.xx) on Sun 6 May 2007 at 17:54
[ Send Message | View Weblogs ]
I'm trying to recover data from a couple of laptop drives and I see there is another program with the suspiciously similar name of 'dd_rescue'. They appear to do pretty much the same thing - does anyone know more about this other one?

It has a site at http://www.garloff.de/kurt/linux/ddrescue/

By the way, my first run of ddrescue backup just finished. Here's what it said:


> ddrescue -B -n /dev/hdb /dev/hdc7 rescued.log


Press Ctrl-C to interrupt
Initial status (read from logfile)
rescued: 0 B, errsize: 0 B, errors: 0
Current status
rescued: 0 B, errsize: 111 GiB, current rate: 0 B/s
ipos: 114473 MiB, errors: 1831576, average rate: 0 B/s
opos: 114473 MiB

>

I'm not too comforted by 'rescued: 0 B'. Does that mean it got nothing at all?

SW

[ Parent | Reply to this comment ]

Posted by summitwulf (70.95.xx.xx) on Mon 7 May 2007 at 03:40
[ Send Message | View Weblogs ]
I'm trying this method out now, but have a couple of questions. First I guess it will help if I describe my setup.

I'm using a VIA EPIA board loaded with a basic (command line) Debian Etch install. The OS resides on a 200 GB IDE drive as hdc, partitioned as follows:

> fdisk -l

Disk /dev/hdc: 200.0 GB, 200049647616 bytes
255 heads, 63 sectors/track, 24321 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/hdc1 * 1 851 6835626 83 Linux
/dev/hdc2 852 24321 188522775 5 Extended
/dev/hdc5 852 936 682731 82 Linux swap / Solaris
/dev/hdc6 937 4583 29294496 83 Linux
/dev/hdc7 4584 24321 158545453+ 83 Linux


hdc6 is a 30 GB partition containing the OS, and hdc7 is a 162.4 GB Linux (ext3) for storing data. I wasn't sure about hdc7 but the data has to go somewhere...?

A damaged disk from a Windows laptop is attached as is seen as hdb:

> ls /proc/ide
drivers hdb hdc ide0 ide1

I have gddrescue and testdisk installed as per the article.

So first I want to backup everything I can from the Windows disk (hdb) and dump it into hdc7 for safekeeping. When I try the first command I get this once it completes:

> ddrescue -B -n /dev/hdb /dev/hdc7 rescued1.log


Press Ctrl-C to interrupt
Initial status (read from logfile)
rescued: 0 B, errsize: 0 B, errors: 0
Current status
rescued: 0 B, errsize: 57231 MiB, current rate: 0 B/s
ipos: 57231 MiB, errors: 915705, average rate: 0 B/s
opos: 57231 MiB

I don't know how to interpret this. It looks like it is reporting that it failed to 'rescue' anything. Can anyone enlighten me? Is something being done wrong? Should the destination partition (hdc7) be formatted as NTFS rather than ext3?

Thanks,

SW

[ Parent | Reply to this comment ]

Posted by ido50 (212.25.xx.xx) on Mon 7 May 2007 at 07:41
[ Send Message | View Weblogs ]
Can you please post the contents of the rescued1.log file?

And have you tried running the second command too?
(ddrescue -B -r 1 /dev/old_disk /dev/new_disk rescued.log)

The command that you executed tried to backup any undamaged data. The second command will try to backup damaged data too. It is possible that all of the data is in a damaged area.

[ Parent | Reply to this comment ]

Posted by Anonymous (24.17.xx.xx) on Fri 29 Feb 2008 at 19:51
I have a 256 MB Memory Card (San Disk) that is corrupted. The card has ard 300 precious pictures. I have been trying to recover the pictures usuing the photrec software but no luck as such. I have a San Disk card reader and when I connect the card reader to my system with my memory card inserted and run the photorec_win exe file I get a "please wait..." message for a long time after which it list just my systems hard disk. It fails to recognize my memory card.

I would be really happy if you could help me with this situation.

[ Parent | Reply to this comment ]

Posted by alonso1999 (98.117.xx.xx) on Fri 17 Apr 2009 at 19:38
[ Send Message ]
I have a problem with this testdisk software Iam trying the recovery of an external 500G usb hardrive, I ran windows check volume tool and fixed errors and after that something went wrong the drive is there but blank it ask me if I want to reformat,
now i ran test disk follow the instructions and still cant boot or a list explore the drive it shows is there it shows the files but when i click out of testdisk and try the drive it shows blank

[ Parent | Reply to this comment ]

Posted by Anonymous (80.197.xx.xx) on Sat 21 Nov 2009 at 02:29
wow man! this testdisk app saved the day!!!!
I knew that my data was still intact after I mistakenly overwrote the partition table (confused /dev/sdc with /dev/sdb ... really really stupid). I just needed a way to restore the partition table on the disk but this cannot be done just like that :) This tool is just perfect for the job! but it needed a Deep research as it uncovered some very old obsolete partition table in the first attempt (quick search).

Anyway, great stuff :)

[ Parent | Reply to this comment ]

Posted by Anonymous (218.186.xx.xx) on Sat 2 Apr 2011 at 11:56
Test disk saved my day too. Even tho it took some learning to move around but it was great. Great stuff. I just used it to rebuild my boot sector and voila my 550 GB Lan Disk is accessible by USB and the files are all deletable. Thanks man. keng

[ Parent | Reply to this comment ]

Posted by Anonymous (94.198.xx.xx) on Wed 18 Jan 2012 at 03:05
My camera SD card was not recognizable by any reason with my newest photos! Used Photorec, my photos appeared by magic and the card is working fine again after formating. The problem was the card reader that was doing this to all the cards...

Thanks for all the instructions!

Long live open source!

B.S.

[ Parent | Reply to this comment ]

Posted by Anonymous (184.46.xx.xx) on Sun 22 Apr 2012 at 14:57
TestDisk is the bombiggidy..

[ Parent | Reply to this comment ]

Posted by Anonymous (190.235.xx.xx) on Wed 18 Sep 2013 at 04:50
Thank you for this very good good-mooded instruction.
for me PhotoRec worked better than TestDisk. my problem was simply recovering deleted foto files from a SD card. in TestDisk too many options and errors occured. PhotoRec was straiht forward to copy the files to my system.
Thank you and hip-hip-hooray!

[ Parent | Reply to this comment ]

Posted by chicoperico (84.148.xx.xx) on Fri 6 Dec 2013 at 10:10
[ Send Message ]
good day!

i recently installed a Debian wheezy on a machine with an MS windows residing on an ntfs partition and a kubuntu linux on several ext3 partitions.

i have done many such installation and was confident about not damaging the other operating systems.

i was bewildered, when i got to the step of assigning the partitions in the installation procedure - the only options for the windows partition were FAT16 and FAT32, no option of choosing ntfs. in an impulse i chose FAT32 and a mount point and went back to the main partition assignment page, assigned names and mount points to the other partitions. i then got suspicious, when i saw big F letters before these partitions, which in previous installations meant the partitions were to be formated, so i went back a step and ran an fdisk in parallel terminal.

the former ntfs partition now showed up as a FAT partition, all other partitions were intact. i used fdisk to reassign it to ntfs and proceded with the installation without mounting the windows partition, and without further problems.

i never formated the windows partition, and the step from committing the error to going back and reassigning the ntfs status did not take longer than five minutes at most. and the partition has about 200GB. so overwriting it with 0's or whatever would have taken much longer.

i now have run testdisk and photorec multiple times, not a single file shows up.

i also ran hexedit -s on this partition. about half of it is filled with 0's, the other half with 1's (or F's, in hexadecimal). only very small chunks have something else on it, which i suppose are the basic informations pertaining to a partition structure.

i am sure i never wrote anything else onto this partition after my mistake. so where have all the data gone??????


[ Parent | Reply to this comment ]

Sign In

Username:

Password:

[Register|Advanced]

 

Flattr

 

Current Poll

Which init system are you using in Debian?






( 1044 votes ~ 6 comments )

 

 

Related Links