1 (edited by sicofante 2009-08-06 13:31:07)

Topic: e2fsck is taking forever

Gparted is usually too slow for resizing, so I frequently use Acronis for that. But for some reason Acronis stopped recognizing my Ubuntu partitions. I tried Paragon yesterday and everything seemed to work fine. However, my two partitions (root and home) appear "empty" now and the system won't boot. I'm running an Ubuntu LiveCD and Gparted and Nautilus both say my home partition still holds some 106 GB in there (it's a 142 GB partition). So I decided to verify the partition with Gparted. (I don't care that much about the root partition. I can always reinstall, but my home partition is very important.)

It's been almost 24 hours now and e2fsck -f -y -v /dev/sda1 is still running. The disk led is blinking so it looks like it's doing something but isn't this taking too long for a 106 GB amount of data?

I know that if I had run e2fsck in a terminal I would have some output telling me what's going on. Is there a way to know that output from Gparted?

How long would it be reasonable to wait?

Thanks for any help.

2

Re: e2fsck is taking forever

Unfortunately GParted does not currently provide a way to see the output of the e2fsck command until after the command has completed.

You could check to see if the e2fsck command is still running with the following command:

ps -ef | grep e2fsck

If the e2fsck command is still running then I would be reluctant to stop it.

Of course if e2fsck is in some kind of infinite loop, then this could be difficult to determine hmm

3 (edited by sicofante 2009-08-06 20:38:51)

Re: e2fsck is taking forever

Thanks for your reply.

e2fsck is running indeed (I used exactly the command you suggested before the first post). The drive access light suggests it's doing something with the disk.

The question is, how long is reasonable to wait?

If I stop it, do I have any chance by using file recovery tools on the partition? (I have experience with NTFS partitions but I have very little with EXT3.)

The wait is killing me... sad

4

Re: e2fsck is taking forever

You could try typing in "dmesg" and look at the messages at the tail of this command.  Perhaps a hardware error is occurring?

Also, 146 GB is quite a large partition, and with 106 GB of data in it, it could indeed take a very long time to file system check (and repair) the entire file system.

5

Re: e2fsck is taking forever

dmesg doesn't seem to have any issues at its tail (only wlan0 related messages which look quite normal to me). I searched for sda in the output of dmesg and here's what I've got:

ubuntu@ubuntu:~$ dmesg | grep sda
[    3.876202] sd 0:0:0:0: [sda] 312581808 512-byte hardware sectors: (160 GB/149 GiB)
[    3.876217] sd 0:0:0:0: [sda] Write Protect is off
[    3.876219] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    3.876242] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    3.876308] sd 0:0:0:0: [sda] 312581808 512-byte hardware sectors: (160 GB/149 GiB)
[    3.876320] sd 0:0:0:0: [sda] Write Protect is off
[    3.876322] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    3.876344] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    3.876347]  sda: sda1 sda2 sda4
[    4.244456] sd 0:0:0:0: [sda] Attached SCSI disk
[    5.947912] sda4: rw=0, want=14422024, limit=12402117
[   75.662733] Adding 1052248k swap on /dev/sda2.  Priority:-1 extents:1 across:1052248k
[  207.992975] sda4: rw=0, want=14422024, limit=12402117
[  224.180718] EXT3 FS on sda1, internal journal
ubuntu@ubuntu:~$ 

I'm just posting it in case any of this makes sense to you.

sda4 holds the root partition, sda2 is my swap partition and sda1 is my home partition. I was able to mount the home partition but it appeared empty. I can't mount the root partition at all but again, I really don't care about it (it just contained a tipycal Ubuntu install, nothing I can't replicate easily).

So you think it's possible that e2fsck is actually repairing the file system at sda1, right? I should wait maybe another 24 hours?

6

Re: e2fsck is taking forever

From the dmesg output, all appears well with the hardware related to the "sda" device.

It is possible that e2fsck is still actually repairing the file system at sda1.

If you already have a good backup of the partition, then I would not worry about cancelling the e2fsck since you would be able to restore the data.

If you do not have a backup of the partition, then I would be tempted to let the process run for perhaps another 24 hours.  Killing the e2fsck process while it runs is likely to cause the file system to be in an inconsistent state.

7

Re: e2fsck is taking forever

Supposing that you booted from the live CD and the e2fsck application is inside the live CD:
Some questions:
1.- What version of e2fsck are you using?
2.- Do you have DMA active while running the "e2fsck" application in the drive you're checking? If DMA is no active in the drive you are e2fsck'ing, then it will take much more time. Utility 'hdparm' can help you in that - you can googleing to know how to activate DMA. Furthermore, Ubuntu can need a few more tricks to enable DMA: http://ubuntuforums.org/showthread.php? … ght=hdparm
CAUTION!: I don't recommend to activate DMA while e2fsck is running!
If you get difficulty to activate DMA in the live Cd you currently have, you can use another live CD. But I wouldn't kill the e2fsck until you have a good backup of the partition, like gedakc wrote.
3.- Didn't you get any output of the e2fsck command?

With these information we can know more about yor case.

Regards.

8 (edited by sicofante 2009-08-07 15:12:58)

Re: e2fsck is taking forever

Thanks videoclock for your interest. Here are the answers to your questions:

1. I'm running an Ubuntu 9.04 LiveCD, which includes version 1.41.4 of the e2fslibs. I understand that's the version of e2fsck too.

ubuntu@ubuntu:~$ e2fsck -V
e2fsck 1.41.4 (27-Jan-2009)
    Al emplear EXT2FS Library version 1.41.4, 27-Jan-2009
ubuntu@ubuntu:~$ 

2. Regarding UDMA, I guess it's ON for both the ODD and the HDD (this is a 4 months old Compaq laptop):

ubuntu@ubuntu:~$ dmesg | grep UDMA
[    1.775122] ata1: SATA max UDMA/133 abar m2048@0x94705000 port 0x94705100 irq 2301
[    1.775125] ata2: SATA max UDMA/133 abar m2048@0x94705000 port 0x94705180 irq 2301
[    1.775131] ata5: SATA max UDMA/133 abar m2048@0x94705000 port 0x94705300 irq 2301
[    1.775134] ata6: SATA max UDMA/133 abar m2048@0x94705000 port 0x94705380 irq 2301
[    2.257297] ata1.00: ATA-8: Hitachi HTS543216L9A300, FB2OC40F, max UDMA/100
[    2.258726] ata1.00: configured for UDMA/100
[    3.173486] ata2.00: ATAPI: Slimtype DVD A  DS8A2L-A, 7H63, max UDMA/100
[    3.188051] ata2.00: configured for UDMA/100
ubuntu@ubuntu:~$ 

3. I'm running e2fsck from inside GParted. In consequence -gedack explained it- I don't have any output from the e2fsck command.

If all this tells you something new, please let me know.

Do you guys think I will have any chance with recovery tools like those mentioned here: http://ubuntuforums.org/showthread.php?t=244676? As time passes by I'm losing my hopes on e2fsck finishing succesfully. It's been about 40 hours now, and counting... sad

9

Re: e2fsck is taking forever

Is the hard drive still active?

If so then it would appear that e2fsck is still performing some operations on the drive.

As for if you decide to kill the process, the testdisk application can be quite useful for recovering partitions, and the photorec tool can be used to recover some types of files.

My hope is that e2fsck will finish soon.  If you decide to kill the process, I hope that the file system can be successfully recovered.

10

Re: e2fsck is taking forever

Can you give the output of these three commands?

 sudo hdparm -tT /dev/sda
sudo hdparm /dev/sda
iostat -x

It can help to know the parameters of yor drive (DMA is active or not, for example), the current speed of your drive and more information.

Regards.

11

Re: e2fsck is taking forever

Thank you so much for your efforts guys. I'm away from my laptop now (I couldn't stand watching it obsessively without being able to do anything and I decided to go out tonight and get some fresh air).

The drive was still blinking when I left home. There's still hope I guess.

I'll be back to you tomorrow with new replies to your questions.

12

Re: e2fsck is taking forever

Hi there again,

Back from a nice weekeng my disk is still spinning and e2fsck hasn't finished. I'm going to sleep now and I'll abort it tomorrow.

Here's the output from the requested commands:

ubuntu@ubuntu:~$ sudo hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   1510 MB in  2.00 seconds = 755.01 MB/sec
 Timing buffered disk reads:  106 MB in  3.00 seconds =  35.29 MB/sec
ubuntu@ubuntu:~$ sudo hdparm /dev/sda

/dev/sda:
 IO_support    =  0 (default) 
 readonly      =  0 (off)
 readahead     = 256 (on)
 geometry      = 19457/255/63, sectors = 312581808, start = 0
ubuntu@ubuntu:~$ iostat -x
Linux 2.6.28-11-generic (ubuntu)     09/08/09     _i686_    (2 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          31,84    1,80   17,42   25,18    0,00   25,21

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda            5995,54     0,97  232,43    0,05   647,35     4,72     2,80     1,27    5,48   3,78  87,97
sda1           5995,31     0,56  232,42    0,02   645,56     1,16     2,78     1,27    5,46   3,78  87,96
sda2              0,05     0,41    0,01    0,03     0,43     3,55    96,12     0,01  135,13  11,66   0,05
sda4              0,16     0,00    0,01    0,00     1,30     0,00   238,78     0,00    5,05   3,19   0,00
sr0               0,94     0,00    0,06    0,00     4,02     0,00    62,87     0,01  127,94 101,49   0,65

13

Re: e2fsck is taking forever

I've got some consequences of the information you've provided:

ubuntu@ubuntu:~$ sudo hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   1510 MB in  2.00 seconds = 755.01 MB/sec
 Timing buffered disk reads:  106 MB in  3.00 seconds =  35.29 MB/sec

1.- You have DMA active in your /dev/sda device, because it reads at 35MB/sec. If you didn't have it active, it possibly were reading at less than 10MB/sec


ubuntu@ubuntu:~$ iostat -x
Linux 2.6.28-11-generic (ubuntu)     09/08/09     _i686_    (2 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          31,84    1,80   17,42   25,18    0,00   25,21

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda            5995,54     0,97  232,43    0,05   647,35     4,72     2,80     1,27    5,48   3,78  87,97
sda1           5995,31     0,56  232,42    0,02   645,56     1,16     2,78     1,27    5,46   3,78  87,96
sda2              0,05     0,41    0,01    0,03     0,43     3,55    96,12     0,01  135,13  11,66   0,05
sda4              0,16     0,00    0,01    0,00     1,30     0,00   238,78     0,00    5,05   3,19   0,00
sr0               0,94     0,00    0,06    0,00     4,02     0,00    62,87     0,01  127,94 101,49   0,65

2.- Your /dev/dsa1 partition is having a lot of I/O activity, so it's very probable that the e2fsck is still running

3.- Moreover, your /dev/sda1 is providing 645 reads sectors per second and 1 write per second to the operating system!


4.- I don't know how long your drive is. It depends on what your OS and  the utility you're using assumes a sector is: If 'hdparm sector size' is 1KB, your drive size has 1,1 TB approx.  If 'hdparm sector size' is 4KB, your drive size has 300 GB approx.

5.- And I don't know how long is 'iostat sector size' in your OS. It's usually 1KB or 4KB. Supposing it's 4KB, yor OS is reading 645,56 * 4 KB = 2582,24 KB per second. Yes, it can seem a slow speed, but think that CPU has to check every sector, so drive has to stop transferring when CPU is checking sectors it has already in main memory.

6.- If you know the above values, you can know the estimated time e2fsck is going to take, supposing it doesn't go inside an infinite loop:

142 GB = 142 * 1024 * 1024 KB = 148897792 KB

645'56 * iostat sector size in KB ------------> 1 sec
148897792 KB ----------------------------------> X

X = (148897792 KB) / (645'56 * iostat sector size in KB )

The X value appears in seconds, so you can change it to days, for example smile

So you can know approximately how many time your e2fsck is going to take:

If 'iostat sector size' is 4KB, the taken time would be: 16 hours approx.
If 'iostat sector size' is 1KB, the taken time would be: 3 days approx.

I Think this calculus are correct, correct them if not wink

Doing these calculus in writes instead of reads are not so correct: you can think that good sectors are not written to the disk.

Well, I thougth that have help you. I'm opened to opinions of experienced people. Maybe someone know a more improved way to know the time approx. a process of this features can take.

Regards

14

Re: e2fsck is taking forever

OK. So let's imagine my system uses 1K blocks. It's been about three days now, so I might as well wait one more night.

Thanks for your insight.

15 (edited by sicofante 2009-08-11 17:30:01)

Re: e2fsck is taking forever

Well, this is it.

e2fsck is still running (it'll be 4 days in a few hours) with no signs of finishing. Before I stop it for good, is there a way to know if it is on a loop?

But more importantly: can I pause the process somehow and start a recovery tool on the disk?

16

Re: e2fsck is taking forever

I am unaware of any way to pause the e2fsck process.  Since you have already checked the dmesg log for errors, I am not aware of another way to determine if e2fsck is in a loop.

I'll keep my fingers crossed that after killing e2fsck you are able to recover your files.

17

Re: e2fsck is taking forever

e2fsck is still running (it'll be 4 days in a few hours) with no signs of finishing. Before I stop it for good, is there a way to know if it is on a loop?

I don't know a way to get it. I'm not a e2fsck guru. But you can do this question to people who know e2fsck internals in this page:
http://sourceforge.net/projects/e2fsprogs/support

e2fsprogs is the project that involves e2fsck utility.

Of course you would have to tell all you have happened and your e2fsck version, which you can know with this command:

e2fsck -V

But more importantly: can I pause the process somehow and start a recovery tool on the disk?

Mostly times yes. If you run 'top' in a shell and you see the PID of your 'e2fsck', you can in another shell run 'kill PID' , replacing PID for the number you would find in the 'top' command. If after that this PID doesn't appear in your 'top' command, your 'e2fsck' process would be nicely killed.  And then you would be able to start your recovery tool.

I find it rare to know your 'e2fsck' is still running...

Regards.

18

Re: e2fsck is taking forever

Well, I finally canceled e2fsck.

Here's the report (sorry it's in Spanish but I hope you'll get it):

GParted 0.4.3

Libparted 1.8.8
Verificar y reparar el sistema de archivos (ext3) en /dev/sda1  167:14:57    ( ERROR )
         
calibrar /dev/sda1  00:00:00    ( ÉXITO )
         
ruta: /dev/sda1
inicio: 12402180
fin: 310472189
tamaño: 298070010 (142.13 GiB)
comprobar errores en el sistema de archivos en /dev/sda1 y (si es posible) arreglarlos    ( EJECUTANDO )
         
e2fsck -f -y -v /dev/sda1

========================================

Yes, that's 167 hours running. Or almost a week.

I think there's a serious design flaw in GParted. It makes no sense to have processes that may take hours (or days) and not tell the user what's going on. I guess I should file a bug about it (if no one else has so far).

Now I'm going to try the different recovery tool and see if can get anything from that brick, errr... HDD.

Thanks for all your efforts.

19

Re: e2fsck is taking forever

Thanks sicofante for reporting all of your progress on this problem.  Your particular case is the longest I have heard of e2fsck running.  This most certainly has been a frustrating issue for you.  sad

Hopefully you are able to recover your data.

A bug request for additional progress information on file system checking and resizing has already been opened regarding this problem.
http://bugzilla.gnome.org/show_bug.cgi?id=467925