[SOLVED] Unallocated space and corrupted NTFS after resize (Page 1 of 2)

**imperia** · 2022-10-12 18:48:04

imperia
Member
Offline

Registered: 2022-10-12
Posts: 14

Topic: [SOLVED] Unallocated space and corrupted NTFS after resize

Hello,

I am trying to recover from what seems crash after or during resize of partition.

Here is what happened:

I wanted to reduce size of disk D of Windows PC, so I can grow disk C. (There is recovery partition between them which I wanted to move after this).
I reduced the size of the second partition from the beginning with 19GB. I took long time because it seems during resize whole partition is moved.
I waited until 55% completed. Partition is ~600GB (~50% full). I had to go so I left the computer. I returned after few days. Computer was still running, but I was unable to wake up the screen. There was no HDD activity.
I restarted it and booted gparted again. The partition size was the same on the graph. And now I have exclamation mark on the partition. The error is:

19GiB of unallocated space within the partition. To grow the file system to fill the partition, select the partition and choose from menu item: Partition - Check

I haven't done Partition - Check.

I ran chkdsk without /f parameter. Result is many (maybe thousand) of records:

File record segment 3F0F4 is an orphan.

TestDisk doesn't seems to see files on the disk. Only few of them. I am doing Deep Partition Search now.

I also tried to mount the partition in linux and during ls command I see multiple I/O errors.

I attached the hard disk to my PC and interrupted chkdsk on boot. When I view the directory structure looks alright.

I am looking for help how to proceed from here.
My guess is that resize was complete. I mean like information was moved, but partition table wasn't updated and/or shrink wasn't completed.

Please help.

**mfleetwo** · 2022-10-12 23:22:52

mfleetwo
Developer
Offline

Registered: 2012-05-18
Posts: 477

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

So you were shrinking partition X (disk D of Windows) by moving the start to the right by 19GB. To achieve this GParted has to:
1. Shrink the file system by 19G.
2. Copy the now smaller file system over the top of itself 19G to the right.
3. Move the start of the partition 19G to the right.
As we can tell from the unallocated space message step 1 happened, you saw >= 55% of step 2 happen, step 3 didn't happen again because of the unallocated space message.

See thread [Solved] Error during partition resize.
That user got into an equivalent situation by growing the start of the partition to the left and it failing during the copy but before overwriting the file system. You go there by shrinking the start to the right and failing after most (or all) of the file system was copied.

Provide output of the following commands:
(Replace sda with the correct drive as needed)

sudo parted /dev/sda unit s print
sudo fdisk -l /dev/sda
sudo gdisk -l /dev/sda
sudo lsblk -o name,maj:min,rm,size,ro,type,fstype,label,mountpoint

Don't access the file system in any way as it will be working with the 19G fragment at the start of the partition, and what it will treat as corrupted junk after that. However what we hope to find is a full copy of the file system at offset 19G.

**imperia** · 2022-10-13 11:09:39

imperia
Member
Offline

Registered: 2022-10-12
Posts: 14

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

Thank you for your help.
Here is the information:

Model: ATA WDC WD8000AARS-0 (scsi)
Disk /dev/sdc: 1562824368s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 

Number  Start       End          Size         Type     File system  Flags
 1      2048s       206847s      204800s      primary  ntfs         boot
 2      206848s     203758528s   203551681s   primary  ntfs
 3      203759616s  204799999s   1040384s     primary  ntfs         msftres
 4      204800000s  1546528767s  1341728768s  primary  ntfs

Disk /dev/sdc: 745.21 GiB, 800166076416 bytes, 1562824368 sectors
Disk model: WDC WD8000AARS-0
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xb8e324cd

Device     Boot     Start        End    Sectors   Size Id Type
/dev/sdc1  *         2048     206847     204800   100M  7 HPFS/NTFS/exFAT
/dev/sdc2          206848  203758528  203551681  97.1G  7 HPFS/NTFS/exFAT
/dev/sdc3       203759616  204799999    1040384   508M 27 Hidden NTFS WinRE
/dev/sdc4       204800000 1546528767 1341728768 639.8G  7 HPFS/NTFS/exFAT

GPT fdisk (gdisk) version 1.0.9

Partition table scan:
  MBR: MBR only
  BSD: not present
  APM: not present
  GPT: not present


***************************************************************
Found invalid GPT and valid MBR; converting MBR to GPT format
in memory. 
***************************************************************

Disk /dev/sdc: 1562824368 sectors, 745.2 GiB
Model: WDC WD8000AARS-0
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 322A91A0-45AE-4BF0-9FFD-85897AA529FD
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 1562824334
Partitions will be aligned on 2048-sector boundaries
Total free space is 16298668 sectors (7.8 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048          206847   100.0 MiB   0700  Microsoft basic data
   2          206848       203758528   97.1 GiB    0700  Microsoft basic data
   3       203759616       204799999   508.0 MiB   2700  Windows RE
   4       204800000      1546528767   639.8 GiB   0700  Microsoft basic data

NAME   MAJ:MIN RM   SIZE RO TYPE FSTYPE   LABEL           MOUNTPOINT
loop0    7:0    0   3.1G  1 loop squashfs                 /usr/lib/live/mount/rootfs/filesystem.squashfs
sda      8:0    0   1.8T  0 disk                          
└─sda1   8:1    0   1.8T  0 part ntfs                     
sdb      8:16   0 232.9G  0 disk                          
├─sdb1   8:17   0   499M  0 part ntfs     Recovery        
├─sdb2   8:18   0   100M  0 part vfat                     
├─sdb3   8:19   0    16M  0 part                          
├─sdb4   8:20   0 231.7G  0 part ntfs                     
└─sdb5   8:21   0   627M  0 part ntfs                     
sdc      8:32   0 745.2G  0 disk                          
├─sdc1   8:33   0   100M  0 part ntfs     System Reserved 
├─sdc2   8:34   0  97.1G  0 part ntfs                     
├─sdc3   8:35   0   508M  0 part ntfs                     
└─sdc4   8:36   0 639.8G  0 part ntfs     D:              
sdd      8:48   1   3.7G  0 disk iso9660  Kali Live       
├─sdd1   8:49   1   3.6G  0 part iso9660  Kali Live       /usr/lib/live/mount/medium
└─sdd2   8:50   1   896K  0 part vfat                     
sde      8:64   1  28.9G  0 disk                          
└─sde1   8:65   1  28.9G  0 part vfat     WIN11           /mnt

**mfleetwo** · 2022-10-13 16:48:29

mfleetwo
Developer
Offline

Registered: 2012-05-18
Posts: 477

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

Now we need to find the exact location of the NTFS file system super blocks in partition /dev/sdc4. Run the following commands:

1. Report NTFS super block at the start

sudo hexdump -C /dev/sdc4 | fgrep 'NTFS    '

That is 4 spaces after NTFS and before the closing single quote. Will report 1 line of output immediately. Cancel with Ctrl-C.

2. Report NTFS super block at ~19G for the copy of the shrunk file system

sudo hexdump -C /dev/sdc4 -s $((18 * 1024 * 1024 * 1024)) /dev/sdc4 | fgrep 'NTFS    '

Starts reading the partition at 18G until it finds it. Again cancel when it's reported 1 line of output. Expect to take about 5 minutes to read the 1G to get to the 19G point.

3. Report NTFS super block at the end for the copy of the shrunk file system

sudo hexdump -C /dev/sdc4 -s $((639 * 1024 * 1024 * 1024)) /dev/sdc4 | fgrep 'NTFS    '

Again starts about 1G short of the end. Cancel when it reports 1 line of output.

**imperia** · 2022-10-13 20:59:38

imperia
Member
Offline

Registered: 2022-10-12
Posts: 14

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

00000000  eb 52 90 4e 54 46 53 20  20 20 20 00 02 08 00 00  |.R.NTFS    .....|
7cd7a0800  eb 52 90 4e 54 46 53 20  20 20 20 00 02 01 00 00  |.R.NTFS    .....|
9ff24ffe00  eb 52 90 4e 54 46 53 20  20 20 20 00 02 08 00 00  |.R.NTFS    .....|

2nd took some time. Like more than 30 minutes. I left computer and when I came back there were like 10 NTFS lines. 7cd7a0800 was the first one. I will be double checking this, because I am using Kali Linux and It turns off display and is unable to wake it up sometimes. This issue got me into this trouble. I thought it was crashed.

**mfleetwo** · 2022-10-14 12:05:30

mfleetwo
Developer
Offline

Registered: 2012-05-18
Posts: 477

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

That is bad. 7cd7a0800 hex is 33512097792 decimal (~31.21 GiB), not 19G, and not a multiple of whole mebibytes (1024 * 1024). All your partitions are mebibyte aligned and GParted, by defaults, works aligning to whole mebibytes. This doesn't match the 19GiB reported in the unallocated space message from the fragment of the file system remaining at the start of the partition. This means the assumption about the file system having been shrunk by 19G and then *successfully* copied 19G to the right isn't reflected on the hard drive. Note that moving a file system to the right involves copying the data startaing at the end and working backwards to the start. Therefore step 2 copy didn't complete.

Using ASCII art the steps look like this:

sdc4, 639G
|ABCDEFGHIJKLMNOPQRSTUVWXYZ|  <-  Original 639G NTFS file system
|ABCDEFGHIJKLMNOPQRSTUVWX  |  <-  NTFS shrunk by 19G
|        GHIJKLMNOPQRSTUVWX|  <-  Partial copy to the right

|ABCDEFGHGHIJKLMNOPQRSTUVWX|  <-  Result
       ^^^^                   <-  Repeated chunk in unknown location in the middle

The file system is very badly damaged. All resonable recovery options involve having another drive with plenty of space to copy all recoverable files to and / or copy the file system itself. Ideally *all* recovery attempts would be performed on a copy of the partition in order to avoid risk damaging the file system further and reducing the chance of data recovery.
(Also don't use GParted to make this copy. For an NTFS file system it uses a file system specific tool that only copies used blocks, in a valid NTFS file system. That isn't what you have).

The recover options I envision so far are:

1. Mount the file system read-only as is and read all possible files. Expect only a small fraction of files to be recoverable from the start of the partition.

2. Use Windows chkdsk on a copy of the partition to see what it can repair. I have no experience of using chkdsk so I have no idea what it will make of this badly damaged file system. Expect some data loss to total data loss from this.

3. Use Photorec to ignore the file system and scan the block for fragments of recognised documents, movies, pictures and archive files. I have no experience of this but it sounds like there will be a labour intensive step reviewing all possible recovered files at the end.

So do you have the necessary space on another drive?
Is the lost data valuable enough to continue?

Also fix your machine so that is doesn't turn off preventing wakeup leading to more data loss!

**imperia** · 2022-10-14 16:21:48

imperia
Member
Offline

Registered: 2022-10-12
Posts: 14

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

What a nightmare and string of bad luck for me.

I left it overnight scanning whole partition for NTFS strings.

00000000  eb 52 90 4e 54 46 53 20  20 20 20 00 02 08 00 00  |.R.NTFS    .....|
7cd7a0800  eb 52 90 4e 54 46 53 20  20 20 20 00 02 01 00 00  |.R.NTFS    .....|
7e1991e90  61 00 73 00 65 00 00 00  4e 54 46 53 20 20 20 20  |a.s.e...NTFS    |
7e19e3c40  eb 52 90 4e 54 46 53 20  20 20 20 00 02 01 00 00  |.R.NTFS    .....|
7e19e5c40  eb 52 00 4e 54 46 53 20  20 20 20 00 02 01 00 00  |.R.NTFS    .....|
7e1a584b0  4e 54 46 53 20 20 20 20  00 00 00 00 25 31 30 75  |NTFS    ....%10u|
7ef40dad0  73 00 00 00 4e 54 46 53  20 20 20 20 00 00 00 00  |s...NTFS    ....|
7f31517a0  73 00 00 00 4e 54 46 53  20 20 20 20 00 00 00 00  |s...NTFS    ....|
7f4c4f980  eb 52 00 4e 54 46 53 20  20 20 20 00 02 01 00 00  |.R.NTFS    .....|
7f4c52980  eb 52 90 4e 54 46 53 20  20 20 20 00 02 01 00 00  |.R.NTFS    .....|
10b6bef790  eb 52 90 4e 54 46 53 20  20 20 20 00 02 08 00 00  |.R.NTFS    .....|
13a5c731c0  4e 54 46 53 20 20 20 20  00 00 00 00 5b 00 55 00  |NTFS    ....[.U.|
13b47478e0  00 00 00 00 d9 4e 54 46  53 20 20 20 20 00 00 00  |.....NTFS    ...|
13b4747920  46 00 53 00 00 00 00 00  4e 54 46 53 20 20 20 20  |F.S.....NTFS    |
13b6a171c0  4e 54 46 53 20 20 20 20  00 00 00 00 5b 00 55 00  |NTFS    ....[.U.|
13b7d32e00  20 4e 54 46 53 20 20 20  20 20 20 20 20 20 32 30  | NTFS         20|
13b7d466e0  20 20 20 20 4e 54 46 53  20 20 20 20 20 20 20 20  |    NTFS        |
13b7d46740  20 20 20 20 20 20 20 4e  54 46 53 20 20 20 20 20  |       NTFS     |
1455689440  74 00 09 00 4e 54 46 53  20 20 20 20 46 49 4c 45  |t...NTFS    FILE|
1dfb545e90  61 00 73 00 65 00 00 00  4e 54 46 53 20 20 20 20  |a.s.e...NTFS    |
1dfb597c40  eb 52 90 4e 54 46 53 20  20 20 20 00 02 01 00 00  |.R.NTFS    .....|
1dfb599c40  eb 52 00 4e 54 46 53 20  20 20 20 00 02 01 00 00  |.R.NTFS    .....|
1dfb60c4b0  4e 54 46 53 20 20 20 20  00 00 00 00 25 31 30 75  |NTFS    ....%10u|
1e094b4ad0  73 00 00 00 4e 54 46 53  20 20 20 20 00 00 00 00  |s...NTFS    ....|
1e0d2867a0  73 00 00 00 4e 54 46 53  20 20 20 20 00 00 00 00  |s...NTFS    ....|
1e0edbe980  eb 52 00 4e 54 46 53 20  20 20 20 00 02 01 00 00  |.R.NTFS    .....|
1e0edc1980  eb 52 90 4e 54 46 53 20  20 20 20 00 02 01 00 00  |.R.NTFS    .....|
21c54f8800  eb 52 90 4e 54 46 53 20  20 20 20 00 02 01 00 00  |.R.NTFS    .....|
2c367d43c0  eb fe 55 aa eb 52 90 4e  54 46 53 20 20 20 20 00  |..U..R.NTFS    .|
2c36eac3c0  eb fe 55 aa eb 52 90 4e  54 46 53 20 20 20 20 00  |..U..R.NTFS    .|
2c3849e3c0  eb fe 55 aa eb 52 90 4e  54 46 53 20 20 20 20 00  |..U..R.NTFS    .|
2d20c2b590  00 23 26 40 01 00 00 00  4e 54 46 53 20 20 20 20  |.#&@....NTFS    |
2dcb8cbd00  00 86 12 80 01 00 00 00  4e 54 46 53 20 20 20 20  |........NTFS    |
2ea00b3090  4e 54 46 53 20 20 20 20  00 00 00 00 00 00 00 00  |NTFS    ........|
hexdump: /dev/sdc4: Input/output error

Notice the last line The hard drive had bad sector or something. I guess this is the reason the original move/copy operation failed. This is after the 55% of the that was copied/moved.
hexdump: /dev/sdc4: Input/output error

Here is the dmesg:

[54175.290033] ata3.00: exception Emask 0x0 SAct 0x180000 SErr 0x0 action 0x0
[54175.290046] ata3.00: irq_stat 0x40000008
[54175.290052] ata3.00: failed command: READ FPDMA QUEUED
[54175.290054] ata3.00: cmd 60/00:98:e0:1c:8d/01:00:25:00:00/40 tag 19 ncq dma 131072 in
                        res 41/40:00:0f:1d:8d/00:00:25:00:00/40 Emask 0x409 (media error) <F>                                                                           
[54175.290065] ata3.00: status: { DRDY ERR }
[54175.290068] ata3.00: error: { UNC }
[54175.295464] ata3.00: configured for UDMA/133
[54175.295485] sd 2:0:0:0: [sdc] tag#19 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[54175.295491] sd 2:0:0:0: [sdc] tag#19 Sense Key : Medium Error [current] 
[54175.295494] sd 2:0:0:0: [sdc] tag#19 Add. Sense: Unrecovered read error - auto reallocate failed
[54175.295498] sd 2:0:0:0: [sdc] tag#19 CDB: Read(10) 28 00 25 8d 1c e0 00 01 00 00
[54175.295500] I/O error, dev sdc, sector 630005007 op 0x0:(READ) flags 0x80700 phys_seg 7 prio class 0
[54175.295523] ata3: EH complete
[54175.494023] ata3.00: exception Emask 0x0 SAct 0x10000000 SErr 0x0 action 0x0
[54175.494036] ata3.00: irq_stat 0x40000008
[54175.494050] ata3.00: failed command: READ FPDMA QUEUED
[54175.494052] ata3.00: cmd 60/08:e0:08:1d:8d/00:00:25:00:00/40 tag 28 ncq dma 4096 in
                        res 41/40:00:0f:1d:8d/00:00:25:00:00/40 Emask 0x409 (media error) <F>                                                                           
[54175.494059] ata3.00: status: { DRDY ERR }
[54175.494060] ata3.00: error: { UNC }
[54175.499427] ata3.00: configured for UDMA/133
[54175.499449] sd 2:0:0:0: [sdc] tag#28 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[54175.499454] sd 2:0:0:0: [sdc] tag#28 Sense Key : Medium Error [current] 
[54175.499466] sd 2:0:0:0: [sdc] tag#28 Add. Sense: Unrecovered read error - auto reallocate failed
[54175.499469] sd 2:0:0:0: [sdc] tag#28 CDB: Read(10) 28 00 25 8d 1d 08 00 00 08 00
[54175.499470] I/O error, dev sdc, sector 630005007 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[54175.499475] Buffer I/O error on dev sdc4, logical block 53150625, async page read
[54175.499486] ata3: EH complete

I will find free space to try any recovery options.
This is my boss's old hard drive.
I guess it is valuable.

**mfleetwo** · 2022-10-15 00:47:06

mfleetwo
Developer
Offline

Registered: 2012-05-18
Posts: 477

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

Yes, when GParted encountered the I/O error from the faulty sector it would have failed at that point. GParted should have rolled back all the data copying it had don't up to that point but it looks like that failed too. Without Save Details we'll never know.

This from dmesg "Unrecovered read error - auto reallocate failed" says it failed to read a sector from the hard drive. Run smartctl and query the S.M.A.R.T. Attributes.

sudo smartctl -A /dev/sdc

I expect you will have at least 1 sector counted as:
ID# 197 Current_Pending_Sector
ID# 198 Offline_Uncorrectable

Because of this you first need to block copy the partition sdc4 to a partition on a new drive that is equal or larger than 1341728768 512-byte sectors. You need to do it using ddrescue. It is usually available as an installable package in most Linux distributions.
Run as:

sudo ddrescue -n /dev/sdc4 /dev/NEWPARTITION rescue.log
sudo ddrescue -d -r3 /dev/sdc4 /dev/NEWPARTITION rescue.log

The first ddrescue will do a fast first pass copy. The second ddrescue will try down to the individual sector up to 3 times to get portions failed in the first pass.

**mfleetwo** · 2022-10-15 01:06:27

mfleetwo
Developer
Offline

Registered: 2012-05-18
Posts: 477

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

An "Unrecovered read" only means that the 512-bytes when read didn't match the checksum so the drive known it doesn't know what the data should be, hence the error. After ddrescue has copied the partition it can be corrected by simply writing new data to the sector. As long as there is only 1 or a few such sectors the disk will be as safe as any other to continue to use.

**imperia** · 2022-10-16 10:02:34

imperia
Member
Offline

Registered: 2022-10-12
Posts: 14

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

Here is the full smartctl output:

smartctl 7.3 2022-02-28 r5338 [x86_64-linux-5.18.0-kali5-amd64] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       320
  3 Spin_Up_Time            0x0027   134   124   021    Pre-fail  Always       -       6291
  4 Start_Stop_Count        0x0032   087   087   000    Old_age   Always       -       13351
  5 Reallocated_Sector_Ct   0x0033   199   199   140    Pre-fail  Always       -       2
  7 Seek_Error_Rate         0x002e   100   253   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   055   055   000    Old_age   Always       -       33560
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   087   087   000    Old_age   Always       -       13038
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       567
193 Load_Cycle_Count        0x0032   001   001   000    Old_age   Always       -       1096198
194 Temperature_Celsius     0x0022   119   096   000    Old_age   Always       -       28
196 Reallocated_Event_Count 0x0032   199   199   000    Old_age   Always       -       1
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       1
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       1
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       1

I will try to copy the partition to new drive now.

**imperia** · 2022-10-17 18:06:17

imperia
Member
Offline

Registered: 2022-10-12
Posts: 14

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

I installed gddrescue package (GNU ddrescue). ddrescue is version 1.26.

sudo ddrescue -n /dev/sdc4 /dev/NEWPARTITION rescue.log

took 9 hours. I created empty sdb6 partition because else it creates file in /dev directory and doesn't write to partition. I had to use --force parameter too. I wonder why it too so much time.

After that when I ran:

sudo ddrescue -d -r3 /dev/sdc4 /dev/NEWPARTITION rescue.log

it took like 3-4 seconds to complete.

Now I ran it again today and it seems like it is doing the whole partition again.
I'll leave it going. It is 20% completed for 30 minutes.

Here is the current rescue.log file:

# Mapfile. Created by GNU ddrescue version 1.26
# Command line: ddrescue -d -r3 /dev/sdc4 /dev/sdb6 rescue.log --force
# Start time:   2022-10-17 18:38:46
# Current time: 2022-10-17 18:56:02
# Copying non-tried blocks... Pass 1 (forwards)
# current_pos  current_status  current_pass
0x13E3240000     ?               1
#      pos        size  status
0x00000000  0x13E3250000  +
0x13E3250000  0x8C0F2B0000  ?

And this was the first -n parameter run:

 Mapfile. Created by GNU ddrescue version 1.26
# Command line: ddrescue -n /dev/sdc4 /dev/sdb6 rescue.log
# Start time:   2022-10-16 11:23:09
# Current time: 2022-10-16 11:23:51
# Copying non-tried blocks... Pass 1 (forwards)
# current_pos  current_status  current_pass
0xF4310000     ?               1
#      pos        size  status
0x00000000  0xF4310000  +
0xF4310000  0x9EFE1F0000  ?

**mfleetwo** · 2022-10-18 00:06:00

mfleetwo
Developer
Offline

Registered: 2012-05-18
Posts: 477

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

imperia wrote:

And this was the first -n parameter run:

 Mapfile. Created by GNU ddrescue version 1.26
# Command line: ddrescue -n /dev/sdc4 /dev/sdb6 rescue.log
# Start time:   2022-10-16 11:23:09
# Current time: 2022-10-16 11:23:51
# Copying non-tried blocks... Pass 1 (forwards)
# current_pos  current_status  current_pass
0xF4310000     ?               1
#      pos        size  status
0x00000000  0xF4310000  +
0xF4310000  0x9EFE1F0000  ?

This output records the situation ~4G and 42 seconds into the first copy. Not a completed 639G and 9 hour copy.

imperia wrote:

Here is the current rescue.log file:

# Mapfile. Created by GNU ddrescue version 1.26
# Command line: ddrescue -d -r3 /dev/sdc4 /dev/sdb6 rescue.log --force
# Start time:   2022-10-17 18:38:46
# Current time: 2022-10-17 18:56:02
# Copying non-tried blocks... Pass 1 (forwards)
# current_pos  current_status  current_pass
0x13E3240000     ?               1
#      pos        size  status
0x00000000  0x13E3250000  +
0x13E3250000  0x8C0F2B0000  ?

And this is (still) doing pass 1, ~85G and 27 minutes into the copy.

Each run of ddrescue should use the same log file. It records the successfully copied blocks and sectors and on subsequent runs / passes it only attempts sectors it hasn't already copied.

Is there a reason ddrescue's log either didn't write successfully for the first run or the second run isn't using the same log?

**imperia** · 2022-10-18 05:53:21

imperia
Member
Offline

Registered: 2022-10-12
Posts: 14

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

┌──(root㉿kali)-[~]
└─# ddrescue -d -r3 /dev/sdc4 /dev/sdb6 rescue.log --force
GNU ddrescue 1.26
Press Ctrl-C to interrupt
Initial status (read from mapfile)
rescued: 4096 MB, tried: 0 B, bad-sector: 0 B, bad areas: 0

Current status
     ipos:  686965 MB, non-trimmed:    65536 B,  current rate:  62521 kB/s
     opos:  686965 MB, non-scraped:        0 B,  average rate:  31414 kB/s
non-tried:    6881 kB,  bad-sector:        0 B,    error rate:       0 B/s
  rescued:  686958 MB,   bad areas:        0,        run time:  6h  2m 16s
pct rescued:   99.99%, read errors:        1,  remaining time:          1s
                              time since last successful read:          0s
Copying non-tried blocks... Pass 1 (forwards)
     ipos:  217705 MB, non-trimmed:    65536 B,  current rate:  44630 kB/s
     opos:  217705 MB, non-scraped:        0 B,  average rate:  31414 kB/s
non-tried:        0 B,  bad-sector:        0 B,    error rate:       0 B/s
  rescued:  686965 MB,   bad areas:        0,        run time:  6h  2m 17s
pct rescued:   99.99%, read errors:        1,  remaining time:          1s
                              time since last successful read:          0s
Copying non-tried blocks... Pass 2 (backwards)
     ipos:  217704 MB, non-trimmed:        0 B,  current rate:   22528 B/s
     opos:  217704 MB, non-scraped:        0 B,  average rate:  31410 kB/s
non-tried:        0 B,  bad-sector:      512 B,    error rate:       0 B/s
  rescued:  686965 MB,   bad areas:        1,        run time:  6h  2m 20s
pct rescued:   99.99%, read errors:        2,  remaining time:          1s
                              time since last successful read:          0s
Trimming failed blocks... (forwards)          
     ipos:  217704 MB, non-trimmed:        0 B,  current rate:   22528 B/s
     opos:  217704 MB, non-scraped:        0 B,  average rate:  31410 kB/s
non-tried:        0 B,  bad-sector:      512 B,    error rate:       0 B/s
  rescued:  686965 MB,   bad areas:        1,        run time:  6h  2m 20s
pct rescued:   99.99%, read errors:        3,  remaining time:          1s
                              time since last successful read:          0s
Retrying bad sectors... Retry 1 (forwards)
     ipos:  217704 MB, non-trimmed:        0 B,  current rate:   22528 B/s
     opos:  217704 MB, non-scraped:        0 B,  average rate:  31410 kB/s
non-tried:        0 B,  bad-sector:      512 B,    error rate:       0 B/s
  rescued:  686965 MB,   bad areas:        1,        run time:  6h  2m 20s
pct rescued:   99.99%, read errors:        4,  remaining time:          1s
                              time since last successful read:          0s
Retrying bad sectors... Retry 2 (backwards)
     ipos:  217704 MB, non-trimmed:        0 B,  current rate:       0 B/s
     opos:  217704 MB, non-scraped:        0 B,  average rate:  31407 kB/s
non-tried:        0 B,  bad-sector:      512 B,    error rate:       0 B/s
  rescued:  686965 MB,   bad areas:        1,        run time:  6h  2m 21s
pct rescued:   99.99%, read errors:        5,  remaining time:          1s
                              time since last successful read:          0s
Retrying bad sectors... Retry 3 (forwards) 
Finished

This is last run
PS. I am posting this now before I go to work. I will answer your message a bit later, Because I have to go.

**imperia** · 2022-10-18 07:42:45

imperia
Member
Offline

Registered: 2022-10-12
Posts: 14

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

└─# cat /root/rescue.log
# Mapfile. Created by GNU ddrescue version 1.26
# Command line: ddrescue -d -r3 /dev/sdc4 /dev/sdb6 rescue.log --force
# Start time:   2022-10-17 18:38:46
# Current time: 2022-10-18 00:41:12
# Finished
# current_pos  current_status  current_pass
0x32B03A1E00     +               3
#      pos        size  status
0x00000000  0x32B03A1E00  +
0x32B03A1E00  0x00000200  -
0x32B03A2000  0x6D4215E000  +

**imperia** · 2022-10-19 08:04:33

imperia
Member
Offline

Registered: 2022-10-12
Posts: 14

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

So, I am not quite sure why this happened with previous log files, but the last one should be alright. The partition is now copied to new drive. As far as I understand I have 1 bad sector.

I red in this thread Re: [Solved] Error during partition resize
that GParted

stopped because of an I/O error, and clicking on Retry, Cancel or Ignore had no effect.

So it wont roll back operation without user intervention in case of bad sector. I guess at this time I restarted the computer thinking it was crashed. (I was using 1.2.0 version of GParted in case something changed in the logic on the program)

If I am right I should be able to sew back H to I and eliminate the "Repeated chunk in unknown location in the middle"
(removing the duplicated GH part)

|ABCDEFGHGHIJKLMNOPQRSTUVWX|  <-  Result
       ^^^^                   <-  Repeated chunk in unknown location in the middle

I saw some guys did it with scripts like:
Re: [SOLVED] gparted crashed during resizing

and:
gparted freeze during NTFS partition resize
this guy made scripts to findDuplicateSector.py and to findCopyInterruptLocation.py

I need your help to tell me if my logic is right and if this is going to work and also to fill the sector values in the .py code.
And eventually how to combine the parts together.

Thank you for your help.

**mfleetwo** · 2022-10-19 13:51:46

mfleetwo
Developer
Offline

Registered: 2012-05-18
Posts: 477

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

imperia wrote:

# cat /root/rescue.log
# Mapfile. Created by GNU ddrescue version 1.26
# Command line: ddrescue -d -r3 /dev/sdc4 /dev/sdb6 rescue.log --force
# Start time:   2022-10-17 18:38:46
# Current time: 2022-10-18 00:41:12
# Finished
# current_pos  current_status  current_pass
0x32B03A1E00     +               3
#      pos        size  status
0x00000000  0x32B03A1E00  +
0x32B03A1E00  0x00000200  -
0x32B03A2000  0x6D4215E000  +

This ddrescue log file says ~639 GiB were copied successfully with just 1 512-byte sector (0x00000200) failing at byte offset 217704963584 (0x32B03A1E00).

On the assumption that you didn't fill partition sdb6 with zeros before you started copying to it, or otherwise know partition sdb6 was filled with zeros, you should write 1 512-byte sector of zeros to that non-copied sector. Otherwise existing data may look like corrupted data in the file system. Zeros will still be corrupted data in the file system but clearly doesn't represent either valid file content or file system metadata.

Check what the sector contains with:

dd if=/dev/sdb6 bs=512 skip=$((0x32B03A1E00 / 512)) count=1 2> /dev/null | hexdump -C

And write zeros with:

dd if=/dev/zero bs=512 seek=$((0x32B03A1E00 / 512)) count=1 of=/dev/sdb6

Yes it should be possible to identify the repeated chunk in the file system and join it back together. I'm still to fully understand those two examples and will advise further when I do.

**imperia** · 2022-10-19 14:30:49

imperia
Member
Offline

Registered: 2022-10-12
Posts: 14

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

sdb6 partition was only about 10 percent used, and the output of the command:

dd if=/dev/sdb6 bs=512 skip=$((0x32B03A1E00 / 512)) count=1 2> /dev/null | hexdump -C
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000200

so I don't need to zero fill it i guess.

**mfleetwo** · 2022-10-20 11:53:32

mfleetwo
Developer
Offline

Registered: 2012-05-18
Posts: 477

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

1. Re: [SOLVED] gparted crashed during resizing

This first case was an ext2/3/4 file system and they used a tool from The Sleuth Kit which reported locations of it's internal metadata to work out where the repeated data was. I don't know about Sleuth Kit or whether it provides a similar tool for NTFS or how to use it.

2. gparted freeze during NTFS partition resize

This second case seemed to know within a range where GParted had gotten up to with it's copy step from what the UI showed them. The findDuplicateSector.py script needs to know a sector that will be duplicated. We don't know that and I don't know how to examine the NTFS internals to try to work that out. In the general case of checking every sector matches every other sector takes reading your 639G partition 1340080128 times! ETA ~1 million years!

**mfleetwo** · 2022-10-20 11:56:21

mfleetwo
Developer
Offline

Registered: 2012-05-18
Posts: 477

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

Instead I wrote matching-blocks.py script. It reads it's input once and reports which blocks match, or not, with which other blocks. It doesn't care about what the input contains.

Save the script and make it executable. It should run as both python 2 and 3. Run as:

matching-blocks.py < /dev/sdb6 > matching.log

It should take approximately as long to run as it takes to read the whole of the partition once and do an md5sum on it.

Progress can be monitored by using the standard capability of Linux to report a processes position within an open file handle.

cat /proc/$MYPID/fdinfo/$HANDLE

After matching-blocks.py has been started, run this for a "nice" progress indicator:

size=`blockdev --getsize /dev/sdb6`
pid=`ps -ef | awk '/awk/ {next} /matching-blocks.py/ {print $2}'`
watch "awk '/pos:/ {pos=\$2/512} END {printf \"%d / %d  %.2f%%\", pos, $size, pos*100/$size}' /proc/$pid/fdinfo/0"

**imperia** · 2022-10-20 15:01:12

imperia
Member
Offline

Registered: 2022-10-12
Posts: 14

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

Operation completed. List is quite big.
https://we.tl/t-se0tTOxZaA

**mfleetwo** · 2022-10-20 18:24:12

mfleetwo
Developer
Offline

Registered: 2012-05-18
Posts: 477

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

I've analysed the data.

[user@localhost]$ zcat matching.log.gz | ./analyse-matches.sh
Num of  Offset to single
blocks  matching copy
------- ----------------
  19585 -19893
  19585 19893
     86 -86255
     86 86255
     17 -536
     17 536
      8 -9
      8 9
      7 -4917
      7 4917

There are 19585 1 MiB blocks with a single duplicate copy where the offset between both sets of copies is +/-19893. This is a very clear signal. The overwriting copy was moving the start of the file system by 19893 MiBs.

Back in the matching.log output, blocks with a single matching copy at offset of +/-19893 are:

Block   deltas to matching blocks
------- -------------------------
 207632 [19893]
...
 227524 [19893]
 227525 [-19893]
...
 247417 [-19893]

So it's like this:
0 - 207631 MiB - Start of FS, single copy
207632 - 227524 MiB - More of FS, first copy of duplicate section
227525 - 247417 MiB - More of FS, second copy of duplicate section
247417 - 748452 MiB - End of FS, single copy

Now we just need to plan the actions to put the two pieces of the file system back together properly.

**mfleetwo** · 2022-10-20 18:39:15

mfleetwo
Developer
Offline

Registered: 2012-05-18
Posts: 477

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

My high level plan is this:

1. Zero fill the original partition, /dev/sdc4, to get that unreadable block re-written and make sure the disk is good.
Also check SMART attributes for healthy drive indicators.

2. Copy the 2 pieces of the file system back together into partition /dev/sdc4.

3. In Windows chkdsk sdc4.

Does this sound okay?

**imperia** · 2022-10-20 18:53:19

imperia
Member
Offline

Registered: 2022-10-12
Posts: 14

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

I am not sure I am going to use the original disk anymore (sdc4). I have more hard drives available to reconstruct the data to.
So I think to install 3rd new disk to copy data from sdb6 to it or from sdc4 skipping the "empty part" and writing 00 in the place of the bad sector.
Or eventually I can try to use HDD Regenerator to try to restore that bad sector. It really restored some hard drives bad sectors for me before.

PS. Or reconstruct the data to sbd6, try to recover that bad sector with the program I mentioned and copy back to original hard drive if bad sector is fixed.

**imperia** · 2022-10-24 15:25:27

imperia
Member
Offline

Registered: 2022-10-12
Posts: 14

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

Hello, I was away few days.

Can you help me out with the commands to copy the parts and join them together on new disk?
And also eventually if you can tell me the sector of the disk that is bad.
I don't want to try to scan and repair the whole disk. Just this broken sector.
Program accepts sectors as input. Which one should I use? The one from dmesg error message probably?

I/O error, dev sdc, sector 630005007 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0p

**mfleetwo** · 2022-10-24 17:30:32

mfleetwo
Developer
Offline

Registered: 2012-05-18
Posts: 477

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

Sorry, I've been very busy with my $job over the last few days.

ddrescue log said byte offset 0x32B03A1E00 into partition /dev/sdc4 was the bad one.
0x32B03A1E00 hex = 217704963584 dec.
217704963584 / 512 = 425205007 sector within /dev/sdc4.

fdisk output said partition /dev/sdc4 starts at sector 204800000.
425205007 + 204800000 = 630005007.

Bad sector on drive sdc is: 630005007

This also matches the sector number reported with the "Unrecovered read error" in the dmesg output too.

[SOLVED] Unallocated space and corrupted NTFS after resize (Page 1 of 2)

Posts: 1 to 25 of 31

1 Topic by imperia 2022-10-12 18:48:04 (edited by mfleetwo 2022-10-29 09:37:53)

Topic: [SOLVED] Unallocated space and corrupted NTFS after resize

2 Reply by mfleetwo 2022-10-12 23:22:52

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

3 Reply by imperia 2022-10-13 11:09:39

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

4 Reply by mfleetwo 2022-10-13 16:48:29

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

5 Reply by imperia 2022-10-13 20:59:38 (edited by imperia 2022-10-13 21:01:05)

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

6 Reply by mfleetwo 2022-10-14 12:05:30

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

7 Reply by imperia 2022-10-14 16:21:48

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

8 Reply by mfleetwo 2022-10-15 00:47:06

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

9 Reply by mfleetwo 2022-10-15 01:06:27

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

10 Reply by imperia 2022-10-16 10:02:34

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

11 Reply by imperia 2022-10-17 18:06:17 (edited by imperia 2022-10-17 18:17:15)

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

12 Reply by mfleetwo 2022-10-18 00:06:00

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

13 Reply by imperia 2022-10-18 05:53:21 (edited by imperia 2022-10-18 06:11:37)

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

14 Reply by imperia 2022-10-18 07:42:45

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

15 Reply by imperia 2022-10-19 08:04:33 (edited by imperia 2022-10-19 08:06:11)

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

16 Reply by mfleetwo 2022-10-19 13:51:46

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

17 Reply by imperia 2022-10-19 14:30:49

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

18 Reply by mfleetwo 2022-10-20 11:53:32

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

19 Reply by mfleetwo 2022-10-20 11:56:21

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

20 Reply by imperia 2022-10-20 15:01:12

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

21 Reply by mfleetwo 2022-10-20 18:24:12

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

22 Reply by mfleetwo 2022-10-20 18:39:15

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

23 Reply by imperia 2022-10-20 18:53:19 (edited by imperia 2022-10-20 19:07:39)

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

24 Reply by imperia 2022-10-24 15:25:27

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

25 Reply by mfleetwo 2022-10-24 17:30:32

Re: [SOLVED] Unallocated space and corrupted NTFS after resize

Posts: 1 to 25 of 31