Large File System offline

Issues related to hardware problems
Post Reply
cciardi
Posts: 3
Joined: 2012/08/29 18:18:04

Large File System offline

Post by cciardi » 2012/10/31 01:03:18

I support a CentOS 5.3 install that mounts iSCSI partition which is 5.4Tb, formatted EXT3. This was set up more than 2 years ago and has been online since. The owner of the server wanted to shutdown the equipment due to the impending storm. When the restart happened, the server mounted the partition but immediately started reporting errors.

Oct 30 20:30:26 mail kernel: kjournald starting. Commit interval 5 seconds
Oct 30 20:30:26 mail kernel: EXT3-fs warning (device sdb1): ext3_clear_journal_err: Filesystem error recorded from previous mount: IO failure
Oct 30 20:30:26 mail kernel: EXT3-fs warning (device sdb1): ext3_clear_journal_err: Marking fs in need of filesystem check.
Oct 30 20:30:26 mail kernel: EXT3-fs warning: mounting fs with errors, running e2fsck is recommended
Oct 30 20:30:26 mail kernel: EXT3 FS on sdb1, internal journal
Oct 30 20:30:26 mail kernel: EXT3-fs: mounted filesystem with ordered data mode.
Oct 30 20:31:08 mail kernel: attempt to access beyond end of device
Oct 30 20:31:08 mail kernel: sdb1: rw=0, want=5115740184, limit=3081962575
Oct 30 20:31:08 mail kernel: EXT3-fs error (device sdb1): ext3_get_inode_loc: unable to read inode block - inode=319733761, block=639467522
Oct 30 20:31:08 mail kernel: Aborting journal on device sdb1.
Oct 30 20:31:08 mail kernel: attempt to access beyond end of device
Oct 30 20:31:08 mail kernel: sdb1: rw=0, want=7095975960, limit=3081962575
Oct 30 20:31:08 mail kernel: EXT3-fs error (device sdb1): ext3_get_inode_loc: unable to read inode block - inode=443498497, block=886996994

Although the file system appears mounted. There are obvious errors when
[root ~]# ls -l /mnt/iscsi1/
total 16
?--------- ? ? ? ? ? backup
drwx------ 2 root root 16384 Nov 16 2010 lost+found
?--------- ? ? ? ? ? terastore

I attempted an fsck.ext3 but got this error and aborted

[root ~]# fsck.ext3 /dev/sdb1
e2fsck 1.39 (29-May-2006)
The filesystem size (according to the superblock) is 1458987145 blocks
The physical size of the device is 385245321 blocks
Either the superblock or the partition table is likely to be corrupt!
Abort? yes

So I started looking around at the partition.

[root ~]# fdisk -l /dev/sdb

WARNING: The size of this disk is 6.0 TB (5976019304448 bytes).
DOS partition table format can not be used on drives for volumes
larger than 2.2 TB (2199023255040 bytes). Use parted(1) and GUID
partition table format (GPT).


Disk /dev/sdb: 5976.0 GB, 5976019304448 bytes
255 heads, 63 sectors/track, 726542 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdb1 1 191844 1540981287+ 83 Linux

It was created with a GPT partition table, since the partition was greater than 2.2Tb. So then I checked it with parted.

[root ~]# parted /dev/sdb print

Model: BUFFALO VIRTUAL-DISK (scsi)
Disk /dev/sdb: 5976GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags
1 32.3kB 1578GB 1578GB primary ext3

I noticed the partition table was labeled MSDOS and while the disk size is correct, the partition size of slice 1 is obviously off.


At this point I'm stuck, I'm hesitant to force the fsck, as obviously parted is not seeing the partition as GPT.
I did run into gdisk which might have some functions of converting msdos partition to GPT, but I'm not sure that's the answer either.
So how would the partition change from GPT to msdos and more importantly, what suggestions might you have so I can mount this file-system.
Thanks -Chris

Post Reply