Centos server giving I/O Error

Issues related to hardware problems
Post Reply
mashkoorqadir
Posts: 70
Joined: 2012/06/27 07:55:01

Centos server giving I/O Error

Post by mashkoorqadir » 2014/06/05 08:19:49

Hi guys,

My CentOS 5.4 server is running since 2009 working fine since yesterday. it suddenly stop responding. its very critical server for the company. I saw repeated following line and was unable to stop that.

/etc/rc.d/rc.sysinit: line 947: /bin/usleep: Input/Output error
This line just repeats over and over and I cant stop it with control-c it.
when i reboot the machine it failed mount local partitions and stuck. one more time i rebooted the message on the screen was,


setting host name xyz
setting up logical volume mangment:
/1: superblock has an invalid ext3 journal (inode 8)
clear
*** ext3 journal has been deleted - file system in ext2 only ***
Error reading block 2056 (attempt to read block from filesystem resulted in short read). Remounting root file system in
read-write mode. ok
Mounting local filesystems Failed
Enabling local filesystem quotas: Failed
Enabling /etc/fstab swaps: Failed

and hang at this stage. Again i rebooted this time server started and loaded most of the process at login stage it gave following messages.
end_request: I/O error, dev sdb, sector 247392428
EXT3-fs error (device sda3): ext3_find_entry: reading directory 528278814 offset 0

end_request: I/O error, dev sdb, sector 247392428
EXT3-fs error (device sda3): ext3_find_entry: reading directory 528278814 offset 0

end_request: I/O error, dev sdb, sector 247392428
EXT3-fs error (device sda3): ext3_find_entry: reading directory 528278814 offset 0

end_request: I/O error, dev sdb, sector 71
end_request: I/O error, dev sdb, sector 63
end_request: I/O error, dev sdb, sector 17414208
end_request: I/O error, dev sdb, sector 17414440
end_request: I/O error, dev sdb, sector 1028160
end_request: I/O error, dev sdb, sector 1028168
end_request: I/O error, dev sdb, sector 1028160
end_request: I/O error, dev sdb, sector 625137212
end_request: I/O error, dev sdb, sector 625137324
end_request: I/O error, dev sdb, sector 17414460
end_request: I/O error, dev sdb, sector 17414468
end_request: I/O error, dev sdb, sector 17414460
INIT: Id "x" respawaning toofast: disabled for 5 mints

I attached this DISK to an other running system and try to run the fsck command i was unable to run, the following error popups.
e2fsck: Attempt to read block from filesystem resulted in short read while to open /dev/sdb
Could this be a zero-length partition

where as fdisk -l displays only one drive.(its own drive)
and i am also unable to mount the Drive. when saw the disk with GUI interface with Disk management utility the disk shows unknown partition types.
I have googled this and gotten all sorts of ideas. Evey thing from bad RAM to bad DISK to config problems. But no changes were made.
How it could be fixed. Please help.

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Centos server giving I/O Error

Post by TrevorH » 2014/06/05 09:08:12

Looks to me like your hard disk is dying or dead. Run smartctl -a /dev/sdb (change the device name according to where it's now located) and see what that reports. If it fails the SMART test then the disk is pretty much dead and you can mostly write off the data that was on it. You might be able to recover some of it using a utility called dd_rescue but it would probably be quicker to buy a new disk (almost certainly required anyway) and restore from your backups.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

mashkoorqadir
Posts: 70
Joined: 2012/06/27 07:55:01

Re: Centos server giving I/O Error

Post by mashkoorqadir » 2014/06/05 10:33:50

Thanks dear TrevorH,

is there any way to rescue my important data is this case.

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Centos server giving I/O Error

Post by TrevorH » 2014/06/05 10:43:16

Depends on how much you are willing to spend. You can try dd_rescue to copy what can be read (if anything) from the disk to a file on a different disk and then try to recover from there. You can send the disk to a data recovery company who will charge you a lot of money and probably recover more. If you choose the latter then be aware that the more you attempt yourself, it's likely that your attempts will damage the disk more and lessen the chances of data recovery.

Or restore from backups.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

Post Reply