Very slow file operations

Installing, Configuring, Troubleshooting server daemons such as Web and Mail
Post Reply
abhishek76
Posts: 4
Joined: 2015/02/18 17:16:05

Very slow file operations

Post by abhishek76 » 2015/02/18 17:55:28

I recently rebuild a machine with Centos 5.8 with a brand new disk. This machine is old and has been running 5.8 for a long period with no issues so the only thing which changed recently is the new much larger disk. However all the file operations are extremely slow since the machine was rebuilt with the new disk. My knowledge about OS admin stuff is very limited and I will appreciate if someone can point me to where should I be looking at for finding the root cause. What commands or utilities I could use?

There is a java based back up program running on this machine which is constantly reading from the disk but the same program was running on this machine before the rebuild too and machine has been behaving fine for almost 2 years while this program ran at all times. But now the machine is not very usable and even the back up program is crawling.

This is the output of the iostat command:
avg-cpu: %user %nice %system %iowait %steal %idle
0.67 0.41 1.70 52.66 0.00 44.56

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 24.87 197.21 53.98 238224706 65205070
sda1 0.00 0.00 0.00 2266 14
sda2 24.87 197.21 53.98 238222160 65205056
sdb 91.76 138.66 903.42 167496875 1091305280
sdb1 91.76 138.66 903.42 167496635 1091305280
dm-0 157.98 335.87 957.40 405716746 1156511560
dm-1 0.00 0.00 0.00 1376 216

And here is the output from the iotop command:
8723 be/4 root 0.00 B/s 0.00 B/s 0.00 % 98.49 % [pdflush]
588 be/3 root 0.00 B/s 0.00 B/s 0.00 % 75.71 % [kjournald]
8 be/3 root 0.00 B/s 0.00 B/s 0.00 % 5.19 % [events/0]
13161 be/4 root 0.00 B/s 0.00 B/s 0.00 % 1.25 % java -Dfile.encoding=UTF-8 -Dapp=CrashPla~ang:./skin com.backup42.desktop.CPDesktop
10668 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.63 % artsd -F 10 -S 4096 -s 60 -m artsmessage -l 3 -f
16681 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.09 % sshd: root [priv]
3792 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.09 % python -E /usr/bin/sealert -s
13060 be/7 root 0.00 B/s 0.00 B/s 0.00 % 0.09 % java -Dfile.encoding=UTF-8 -Dapp=CrashPla~hplan/lang com.backup42.service.CPService
13141 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % java -Dfile.encoding=UTF-8 -Dapp=CrashPla~ang:./skin com.backup42.desktop.CPDesktop
10672 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % artsmessage -i Sound server informational~l continue, using the null output device.
13049 be/7 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % java -Dfile.encoding=UTF-8 -Dapp=CrashPla~hplan/lang com.backup42.service.CPService
13162 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % java -Dfile.encoding=UTF-8 -Dapp=CrashPla~ang:./skin com.backup42.desktop.CPDeskt

I dont know what to look for here..
Can someone please help?

Thanks
--Abhishek

gerald_clark
Posts: 10642
Joined: 2005/08/05 15:19:54
Location: Northern Illinois, USA

Re: Very slow file operations

Post by gerald_clark » 2015/02/18 18:12:54

If the new drive is an advanced format drive ( 4K sectors ) you must align all your partitions and logical volumes on 4K boundaries or you will have poor disk performance.

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Very slow file operations

Post by TrevorH » 2015/02/18 18:27:27

Please note that CentOS 5.8 is old and out of date and badly in need of yum update. The current CentOS 5 is 5.11. 5.8 was released in March 2012 so you are missing nearly 3 years worth of fixes and enhancements. Several of those fixes are likely to be security related.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

abhishek76
Posts: 4
Joined: 2015/02/18 17:16:05

Re: Very slow file operations

Post by abhishek76 » 2015/02/18 21:44:56

Hi Gerald,
Thanks much for the reply.
Looks like the sector size is 512 on both the drives.
Here is the output of 'parted print all' command. are there any other clues here?

Model: ATA WDC WD10EADS-00L (scsi)
Disk /dev/sda: 1000GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags
1 32.3kB 107MB 107MB primary ext3 boot
2 107MB 1000GB 1000GB primary lvm

Model: ATA ST4000DM000-1F21 (scsi)
Disk /dev/sdb: 4001GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 17.4kB 4001GB 4001GB lvm


Error: Unable to open /dev/md0 - unrecognised disk label.

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Very slow file operations

Post by TrevorH » 2015/02/18 23:31:15

I think the Western Digital drive is the last model before they transitioned to the 4KB sector "Advanced Format" models so it's almost certainly 512b sectors. The Seagate is not and I think parted is lying to you. That is a 4KB sector drive and a starting position of 17.4KB does not sound like it's properly aligned to me.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

abhishek76
Posts: 4
Joined: 2015/02/18 17:16:05

Re: Very slow file operations

Post by abhishek76 » 2015/02/19 14:44:48

Hi Trevor,
So how do I fix this?
Is there any other program which will tell me the correct sector size?

Thanks
--Abhishek

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Very slow file operations

Post by TrevorH » 2015/02/19 20:37:53

If your partition is misaligned then I am unawre of any way to correct this except by removing the partition, creating it in the correct place, formatting it and reinstalling (if it's your o/s boot drive).
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

Post Reply