CentOS Icon CentOS Logo
CentOS Text
   
  
www.centos.org Forum Index
   CentOS 6 - General Support
  [SOLVED] DELL R815 + H700 + H800 + MD1200 I/O Performance problem

 

 Bottom   Previous Topic   Next Topic
  •  Rate Thread
      Rate this Thread
      Excellent
      Good
      Average
      Bad
      Terrible
Poster Thread
  •  deckardxxi
      deckardxxi
[SOLVED] DELL R815 + H700 + H800 + MD1200 I/O Performance problem
#1
Newbie
Joined: 2012/4/11
From
Posts: 4
Hello,

I'm having a I/O performance problem with new server that I purchased:

Specs R815
-------------------------------------------------
2xCPU 16 Core AMD Opteron 6282 SE
64GB RAM
Raid controller H700 1GB cache NV
- 2HD 74GB SAS 15Krpm RAID1 stripe 16k (OS Centos 6.2) sda
- 4HD 146GB SAS 15Krpm RAID10 stripe 16k (ext4 bs 4096, no barriers) sdb -> /vol01
Raid controller H800 1GB cache nv
- MD1200 12HD 300GB SAS 15Krpm RAID10 stripe 256k (For DB Postgres 8.3.18) (ext4 bs 4096, stride 64, stripe-width 384, no barriers) sdc -> /vol02

My problem is I/O speed is very bad in all disk. if I execute "dd if=/dev/zero of=//DD bs=4M count=1024 conv=fdatasync" I obtain te results:
sda - 69,7MB/s
sdb - 80,9 MB/s
sdc - 125 MB/s

Reading results are also extrange:
sda - 702 MB/s
sdb - 587 MB/s
sdc - 583 MB/s

sda(Raid1) has better performance that sdb(RAID10 4Disk) or sdc(RAID10 12Disk)

When exec "hdparm -tT device" I obtain:
/dev/sda:
Timing cached reads: 4986 MB in 2.00 seconds = 2494.23 MB/sec
Timing buffered disk reads: 434 MB in 3.01 seconds = 144.21 MB/sec

/dev/sdb:
Timing cached reads: 4852 MB in 2.00 seconds = 2427.49 MB/sec
Timing buffered disk reads: 1122 MB in 3.00 seconds = 373.58 MB/sec

/dev/sdc:
Timing cached reads: 5192 MB in 2.00 seconds = 2597.10 MB/sec
Timing buffered disk reads: 2828 MB in 3.00 seconds = 942.62 MB/sec

But if I execute hdparm two times, the second read is very, very bad:
/dev/sda:
Timing cached reads: 1186 MB in 2.00 seconds = 592.88 MB/sec
Timing buffered disk reads: 436 MB in 3.01 seconds = 144.96 MB/sec

/dev/sdb:
Timing cached reads: 1186 MB in 2.00 seconds = 593.55 MB/sec
Timing buffered disk reads: 642 MB in 3.01 seconds = 213.55 MB/sec

/dev/sdc:
Timing cached reads: 1194 MB in 2.00 seconds = 597.07 MB/sec
Timing buffered disk reads: 606 MB in 3.04 seconds = 199.46 MB/sec

I go from 2,5GBps to 600MBps!!!

I have installed tuned and apply profile "throughput-performance" with similar results.
Also I change readahead to 65536 without results...

Getinfo Base:

Basic system information.
== BEGIN uname -rmi ==
2.6.32-220.7.1.el6.x86_64 x86_64 x86_64
== END   uname -rmi ==

== BEGIN rpm -qa \*-release\* ==
centos-release-6-2.el6.centos.7.x86_64
== END   rpm -qa \*-release\* ==

== BEGIN cat /etc/redhat-release ==
CentOS release 6.2 (Final)
Santiago
== END   cat /etc/redhat-release ==

== BEGIN getenforce ==
Disabled
== END   getenforce ==

== BEGIN free -m ==
             total       used       free     shared    buffers     cached
Mem:         64395      16182      48213          0        262      14413
-/+ buffers/cache:       1506      62889
Swap:         2047          0       2047
== END   free -m ==



Getinfo Disk

== BEGIN uname -rmi ==
2.6.32-220.7.1.el6.x86_64 x86_64 x86_64
== END   uname -rmi ==

== BEGIN rpm -qa \*-release\* ==
centos-release-6-2.el6.centos.7.x86_64
== END   rpm -qa \*-release\* ==

== BEGIN cat /etc/redhat-release ==
CentOS release 6.2 (Final)
Santiago
== END   cat /etc/redhat-release ==

== BEGIN getenforce ==
Disabled
== END   getenforce ==

== BEGIN free -m ==
             total       used       free     shared    buffers     cached
Mem:         64395      16181      48213          0        262      14413
-/+ buffers/cache:       1505      62889
Swap:         2047          0       2047
== END   free -m ==

== BEGIN cat /etc/fstab ==

#
# /etc/fstab
# Created by anaconda on Mon Mar 19 17:16:48 2012
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=ebed6903-dde6-4d07-a609-a58f4a14223a /                       ext4    defaults        1 1
UUID=808a9405-8973-4b20-bcf0-23b0432e18dc /boot                   ext4    defaults        1 2
UUID=c1b5baeb-5f9c-43f8-bd64-22eb7a243d82 /srv                    ext4    defaults        1 2
UUID=e94ba50f-0cfe-4436-ae27-a3b93eb029f4 /tmp                    ext4    defaults        1 2
UUID=97baf08e-c0f1-4ef0-af13-a32ccb0eb6e9 swap                    swap    defaults        0 0
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0

LABEL=vol01		/vol01			ext4	barrier=0,noatime,nodiratime	0 0
LABEL=vol02             /vol02                  ext4    barrier=0,noatime,nodiratime    0 0
== END   cat /etc/fstab ==

== BEGIN df -h ==
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3              20G  3.2G   16G  18% /
tmpfs                  32G     0   32G   0% /dev/shm
/dev/sda1             194M   56M  129M  31% /boot
/dev/sda6              43G  4.2G   37G  11% /srv
/dev/sda5             2.0G   68M  1.9G   4% /tmp
/dev/sdb1             268G  207G   49G  82% /vol01
/dev/sdc1             1.7T  415G  1.2T  27% /vol02
== END   df -h ==

== BEGIN fdisk -l ==

Disk /dev/sdb: 292.3 GB, 292326211584 bytes
255 heads, 63 sectors/track, 35539 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000d3d81

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1       35540   285472768   83  Linux

Disk /dev/sda: 72.7 GB, 72746008576 bytes
255 heads, 63 sectors/track, 8844 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x4ecb89da

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          26      204800   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2              26         287     2097152   82  Linux swap / Solaris
Partition 2 does not end on cylinder boundary.
/dev/sda3             287        2898    20971520   83  Linux
/dev/sda4            2898        8845    47766528    5  Extended
/dev/sda5            2898        3159     2097152   83  Linux
/dev/sda6            3159        8845    45667328   83  Linux

Disk /dev/sdc: 1796.6 GB, 1796638507008 bytes
255 heads, 63 sectors/track, 218428 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000a494c

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1      218429  1754527744   83  Linux
== END   fdisk -l ==

== BEGIN parted -l ==
Model: DELL PERC H700 (scsi)
Disk /dev/sda: 72.7GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type      File system     Flags
 1      1049kB  211MB   210MB   primary   ext4            boot
 2      211MB   2358MB  2147MB  primary   linux-swap(v1)
 3      2358MB  23.8GB  21.5GB  primary   ext4
 4      23.8GB  72.7GB  48.9GB  extended
 5      23.8GB  26.0GB  2147MB  logical   ext4
 6      26.0GB  72.7GB  46.8GB  logical   ext4


Model: DELL PERC H700 (scsi)
Disk /dev/sdb: 292GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End    Size   Type     File system  Flags
 1      1049kB  292GB  292GB  primary  ext4


Model: DELL PERC H800 (scsi)
Disk /dev/sdc: 1797GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  1797GB  1797GB  primary  ext4


== END   parted -l ==

== BEGIN blkid ==
/dev/sda1: UUID="808a9405-8973-4b20-bcf0-23b0432e18dc" TYPE="ext4" 
/dev/sda2: UUID="97baf08e-c0f1-4ef0-af13-a32ccb0eb6e9" TYPE="swap" 
/dev/sda3: UUID="ebed6903-dde6-4d07-a609-a58f4a14223a" TYPE="ext4" 
/dev/sda5: UUID="e94ba50f-0cfe-4436-ae27-a3b93eb029f4" TYPE="ext4" 
/dev/sda6: UUID="c1b5baeb-5f9c-43f8-bd64-22eb7a243d82" TYPE="ext4" 
/dev/sdb1: LABEL="vol01" UUID="06673665-5750-4bf8-bef4-705febb438e3" TYPE="ext4" 
/dev/sdc1: LABEL="vol02" UUID="daf063f4-db98-4b62-8b36-a9578c268814" TYPE="ext4" 
== END   blkid ==

== BEGIN cat /proc/mdstat ==
Personalities : 
unused devices: <none>
== END   cat /proc/mdstat ==

== BEGIN pvs ==
== END   pvs ==

== BEGIN vgs ==
  No volume groups found
== END   vgs ==

== BEGIN lvs ==
  No volume groups found
== END   lvs ==



Getinfo Driver

== BEGIN uname -rmi ==
2.6.32-220.7.1.el6.x86_64 x86_64 x86_64
== END   uname -rmi ==

== BEGIN rpm -qa \*-release\* ==
centos-release-6-2.el6.centos.7.x86_64
== END   rpm -qa \*-release\* ==

== BEGIN cat /etc/redhat-release ==
CentOS release 6.2 (Final)
Santiago
== END   cat /etc/redhat-release ==

== BEGIN getenforce ==
Disabled
== END   getenforce ==

== BEGIN free -m ==
             total       used       free     shared    buffers     cached
Mem:         64395      16183      48212          0        262      14413
-/+ buffers/cache:       1507      62888
Swap:         2047          0       2047
== END   free -m ==

== BEGIN rpm -qa kernel\* | sort ==
kernel-2.6.32-220.7.1.el6.x86_64
kernel-2.6.32-220.el6.x86_64
kernel-firmware-2.6.32-220.7.1.el6.noarch
kernel-headers-2.6.32-220.7.1.el6.centos.plus.x86_64
== END   rpm -qa kernel\* | sort ==

== BEGIN lspci -nn ==
00:00.0 Host bridge [0600]: ATI Technologies Inc RD890 PCI to PCI bridge (external gfx0 port A) [1002:5a13] (rev 02)
00:02.0 PCI bridge [0604]: ATI Technologies Inc RD890 PCI to PCI bridge (PCI express gpp port B) [1002:5a16]
00:03.0 PCI bridge [0604]: ATI Technologies Inc RD890 PCI to PCI bridge (PCI express gpp port C) [1002:5a17]
00:04.0 PCI bridge [0604]: ATI Technologies Inc RD890 PCI to PCI bridge (PCI express gpp port D) [1002:5a18]
00:09.0 PCI bridge [0604]: ATI Technologies Inc RD890 PCI to PCI bridge (PCI express gpp port H) [1002:5a1c]
00:11.0 SATA controller [0106]: ATI Technologies Inc SB7x0/SB8x0/SB9x0 SATA Controller [IDE mode] [1002:4390]
00:12.0 USB controller [0c03]: ATI Technologies Inc SB7x0/SB8x0/SB9x0 USB OHCI0 Controller [1002:4397]
00:12.1 USB controller [0c03]: ATI Technologies Inc SB7x0 USB OHCI1 Controller [1002:4398]
00:12.2 USB controller [0c03]: ATI Technologies Inc SB7x0/SB8x0/SB9x0 USB EHCI Controller [1002:4396]
00:13.0 USB controller [0c03]: ATI Technologies Inc SB7x0/SB8x0/SB9x0 USB OHCI0 Controller [1002:4397]
00:13.1 USB controller [0c03]: ATI Technologies Inc SB7x0 USB OHCI1 Controller [1002:4398]
00:13.2 USB controller [0c03]: ATI Technologies Inc SB7x0/SB8x0/SB9x0 USB EHCI Controller [1002:4396]
00:14.0 SMBus [0c05]: ATI Technologies Inc SBx00 SMBus Controller [1002:4385] (rev 3d)
00:14.3 ISA bridge [0601]: ATI Technologies Inc SB7x0/SB8x0/SB9x0 LPC host controller [1002:439d]
00:14.4 PCI bridge [0604]: ATI Technologies Inc SBx00 PCI to PCI Bridge [1002:4384]
00:18.0 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 0 [1022:1600]
00:18.1 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 1 [1022:1601]
00:18.2 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 2 [1022:1602]
00:18.3 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 3 [1022:1603]
00:18.4 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 4 [1022:1604]
00:18.5 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 5 [1022:1605]
00:19.0 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 0 [1022:1600]
00:19.1 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 1 [1022:1601]
00:19.2 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 2 [1022:1602]
00:19.3 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 3 [1022:1603]
00:19.4 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 4 [1022:1604]
00:19.5 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 5 [1022:1605]
00:1a.0 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 0 [1022:1600]
00:1a.1 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 1 [1022:1601]
00:1a.2 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 2 [1022:1602]
00:1a.3 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 3 [1022:1603]
00:1a.4 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 4 [1022:1604]
00:1a.5 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 5 [1022:1605]
00:1b.0 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 0 [1022:1600]
00:1b.1 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 1 [1022:1601]
00:1b.2 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 2 [1022:1602]
00:1b.3 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 3 [1022:1603]
00:1b.4 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 4 [1022:1604]
00:1b.5 Host bridge [0600]: Advanced Micro Devices [AMD] Family 15h Processor Function 5 [1022:1605]
01:00.0 Ethernet controller [0200]: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet [14e4:1639] (rev 20)
01:00.1 Ethernet controller [0200]: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet [14e4:1639] (rev 20)
02:00.0 Ethernet controller [0200]: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet [14e4:1639] (rev 20)
02:00.1 Ethernet controller [0200]: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet [14e4:1639] (rev 20)
03:00.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8624 24-lane, 6-Port PCI Express Gen 2 (5.0 GT/s) Switch [ExpressLane] [10b5:8624] (rev bb)
04:00.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8624 24-lane, 6-Port PCI Express Gen 2 (5.0 GT/s) Switch [ExpressLane] [10b5:8624] (rev bb)
04:01.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8624 24-lane, 6-Port PCI Express Gen 2 (5.0 GT/s) Switch [ExpressLane] [10b5:8624] (rev bb)
04:04.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8624 24-lane, 6-Port PCI Express Gen 2 (5.0 GT/s) Switch [ExpressLane] [10b5:8624] (rev bb)
04:05.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8624 24-lane, 6-Port PCI Express Gen 2 (5.0 GT/s) Switch [ExpressLane] [10b5:8624] (rev bb)
05:00.0 RAID bus controller [0104]: LSI Logic / Symbios Logic MegaRAID SAS 2108 [Liberator] [1000:0079] (rev 05)
0a:03.0 VGA compatible controller [0300]: Matrox Graphics, Inc. MGA G200eW WPCM450 [102b:0532] (rev 0a)
20:00.0 Host bridge [0600]: ATI Technologies Inc RD890 Northbridge only dual slot (2x8) PCI-e GFX Hydra part [1002:5a12] (rev 02)
20:02.0 PCI bridge [0604]: ATI Technologies Inc RD890 PCI to PCI bridge (PCI express gpp port B) [1002:5a16]
20:03.0 PCI bridge [0604]: ATI Technologies Inc RD890 PCI to PCI bridge (PCI express gpp port C) [1002:5a17]
20:0b.0 PCI bridge [0604]: ATI Technologies Inc RD890 PCI to PCI bridge (NB-SB link) [1002:5a1f]
22:00.0 RAID bus controller [0104]: LSI Logic / Symbios Logic MegaRAID SAS 2108 [Liberator] [1000:0079] (rev 05)
== END   lspci -nn ==

== BEGIN lsusb ==
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 002: ID 0424:2514 Standard Microsystems Corp. USB 2.0 Hub
Bus 003 Device 002: ID 0624:0248 Avocent Corp. 
== END   lsusb ==

== BEGIN rpm -qa kmod\* kmdl\* ==
== END   rpm -qa kmod\* kmdl\* ==



Also I put exit of "megacli -AdpAllInfo -aALL|less"


Adapter #0

==============================================================================
                    Versions
                ================
Product Name    : PERC H700 Integrated
Serial No       : 21201YN
FW Package Build: 12.10.3-0001

                    Mfg. Data
                ================
Mfg. Date       : 01/05/12
Rework Date     : 01/05/12
Revision No     : A05
Battery FRU     : N/A

                Image Versions in Flash:
                ================
BIOS Version       : 3.18.00_4.09.05.00_0x0416A000
FW Version         : 2.100.03-1526
Preboot CLI Version: 04.04-010:#%00008
Ctrl-R Version     : 2.02-0025.1
NVDATA Version: 2.07.03-0003
Boot Block Version : 2.02.00.00-0000
BOOT Version: 01.250.04.219

                Pending Images in Flash
                ================
None

                PCI Info
                ================
Vendor Id       : 1000
Device Id       : 0079
SubVendorId     : 1028
SubDeviceId     : 1f17

Host Interface  : PCIE

Number of Frontend Port: 0 
Device Interface  : PCIE

Number of Backend Port: 8 
Port  :  Address
0        500000e1195cb9d2 
1        500000e1195cb9e2 
2        500000e1195cc302 
3        500000e1195cb792 
4        500000e1190d86b2 
5        500000e1190ebec2 
6        0000000000000000 
7        0000000000000000 

                HW Configuration
                ================
SAS Address     : 5d4ae520775b1f00
BBU             : Present
Alarm           : Absent
NVRAM           : Present
Serial Debugger : Present
Memory          : Present
Flash           : Present
Memory Size     : 1024MB
TPM             : Absent

                Settings
                ================
Current Time                     : 17:20:48 4/11, 2012
Predictive Fail Poll Interval    : 300sec
Interrupt Throttle Active Count  : 16
Interrupt Throttle Completion    : 50us
Rebuild Rate                     : 30%
PR Rate                          : 30%
Resynch Rate                     : 30%
Check Consistency Rate           : 30%
Reconstruction Rate              : 30%
Cache Flush Interval             : 4s
Max Drives to Spinup at One Time : 4
Delay Among Spinup Groups        : 12s
Physical Drive Coercion Mode     : 128MB
Cluster Mode                     : Disabled
Alarm                            : Disabled
Auto Rebuild                     : Enabled
Battery Warning                  : Enabled
Ecc Bucket Size                  : 15
Ecc Bucket Leak Rate             : 1440 Minutes
Restore HotSpare on Insertion    : Disabled
Expose Enclosure Devices         : Disabled
Maintain PD Fail History         : Disabled
Host Request Reordering          : Enabled
Auto Detect BackPlane Enabled    : SGPIO/i2c SEP
Load Balance Mode                : Auto
Use FDE Only                     : Yes
Security Key Assigned            : No
Security Key Failed              : No
Security Key Not Backedup        : No

Any Offline VD Cache Preserved   : No

                Capabilities
                ================
RAID Level Supported             : RAID0, RAID1, RAID5, RAID6, RAID10, RAID50, RAID60, PRL 11, PRL 11 with spanning
Supported Drives                 : SAS, SATA

Allowed Mixing:

Mix in Enclosure Allowed

                Status
                ================
ECC Bucket Count                 : 0

                Limitations
                ================
Max Arms Per VD         : 32 
Max Spans Per VD        : 8 
Max Arrays              : 128 
Max Number of VDs       : 64 
Max Parallel Commands   : 1008 
Max SGE Count           : 60 
Max Data Transfer Size  : 8192 sectors 
Max Strips PerIO        : 42 
Min Stripe Size         : 8 KB
Max Stripe Size         : 1.0 MB

                Device Present
                ================
Virtual Drives    : 2 
  Degraded        : 0 
  Offline         : 0 
Physical Devices  : 7 
  Disks           : 6 
  Critical Disks  : 0 
  Failed Disks    : 0 

                Supported Adapter Operations
                ================
Rebuild Rate                    : Yes
CC Rate                         : Yes
BGI Rate                        : Yes
Reconstruct Rate                : Yes
Patrol Read Rate                : Yes
Alarm Control                   : Yes
Cluster Support                 : No
BBU                             : Yes
Spanning                        : Yes
Dedicated Hot Spare             : Yes
Revertible Hot Spares           : Yes
Foreign Config Import           : Yes
Self Diagnostic                 : Yes
Allow Mixed Redundancy on Array : No
Global Hot Spares               : Yes
Deny SCSI Passthrough           : No
Deny SMP Passthrough            : No
Deny STP Passthrough            : No
Support Security                : Yes

                Supported VD Operations
                ================
Read Policy          : Yes
Write Policy         : Yes
IO Policy            : Yes
Access Policy        : Yes
Disk Cache Policy    : Yes
Reconstruction       : Yes
Deny Locate          : No
Deny CC              : No
Allow Ctrl Encryption: No
Enable LDBBM         : Yes

                Supported PD Operations
                ================
Force Online                            : Yes
Force Offline                           : Yes
Force Rebuild                           : Yes
Deny Force Failed                       : No
Deny Force Good/Bad                     : No
Deny Missing Replace                    : No
Deny Clear                              : No
Deny Locate                             : No
Disable Copyback                        : No
Enable Copyback on SMART                : No
Enable Copyback to SSD on SMART Error   : No
Enable SSD Patrol Read                  : No
PR Correct Unconfigured Areas           : Yes
Enable Spin Down of UnConfigured Drives : No

                Error Counters
                ================
Memory Correctable Errors   : 0 
Memory Uncorrectable Errors : 0 

                Cluster Information
                ================
Cluster Permitted     : No
Cluster Active        : No

                Default Settings
                ================
Phy Polarity                     : 0 
Phy PolaritySplit                : 0 
Background Rate                  : 30 
Stripe Size                      : 64kB
Flush Time                       : 4 seconds
Write Policy                     : WB
Read Policy                      : Adaptive
Cache When BBU Bad               : Disabled
Cached IO                        : No
SMART Mode                       : Mode 6
Alarm Disable                    : Yes
Coercion Mode                    : 128MB
ZCR Config                       : Unknown
Dirty LED Shows Drive Activity   : No
BIOS Continue on Error           : No
Spin Down Mode                   : None
Allowed Device Type              : SAS/SATA Mix
Allow Mix in Enclosure           : Yes
Allow HDD SAS/SATA Mix in VD     : No
Allow SSD SAS/SATA Mix in VD     : No
Allow HDD/SSD Mix in VD          : No
Allow SATA in Cluster            : No
Max Chained Enclosures           : 1 
Disable Ctrl-R                   : No
Enable Web BIOS                  : No
Direct PD Mapping                : Yes
BIOS Enumerate VDs               : Yes
Restore Hot Spare on Insertion   : No
Expose Enclosure Devices         : No
Maintain PD Fail History         : No
Disable Puncturing               : No
Zero Based Enclosure Enumeration : Yes
PreBoot CLI Enabled              : No
LED Show Drive Activity          : Yes
Cluster Disable                  : Yes
SAS Disable                      : No
Auto Detect BackPlane Enable     : SGPIO/i2c SEP
Use FDE Only                     : Yes
Enable Led Header                : No
Delay during POST                : 2 
EnableCrashDump                  : No
EnableLDBBM                      : Yes
EnableJBOD                       : No
Un-Certified Hard Disk Drives    : Allow
Treat Single span R1E as R10     : Yes
Max LD per array                 : 16
Adapter #1

==============================================================================
                    Versions
                ================
Product Name    : PERC H800 Adapter
Serial No       : 1BD008K
FW Package Build: 12.10.3-0001

                    Mfg. Data
                ================
Mfg. Date       : 11/13/11
Rework Date     : 11/13/11
Revision No     : A05
Battery FRU     : N/A

                Image Versions in Flash:
                ================
BIOS Version       : 3.18.00_4.09.05.00_0x0416A000
FW Version         : 2.100.03-1526
Preboot CLI Version: 04.04-010:#%00008
Ctrl-R Version     : 2.02-0025.1
NVDATA Version: 2.07.03-0003
Boot Block Version : 2.02.00.00-0000
BOOT Version: 01.250.04.219

                Pending Images in Flash
                ================
None

                PCI Info
                ================
Vendor Id       : 1000
Device Id       : 0079
SubVendorId     : 1028
SubDeviceId     : 1f15

Host Interface  : PCIE

Number of Frontend Port: 0 
Device Interface  : PCIE

Number of Backend Port: 8 
Port  :  Address
0        500c04f2a6a959bf 
1        500c04f2a6a9593f 
2        0000000000000000 
3        0000000000000000 
4        0000000000000000 
5        0000000000000000 
6        0000000000000000 
7        0000000000000000 

                HW Configuration
                ================
SAS Address     : 5782bcb077a3ee00
BBU             : Present
Alarm           : Absent
NVRAM           : Present
Serial Debugger : Present
Memory          : Present
Flash           : Present
Memory Size     : 1024MB
TPM             : Absent

                Settings
                ================
Current Time                     : 17:20:49 4/11, 2012
Predictive Fail Poll Interval    : 300sec
Interrupt Throttle Active Count  : 16
Interrupt Throttle Completion    : 50us
Rebuild Rate                     : 30%
PR Rate                          : 30%
Resynch Rate                     : 30%
Check Consistency Rate           : 30%
Reconstruction Rate              : 30%
Cache Flush Interval             : 4s
Max Drives to Spinup at One Time : 4
Delay Among Spinup Groups        : 12s
Physical Drive Coercion Mode     : 128MB
Cluster Mode                     : Disabled
Alarm                            : Disabled
Auto Rebuild                     : Enabled
Battery Warning                  : Enabled
Ecc Bucket Size                  : 15
Ecc Bucket Leak Rate             : 1440 Minutes
Restore HotSpare on Insertion    : Disabled
Expose Enclosure Devices         : Disabled
Maintain PD Fail History         : Disabled
Host Request Reordering          : Enabled
Auto Detect BackPlane Enabled    : SGPIO/i2c SEP
Load Balance Mode                : Auto
Use FDE Only                     : Yes
Security Key Assigned            : No
Security Key Failed              : No
Security Key Not Backedup        : No

Any Offline VD Cache Preserved   : No

                Capabilities
                ================
RAID Level Supported             : RAID0, RAID1, RAID5, RAID6, RAID10, RAID50, RAID60, PRL 11, PRL 11 with spanning
Supported Drives                 : SAS, SATA

Allowed Mixing:

Mix in Enclosure Allowed

                Status
                ================
ECC Bucket Count                 : 0

                Limitations
                ================
Max Arms Per VD         : 32 
Max Spans Per VD        : 8 
Max Arrays              : 128 
Max Number of VDs       : 64 
Max Parallel Commands   : 1008 
Max SGE Count           : 60 
Max Data Transfer Size  : 8192 sectors 
Max Strips PerIO        : 42 
Min Stripe Size         : 8 KB
Max Stripe Size         : 1.0 MB

                Device Present
                ================
Virtual Drives    : 1 
  Degraded        : 0 
  Offline         : 0 
Physical Devices  : 13 
  Disks           : 12 
  Critical Disks  : 0 
  Failed Disks    : 0 

                Supported Adapter Operations
                ================
Rebuild Rate                    : Yes
CC Rate                         : Yes
BGI Rate                        : Yes
Reconstruct Rate                : Yes
Patrol Read Rate                : Yes
Alarm Control                   : Yes
Cluster Support                 : No
BBU                             : Yes
Spanning                        : Yes
Dedicated Hot Spare             : Yes
Revertible Hot Spares           : Yes
Foreign Config Import           : Yes
Self Diagnostic                 : Yes
Allow Mixed Redundancy on Array : No
Global Hot Spares               : Yes
Deny SCSI Passthrough           : No
Deny SMP Passthrough            : No
Deny STP Passthrough            : No
Support Security                : Yes

                Supported VD Operations
                ================
Read Policy          : Yes
Write Policy         : Yes
IO Policy            : Yes
Access Policy        : Yes
Disk Cache Policy    : Yes
Reconstruction       : Yes
Deny Locate          : No
Deny CC              : No
Allow Ctrl Encryption: No
Enable LDBBM         : Yes

                Supported PD Operations
                ================
Force Online                            : Yes
Force Offline                           : Yes
Force Rebuild                           : Yes
Deny Force Failed                       : No
Deny Force Good/Bad                     : No
Deny Missing Replace                    : No
Deny Clear                              : No
Deny Locate                             : No
Disable Copyback                        : No
Enable Copyback on SMART                : No
Enable Copyback to SSD on SMART Error   : No
Enable SSD Patrol Read                  : No
PR Correct Unconfigured Areas           : Yes
Enable Spin Down of UnConfigured Drives : No

                Error Counters
                ================
Memory Correctable Errors   : 0 
Memory Uncorrectable Errors : 0 

                Cluster Information
                ================
Cluster Permitted     : No
Cluster Active        : No

                Default Settings
                ================
Phy Polarity                     : 0 
Phy PolaritySplit                : 0 
Background Rate                  : 30 
Stripe Size                      : 64kB
Flush Time                       : 4 seconds
Write Policy                     : WB
Read Policy                      : Adaptive
Cache When BBU Bad               : Disabled
Cached IO                        : No
SMART Mode                       : Mode 6
Alarm Disable                    : Yes
Coercion Mode                    : 128MB
ZCR Config                       : Unknown
Dirty LED Shows Drive Activity   : No
BIOS Continue on Error           : No
Spin Down Mode                   : None
Allowed Device Type              : SAS/SATA Mix
Allow Mix in Enclosure           : Yes
Allow HDD SAS/SATA Mix in VD     : No
Allow SSD SAS/SATA Mix in VD     : No
Allow HDD/SSD Mix in VD          : No
Allow SATA in Cluster            : No
Max Chained Enclosures           : 4 
Disable Ctrl-R                   : No
Enable Web BIOS                  : No
Direct PD Mapping                : No
BIOS Enumerate VDs               : Yes
Restore Hot Spare on Insertion   : No
Expose Enclosure Devices         : No
Maintain PD Fail History         : No
Disable Puncturing               : No
Zero Based Enclosure Enumeration : Yes
PreBoot CLI Enabled              : No
LED Show Drive Activity          : No
Cluster Disable                  : Yes
SAS Disable                      : No
Auto Detect BackPlane Enable     : SGPIO/i2c SEP
Use FDE Only                     : Yes
Enable Led Header                : No
Delay during POST                : 2 
EnableCrashDump                  : No
EnableLDBBM                      : Yes
EnableJBOD                       : No
Un-Certified Hard Disk Drives    : Allow
Treat Single span R1E as R10     : Yes
Max LD per array                 : 16

Posted on: 2012/4/11 14:24
Create PDF from Post Print
Top
  •  gerald_clark
      gerald_clark
Re: DELL R815 + H700 + H800 + MD1200 I/O Performance problem
#2
Professional Board Member
Joined: 2005/8/5
From Northern Illinois, USA
Posts: 6175
Your partitions are not aligned on 4K boundaries.
Posted on: 2012/4/11 14:29
Create PDF from Post Print
Top
  •  deckardxxi
      deckardxxi
Re: DELL R815 + H700 + H800 + MD1200 I/O Performance problem
#3
Newbie
Joined: 2012/4/11
From
Posts: 4
But if I exec fdisk -ul /dev/sdc:

Disco /dev/sdc: 1796.6 GB, 1796638507008 bytes
255 heads, 63 sectors/track, 218428 cylinders, 3509059584 sectores en total
Units = sectores of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000a494c

Disposit. Inicio    Comienzo      Fin      Bloques  Id  Sistema
/dev/sdc1            2048  3509057535  1754527744   83  Linux


I thought the statrt value (2048) was right. What is the correct value? 4096? Is it depending of the number of disk in the raid?

Thanks!
Posted on: 2012/4/12 8:02
Create PDF from Post Print
Top
  •  pschaff
      pschaff
Re: DELL R815 + H700 + H800 + MD1200 I/O Performance problem
#4
Moderator
Joined: 2006/12/13
From Tidewater, Virginia, North America
Posts: 18773
_________________
Phil

Recommended reading: FAQ & Readme first ; Search hint: google "your topic site:centos.org"; Smart Questions
Posted on: 2012/4/12 11:27
Create PDF from Post Print
Top
  •  deckardxxi
      deckardxxi
Re: DELL R815 + H700 + H800 + MD1200 I/O Performance problem
#5
Newbie
Joined: 2012/4/11
From
Posts: 4
I had read about partition alignment, and I had partitions aligment to 1024K that is divisible by 4K
When I did partitions with parted I used "mkpartfs prymary ext4 1 -1" for align to 1 MB but when later exec getinfo, parted reports partition start in 1049KB... I don't understand

I have repartitioned raid10 with 12 disk with fdisk:


fdisk -lu -b 512 /dev/sdc

Disco /dev/sdc: 1796.6 GB, 1796638507008 bytes
255 heads, 63 sectors/track, 218428 cylinders, 3509059584 sectores en total
Units = sectores of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000e83a8

Disposit. Inicio    Comienzo      Fin      Bloques  Id  Sistema
/dev/sdc1            4096  3509059583  1754527744   83  Linux



Format, and benchmark with DD:

 dd if=/dev/zero of=/vol02/DD bs=4M count=1024 conv=fdatasync
1024+0 records in
1024+0 records out
4294967296 bytes (4,3 GB) copied, 55,6663 s, 77,2 MB/s


Horrible results... But googling I find option oflag=direct
dd if=/dev/zero of=/vol02/DD bs=4M count=1024 conv=fdatasync oflag=direct
1024+0 records in
1024+0 records out
4294967296 bytes (4,3 GB) copied, 6,58982 s, 652 MB/s

dd if=/dev/zero of=/vol02/DD bs=8M count=20000 oflag=direct
20000+0 records in
20000+0 records out
167772160000 bytes (168 GB) copied, 290,751 s, 577 MB/s


I don't understand why speed grow up to 10X...
I have some kernel parameter misconfigured?
Posted on: 2012/4/12 16:15
Create PDF from Post Print
Top
  •  deckardxxi
      deckardxxi
Re: DELL R815 + H700 + H800 + MD1200 I/O Performance problem (SOLVED)
#6
Newbie
Joined: 2012/4/11
From
Posts: 4
Finally the problem was on bios configuration.
The DBPM was set to OS DBPM Dell docu. I change for Maximun Performance and speed access to disk was aparently more faster:
W:512MB
R: 1,2GB
Posted on: 2012/4/16 13:52
Create PDF from Post Print
Top
  •  pschaff
      pschaff
Re: [SOLVED] DELL R815 + H700 + H800 + MD1200 I/O Performance problem
#7
Moderator
Joined: 2006/12/13
From Tidewater, Virginia, North America
Posts: 18773
Thanks for reporting back, and on that basis I presume the thread can be marked [SOLVED] for posterity, so I am doing so on your behalf.
_________________
Phil

Recommended reading: FAQ & Readme first ; Search hint: google "your topic site:centos.org"; Smart Questions
Posted on: 2012/4/16 14:17
Create PDF from Post Print
Top
 Top   Previous Topic   Next Topic

 


 You cannot start a new topic.
 You can view topic.
 You cannot reply to posts.
 You cannot edit your posts.
 You cannot delete your posts.
 You cannot add new polls.
 You cannot vote in polls.
 You cannot attach files to posts.
 You cannot post without approval.




"Linux" is a registered trademark of Linus Torvalds. | All other trademarks are property of their respective owners. | All other content is Copyright @ 2004-2009 by the CentOS Project or "each individual contributor (forums, comments, etc.) unless otherwise assigned".| Theme based on a theme by 7dana.com