sestdiena, 2013. gada 28. decembris

Automatic config backup from HP ProCurve switches

#!/bin/bash
# copy ProCurve config from switches to local files

# SNMP RW comunity
RWCOMMUNITY="private"

# Network segment (last octet) where switches are located.
# in my case - 10.0.0.100 - 10.0.0.150
for i in `seq 100 150` 
  do 
 snmpbulkwalk -t 1 -r 1  -v2c -c public 10.0.0.$i sysName $>/dev/null  && \
        swName=$(snmpbulkwalk -t 1 -r 1  -v2c -c public 10.0.0.$i sysName -OvQ) && \
 echo -en 10.0.0.$i"\t" $swName"\n" && \
 snmpset -v 2c -c "$RWCOMMUNITY" 10.0.0.$i  1.3.6.1.4.1.11.2.14.11.5.1.7.1.5.6.0 i 2 && \
 tftp 10.0.0.$i -c get running-config $swName.cfg && \
        snmpset -v 2c -c "$RWCOMMUNITY" 10.0.0.$i  1.3.6.1.4.1.11.2.14.11.5.1.7.1.5.6.0 i 1 
 done 
Found via http://h30499.www3.hp.com/t5/Switches-Hubs-Modems-Legacy-ITRC/Backup-Procurve-Config-via-SNMP/m-p/3620644#M5881

pirmdiena, 2013. gada 9. septembris

SFTP only pieejas veidošana

Ja nepieciešams izveidot SFTP/SCP only pieeju pilnas SSH sesijas vietā, un tā, lai lietotājs var piekļūt tikai savai mājas direktorijai, var rīkoties sekojoši:

  1. Izveidot direktorijas, kurās tiek klāt lietotāji un montēt ar bind tajās lietotāja mājas direktorijas.. Direktorijās root lietotājam jābūt īpašnieka tiesībām, tāpēc neder uzreiz chroot'oties uz lietotāja mājas direktoriju. 
  2. Modificēt ssh servera uzstādījumus 
  3. Papildu var uzstādīt pam_umask failu tiesībām. 

Direktoriju veidošana un montēšana

Skripts veido direktorijas domēna lietotājiem, kurus iegūst ar "wbinfo -u". Esošās mājas direktorijas ir ar ceļu /home/domain/username.
Divpakāpju direktorijai jābūt tāpēc, ka augstāka līmeņa direktorijai jābūt root īpašumā, bet apakšdirketorija pēc piemontēšanas  būs ar lietotāja tiesībām.

#!/bin/sh
## === 
#  Skripts, kas taisa lietotājiem mājas direktorijas 
#  priekš SFTP sesijām  /home/domain/username
#  Jābūt root owned
## ===
wbinfo -u | grep -v \\$ | while read USR
 do
 if [ ! -d "/srv/$USR" ] ; then 
  mkdir -p /srv/$USR/$USR
 fi 

 # Ja nav piemontēts, montējam ar bind
 /bin/mountpoint -q /srv/$USR/$USR || /bin/mount -o bind /home/domain/$USR /srv/$USR/$USR 

done 

Skriptu var ielikt gan pie rc.local, gan uzstādīt kā cron darbu.

Servera uzstādijumi.

Visām nepieciešamajām domēna lietotāju (primārajām!) grupām pievieno sekojošus ierakstus failā
/etc/ssh/sshd_config
....
Subsystem sftp internal-sftp

Match Group "Domain Users"
    ChrootDirectory /srv/%u
    AllowTCPForwarding no
    X11Forwarding no
    ForceCommand internal-sftp


Match Group "Darbinieki"
    ChrootDirectory /srv/%u
    AllowTCPForwarding no
    X11Forwarding no
    ForceCommand internal-sftp

Pēc tam pārstartējam openssh serveri
Atverot sftp/scp sesiju uz servera, lietotāji redz tikai vienu direktoriju savā lietotājvārdā, kas ir lietotāja mājas direktorija.

Tiesību - umask - uzstādīšana

Specifiskās tiesības/umask SFTP sesijai uzstāda rediģējot failu /etc/pam.d/sshd un pievienojot sekojošu ierakstu: 
...
session    optional pam_umask.so umask=0066
...

ceturtdiena, 2013. gada 8. augusts

MegaRAID SAS 2108 - SW RAID 10 (1+0 - 2 + 2 + 2)

START TESTS IO = cached WC = wt PC = on

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
cached-wt-on                  4000  4096    1  462.85 13.51%     0.025      122.28   0.00000  0.00000  3424
cached-wt-on                  4000  4096    2  609.24 18.90%     0.038      182.24   0.00000  0.00000  3222
cached-wt-on                  4000  4096    4  582.31 77.16%     0.077      440.85   0.00000  0.00000   755
cached-wt-on                  4000  4096    8  601.47 176.7%     0.149      880.28   0.00000  0.00000   340

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
cached-wt-on                  4000  4096    1    0.80 0.539%    14.649      127.61   0.00000  0.00000   148
cached-wt-on                  4000  4096    2    1.56 4.215%    14.876       98.72   0.00000  0.00000    37
cached-wt-on                  4000  4096    4    2.75 11.68%    15.010      114.78   0.00000  0.00000    24
cached-wt-on                  4000  4096    8    4.78 38.97%    17.886      172.75   0.00000  0.00000    12

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
cached-wt-on                  4000  4096    1  448.06 55.54%     0.026      286.06   0.00000  0.00000   807
cached-wt-on                  4000  4096    2  294.28 95.77%     0.078      639.71   0.00000  0.00000   307
cached-wt-on                  4000  4096    4  229.24 166.0%     0.199     1005.29   0.00000  0.00000   138
cached-wt-on                  4000  4096    8  209.42 310.5%     0.436     1184.45   0.00000  0.00000    67

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
cached-wt-on                  4000  4096    1    5.61     0%     0.009        0.06   0.00000  0.00000     0
cached-wt-on                  4000  4096    2    5.37     0%     0.009        0.06   0.00000  0.00000     0
cached-wt-on                  4000  4096    4    5.13 0.831%     0.010        0.07   0.00000  0.00000   617
cached-wt-on                  4000  4096    8    5.20 17.73%     0.017        0.10   0.00000  0.00000    29

START TESTS IO = cached WC = wt PC =

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
cached-wt-off                 4000  4096    1  475.14 16.39%     0.025       53.90   0.00000  0.00000  2898
cached-wt-off                 4000  4096    2  592.56 30.55%     0.039      217.23   0.00000  0.00000  1939
cached-wt-off                 4000  4096    4  574.67 84.23%     0.078      404.92   0.00000  0.00000   682
cached-wt-off                 4000  4096    8  599.66 179.3%     0.150      927.41   0.00000  0.00000   334

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
cached-wt-off                 4000  4096    1    0.80 0.964%    14.614      108.20   0.00000  0.00000    83
cached-wt-off                 4000  4096    2    1.53 4.133%    14.851      120.26   0.00000  0.00000    37
cached-wt-off                 4000  4096    4    2.83 12.06%    14.839       94.00   0.00000  0.00000    23
cached-wt-off                 4000  4096    8    4.78 37.66%    17.597      133.40   0.00000  0.00000    13

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
cached-wt-off                 4000  4096    1  405.33 54.45%     0.028      255.12   0.00000  0.00000   744
cached-wt-off                 4000  4096    2  243.62 83.19%     0.094      964.72   0.00000  0.00000   293
cached-wt-off                 4000  4096    4  182.77 118.7%     0.250     1421.38   0.00000  0.00000   154
cached-wt-off                 4000  4096    8  156.26 212.6%     0.585     1716.30   0.00000  0.00000    73

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
cached-wt-off                 4000  4096    1    5.84     0%     0.010        0.06   0.00000  0.00000     0
cached-wt-off                 4000  4096    2    5.61     0%     0.009        0.06   0.00000  0.00000     0
cached-wt-off                 4000  4096    4    5.68 0.774%     0.010        0.07   0.00000  0.00000   732
cached-wt-off                 4000  4096    8    5.74 24.38%     0.015        0.08   0.00000  0.00000    24

START TESTS IO = cached WC = awb PC = on

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
cached-awb-on                 4000  4096    1  418.05 15.62%     0.028      648.42   0.00000  0.00000  2676
cached-awb-on                 4000  4096    2  467.11 15.63%     0.049      798.03   0.00000  0.00000  2988
cached-awb-on                 4000  4096    4  475.21 47.65%     0.092     1026.38   0.00000  0.00000   997
cached-awb-on                 4000  4096    8  465.94 65.01%     0.184     1639.50   0.00000  0.00000   717

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
cached-awb-on                 4000  4096    1    0.84 0.810%    13.934      115.33   0.00000  0.00000   104
cached-awb-on                 4000  4096    2    1.58 4.261%    14.398      116.67   0.00000  0.00000    37
cached-awb-on                 4000  4096    4    2.93 12.62%    14.281      110.91   0.00000  0.00000    23
cached-awb-on                 4000  4096    8    4.68 31.20%    17.261      253.25   0.00000  0.00000    15

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
cached-awb-on                 4000  4096    1  498.96 61.54%     0.023      545.80   0.00000  0.00000   811
cached-awb-on                 4000  4096    2  497.93 155.0%     0.046      600.66   0.00000  0.00000   321
cached-awb-on                 4000  4096    4  484.76 339.7%     0.094      625.07   0.00000  0.00000   143
cached-awb-on                 4000  4096    8  482.63 971.8%     0.185      512.20   0.00000  0.00000    50

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
cached-awb-on                 4000  4096    1   11.13 0.189%     0.009        0.05   0.00000  0.00000  5859
cached-awb-on                 4000  4096    2   10.71     0%     0.009        0.06   0.00000  0.00000     0
cached-awb-on                 4000  4096    4   10.66 1.092%     0.010        0.08   0.00000  0.00000   977
cached-awb-on                 4000  4096    8   10.21 11.76%     0.017        0.09   0.00000  0.00000    87

START TESTS IO = cached WC = awb PC = off

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
cached-awb-off                4000  4096    1  412.57 14.48%     0.028      784.63   0.00000  0.00000  2849
cached-awb-off                4000  4096    2  518.59 19.50%     0.045      695.77   0.00000  0.00000  2659
cached-awb-off                4000  4096    4  520.66 47.19%     0.083     1043.65   0.00000  0.00000  1103
cached-awb-off                4000  4096    8  519.11 85.76%     0.166     1363.41   0.00000  0.00000   605

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
cached-awb-off                4000  4096    1    0.82 0.674%    14.227      158.82   0.00000  0.00000   122
cached-awb-off                4000  4096    2    1.55 4.241%    14.736      157.93   0.00000  0.00000    37
cached-awb-off                4000  4096    4    2.95 12.04%    14.513      120.55   0.00000  0.00000    25
cached-awb-off                4000  4096    8    4.94 31.88%    17.059      137.71   0.00000  0.00000    16

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
cached-awb-off                4000  4096    1  488.89 61.01%     0.023      456.56   0.00000  0.00000   801
cached-awb-off                4000  4096    2  491.26 149.0%     0.046      512.93   0.00000  0.00000   330
cached-awb-off                4000  4096    4  485.30 328.9%     0.094      529.83   0.00000  0.00000   148
cached-awb-off                4000  4096    8  455.13 925.3%     0.197      562.44   0.00000  0.00000    49

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
cached-awb-off                4000  4096    1    9.74 0.083%     0.009        0.05   0.00000  0.00000 11719
cached-awb-off                4000  4096    2    8.70     0%     0.009        0.07   0.00000  0.00000     0
cached-awb-off                4000  4096    4    9.30 3.175%     0.010        0.09   0.00000  0.00000   293
cached-awb-off                4000  4096    8    8.25 9.569%     0.016        0.10   0.00000  0.00000    86

START TESTS IO = direct WC = wt PC = on

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
direct-wt-on                  4000  4096    1  470.31 13.90%     0.025      122.05   0.00000  0.00000  3382
direct-wt-on                  4000  4096    2  608.38 20.54%     0.038      230.56   0.00000  0.00000  2961
direct-wt-on                  4000  4096    4  593.23 79.81%     0.073      322.20   0.00000  0.00000   743
direct-wt-on                  4000  4096    8  602.60 171.7%     0.146      984.91   0.00000  0.00000   351

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
direct-wt-on                  4000  4096    1    0.79 0.695%    14.800      127.98   0.00000  0.00000   114
direct-wt-on                  4000  4096    2    1.41 3.943%    16.363      126.50   0.00000  0.00000    36
direct-wt-on                  4000  4096    4    2.72 11.50%    15.148      215.17   0.00000  0.00000    24
direct-wt-on                  4000  4096    8    4.62 38.64%    18.004      153.71   0.00000  0.00000    12

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
direct-wt-on                  4000  4096    1  458.64 57.13%     0.025      230.74   0.00000  0.00000   803
direct-wt-on                  4000  4096    2  290.16 94.70%     0.079      824.62   0.00000  0.00000   306
direct-wt-on                  4000  4096    4  229.99 148.4%     0.199      923.75   0.00000  0.00000   155
direct-wt-on                  4000  4096    8  208.63 313.7%     0.433     1068.90   0.00000  0.00000    66

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
direct-wt-on                  4000  4096    1    5.28 0.090%     0.009        0.06   0.00000  0.00000  5859
direct-wt-on                  4000  4096    2    6.08 0.103%     0.009        0.05   0.00000  0.00000  5859
direct-wt-on                  4000  4096    4    6.40 3.657%     0.010        0.08   0.00000  0.00000   175
direct-wt-on                  4000  4096    8    6.17 8.685%     0.017        0.15   0.00000  0.00000    71

START TESTS IO = direct WC = wt PC = off

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
direct-wt-off                 4000  4096    1  467.03 17.57%     0.025      132.30   0.00000  0.00000  2657
direct-wt-off                 4000  4096    2  611.99 34.98%     0.038      211.65   0.00000  0.00000  1749
direct-wt-off                 4000  4096    4  589.11 85.15%     0.075      356.54   0.00000  0.00000   692
direct-wt-off                 4000  4096    8  597.58 180.5%     0.147      892.34   0.00000  0.00000   331

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
direct-wt-off                 4000  4096    1    0.79 0.862%    14.831      100.28   0.00000  0.00000    92
direct-wt-off                 4000  4096    2    1.51 4.213%    15.191      115.55   0.00000  0.00000    36
direct-wt-off                 4000  4096    4    2.72 11.66%    15.295      106.87   0.00000  0.00000    23
direct-wt-off                 4000  4096    8    4.55 37.06%    18.315      142.48   0.00000  0.00000    12

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
direct-wt-off                 4000  4096    1  397.09 52.86%     0.029      239.47   0.00000  0.00000   751
direct-wt-off                 4000  4096    2  240.42 81.34%     0.095     1297.71   0.00000  0.00000   296
direct-wt-off                 4000  4096    4  181.09 126.6%     0.249     1506.76   0.00000  0.00000   143
direct-wt-off                 4000  4096    8  157.02 205.9%     0.584     1870.55   0.00000  0.00000    76

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
direct-wt-off                 4000  4096    1    5.95 0.050%     0.010        0.06   0.00000  0.00000 11719
direct-wt-off                 4000  4096    2    6.00     0%     0.009        0.05   0.00000  0.00000     0
direct-wt-off                 4000  4096    4    5.79 1.582%     0.010        0.06   0.00000  0.00000   366
direct-wt-off                 4000  4096    8    5.94 32.05%     0.017        0.07   0.00000  0.00000    19

START TESTS IO = direct WC = awb PC = on

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
direct-awb-on                 4000  4096    1  409.87 16.53%     0.028      628.68   0.00000  0.00000  2479
direct-awb-on                 4000  4096    2  479.43 17.85%     0.048      546.33   0.00000  0.00000  2686
direct-awb-on                 4000  4096    4  469.68 48.44%     0.094     1023.92   0.00000  0.00000   970
direct-awb-on                 4000  4096    8  470.86 52.61%     0.185     1559.41   0.00000  0.00000   895

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
direct-awb-on                 4000  4096    1    0.81 0.773%    14.482      143.94   0.00000  0.00000   105
direct-awb-on                 4000  4096    2    1.60 4.115%    14.414       96.66   0.00000  0.00000    39
direct-awb-on                 4000  4096    4    2.95 13.23%    14.302       90.20   0.00000  0.00000    22
direct-awb-on                 4000  4096    8    4.91 26.20%    17.121      146.82   0.00000  0.00000    19

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
direct-awb-on                 4000  4096    1  496.21 61.84%     0.023       51.54   0.00000  0.00000   802
direct-awb-on                 4000  4096    2  500.74 151.4%     0.046      346.99   0.00000  0.00000   331
direct-awb-on                 4000  4096    4  493.30 326.0%     0.092      367.83   0.00000  0.00000   151
direct-awb-on                 4000  4096    8  482.48 972.3%     0.185      394.22   0.00000  0.00000    50

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
direct-awb-on                 4000  4096    1   12.62 0.107%     0.009        0.07   0.00000  0.00000 11719
direct-awb-on                 4000  4096    2   12.54 0.213%     0.009        0.06   0.00000  0.00000  5859
direct-awb-on                 4000  4096    4   11.28 3.466%     0.010        0.07   0.00000  0.00000   325
direct-awb-on                 4000  4096    8    9.51 4.543%     0.017        0.09   0.00000  0.00000   209

START TESTS IO = direct WC = awb PC = off

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
direct-awb-off                4000  4096    1  419.91 14.83%     0.028      836.43   0.00000  0.00000  2830
direct-awb-off                4000  4096    2  508.28 20.26%     0.045      741.52   0.00000  0.00000  2508
direct-awb-off                4000  4096    4  515.65 57.17%     0.087     1018.63   0.00000  0.00000   902
direct-awb-off                4000  4096    8  519.17 90.11%     0.168     1168.69   0.00000  0.00000   576

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
direct-awb-off                4000  4096    1    0.83 0.620%    14.187      122.07   0.00000  0.00000   133
direct-awb-off                4000  4096    2    1.57 3.062%    14.386      145.25   0.00000  0.00000    51
direct-awb-off                4000  4096    4    2.93 12.51%    14.586      132.94   0.00000  0.00000    23
direct-awb-off                4000  4096    8    4.81 31.41%    17.062      138.90   0.00000  0.00000    15

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
direct-awb-off                4000  4096    1  489.41 64.14%     0.023       98.94   0.00000  0.00000   763
direct-awb-off                4000  4096    2  475.58 145.4%     0.048      173.13   0.00000  0.00000   327
direct-awb-off                4000  4096    4  463.76 301.8%     0.098      239.71   0.00000  0.00000   154
direct-awb-off                4000  4096    8  440.92 875.4%     0.205      537.00   0.00000  0.00000    50

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
direct-awb-off                4000  4096    1    9.07     0%     0.010        0.07   0.00000  0.00000     0
direct-awb-off                4000  4096    2    9.85 0.168%     0.009        0.06   0.00000  0.00000  5859
direct-awb-off                4000  4096    4    9.51 3.894%     0.010        0.09   0.00000  0.00000   244
direct-awb-off                4000  4096    8    8.65 6.868%     0.018        0.11   0.00000  0.00000   126

Skripts

#!/bin/bash
storcli=/opt/MegaRAID/storcli/storcli64

$storcli /c0/vall set iopolicy=cached
$storcli /c0/vall set wrcache=awb
$storcli /c0/vall set pdcache=on

LOGFILE=/root/tio.md.txt

for iopolicy  in cached  direct
do
 $storcli /c0/vall set iopolicy=$iopolicy
        IO=$iopolicy 
        echo = IO=$IO

 for wrcache in wt awb
 do
  $storcli /c0/vall set wrcache=$wrcache
  WC=$wrcache
  echo == WC=$WC
  for pdcache in on off
  do
   $storcli /c0/vall set pdcache=$pdcache
   PC=$pdcache
   echo === PC=$PC
   echo 
   IDENT=$IO-$WC-$PC
   $storcli /c0/vall show | grep RAID | tee -a $LOGFILE
   echo  ==== START TESTS IO = $IO    WC = $WC    PC = $PC    =====  `date +%T` ==== | tee -a $LOGFILE
   sleep 3 
   tiobench --size 4000 --ident $IDENT --dir=/mnt/test/ --numruns 3 | tee -a $LOGFILE
  done
 done
done

MegaRAID SAS 2108 - HW RAID 10 (2 + 2 + 2)


Base Board Information
        Manufacturer: Supermicro
        Product Name: X9DRi-LN4+/X9DR3-LN4+
q
RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS 2108 [Liberator] (rev 05)
SAS 9260CV-8i
Sistēma startēta ar parametru "mem = 1G"
FS - EXT4

HW RAID 10 

2 + 2 + 2

RWTD

storcli /c0/v0 show 

----------------------------------------------------------
DG/VD TYPE   State Access Consist Cache sCC     Size Name 
----------------------------------------------------------
0/0   RAID10 Optl  RW     No      RWTD  -   5.454 TB      
----------------------------------------------------------
root@ubuntu:~# tiobench --size 2000 --identifier R10-RWTD --numruns 3 
Run #3: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT
Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD                      2000  4096    1  164.01 10.61%     0.071      110.05   0.00000  0.00000  1545
R10-RWTD                      2000  4096    2  136.11 16.77%     0.168      149.97   0.00000  0.00000   811
R10-RWTD                      2000  4096    4  127.26 36.09%     0.346      310.12   0.00000  0.00000   353
R10-RWTD                      2000  4096    8  143.80 82.40%     0.600      512.41   0.00000  0.00000   174

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD                      2000  4096    1    1.05 0.510%    11.167      103.91   0.00000  0.00000   206
R10-RWTD                      2000  4096    2    1.86 3.086%    10.919      100.91   0.00000  0.00000    60
R10-RWTD                      2000  4096    4    3.40 6.955%    11.365      119.16   0.00000  0.00000    49
R10-RWTD                      2000  4096    8    5.30 20.34%    13.479      241.22   0.00000  0.00000    26

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD                      2000  4096    1  244.78 33.16%     0.046      963.48   0.00000  0.00000   738
R10-RWTD                      2000  4096    2  199.02 62.89%     0.112      758.66   0.00000  0.00000   316
R10-RWTD                      2000  4096    4  186.14 117.4%     0.241      560.05   0.00000  0.00000   159
R10-RWTD                      2000  4096    8  177.05 216.8%     0.508      711.64   0.00000  0.00000    82

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD                      2000  4096    1    8.06     0%     0.009        0.06   0.00000  0.00000     0
R10-RWTD                      2000  4096    2    7.78     0%     0.009        0.04   0.00000  0.00000     0
R10-RWTD                      2000  4096    4    7.82     0%     0.010        0.08   0.00000  0.00000     0
R10-RWTD                      2000  4096    8    7.90 9.166%     0.017        0.09   0.00000  0.00000    86

RWTC

storcli /c0/v0 set iopolicy=cached
storcli /c0/v0 show

----------------------------------------------------------
DG/VD TYPE   State Access Consist Cache sCC     Size Name 
----------------------------------------------------------
0/0   RAID10 Optl  RW     No      RWTC  -   5.454 TB      
----------------------------------------------------------
root@ubuntu:~# tiobench --size 2000 --identifier R10-RWTC --numruns 3 
Run #3: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1  169.42 10.76%     0.069      108.65   0.00000  0.00000  1574
R10-RWTC                      2000  4096    2  139.71 17.76%     0.165      162.62   0.00000  0.00000   787
R10-RWTC                      2000  4096    4  133.68 37.36%     0.325      237.25   0.00000  0.00000   358
R10-RWTC                      2000  4096    8  132.39 75.94%     0.645      637.03   0.00000  0.00000   174

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1    1.04 0.694%    11.234       84.40   0.00000  0.00000   150
R10-RWTC                      2000  4096    2    1.98 3.624%    11.001       82.62   0.00000  0.00000    55
R10-RWTC                      2000  4096    4    3.36 5.441%    11.392      165.62   0.00000  0.00000    62
R10-RWTC                      2000  4096    8    5.25 19.31%    13.533      180.47   0.00000  0.00000    27

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1  247.02 30.99%     0.045      322.56   0.00000  0.00000   797
R10-RWTC                      2000  4096    2  203.29 67.52%     0.110      563.23   0.00000  0.00000   301
R10-RWTC                      2000  4096    4  185.36 116.4%     0.241      526.25   0.00000  0.00000   159
R10-RWTC                      2000  4096    8  177.23 225.7%     0.506      616.67   0.00000  0.00000    78

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1    7.41     0%     0.009        0.06   0.00000  0.00000     0
R10-RWTC                      2000  4096    2    7.98     0%     0.009        0.05   0.00000  0.00000     0
R10-RWTC                      2000  4096    4    7.76 1.059%     0.010        0.08   0.00000  0.00000   732
R10-RWTC                      2000  4096    8    7.92 8.446%     0.017        0.07   0.00000  0.00000    94

RWTC - Disk cache enabled

tiobench --size 2000 --identifier R10-RWTC 
Run #1: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1  222.69 15.58%     0.017       24.78   0.00000  0.00000  1428
R10-RWTC                      2000  4096    2  126.77 15.46%     0.061       58.48   0.00000  0.00000   820
R10-RWTC                      2000  4096    4  162.10 44.19%     0.087       66.98   0.00000  0.00000   367
R10-RWTC                      2000  4096    8  126.15 72.59%     0.230      208.33   0.00000  0.00000   174

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1    1.09 0.694%     3.596       27.37   0.00000  0.00000   156
R10-RWTC                      2000  4096    2    2.01 1.441%     3.693       25.86   0.00000  0.00000   139
R10-RWTC                      2000  4096    4    2.94 3.841%     3.876       28.29   0.00000  0.00000    77
R10-RWTC                      2000  4096    8    5.25 17.22%     4.559       70.20   0.00000  0.00000    31

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1  281.20 36.22%     0.013      174.99   0.00000  0.00000   776
R10-RWTC                      2000  4096    2  199.49 64.79%     0.037      171.67   0.00000  0.00000   308
R10-RWTC                      2000  4096    4  188.08 119.7%     0.079      175.89   0.00000  0.00000   157
R10-RWTC                      2000  4096    8  176.24 208.0%     0.169      256.70   0.00000  0.00000    85

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1    8.10     0%     0.003        0.03   0.00000  0.00000     0
R10-RWTC                      2000  4096    2    7.89     0%     0.003        0.02   0.00000  0.00000     0
R10-RWTC                      2000  4096    4    7.79     0%     0.003        0.03   0.00000  0.00000     0
R10-RWTC                      2000  4096    8    7.93 6.496%     0.006        0.03   0.00000  0.00000   122

RWTC - Disk cache disabled

# tiobench --size 2000 --identifier R10-RWTC-PdCacOff 
Run #1: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC-PdCacOff             2000  4096    1  221.14 14.06%     0.018       23.75   0.00000  0.00000  1572
R10-RWTC-PdCacOff             2000  4096    2  122.06 15.99%     0.063       50.35   0.00000  0.00000   763
R10-RWTC-PdCacOff             2000  4096    4  131.95 35.99%     0.108       83.59   0.00000  0.00000   367
R10-RWTC-PdCacOff             2000  4096    8  130.39 73.15%     0.219      183.32   0.00000  0.00000   178

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC-PdCacOff             2000  4096    1    1.11 0.509%     3.532       39.69   0.00000  0.00000   217
R10-RWTC-PdCacOff             2000  4096    2    2.11 3.894%     3.485       26.37   0.00000  0.00000    54
R10-RWTC-PdCacOff             2000  4096    4    2.99 3.209%     3.787       32.58   0.00000  0.00000    93
R10-RWTC-PdCacOff             2000  4096    8    4.89 17.40%     4.410       82.89   0.00000  0.00000    28

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC-PdCacOff             2000  4096    1  218.60 29.25%     0.017      237.86   0.00000  0.00000   747
R10-RWTC-PdCacOff             2000  4096    2  171.45 56.99%     0.043      209.19   0.00000  0.00000   301
R10-RWTC-PdCacOff             2000  4096    4  157.12 102.8%     0.095      250.64   0.00000  0.00000   153
R10-RWTC-PdCacOff             2000  4096    8  138.59 166.9%     0.209      340.25   0.00000  0.00000    83

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC-PdCacOff             2000  4096    1    6.02     0%     0.003        0.02   0.00000  0.00000     0
R10-RWTC-PdCacOff             2000  4096    2    5.87     0%     0.003        0.02   0.00000  0.00000     0
R10-RWTC-PdCacOff             2000  4096    4    5.98     0%     0.003        0.03   0.00000  0.00000     0
R10-RWTC-PdCacOff             2000  4096    8    5.99 6.137%     0.006        0.05   0.00000  0.00000    98

RWTD - Disk cache disabled

root@ubuntu:~# tiobench --size 2000 --identifier R10-RWTD-PdCacOff 
Run #1: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOff             2000  4096    1  225.59 13.71%     0.017       23.27   0.00000  0.00000  1645
R10-RWTD-PdCacOff             2000  4096    2  128.78 18.03%     0.060       74.44   0.00000  0.00000   714
R10-RWTD-PdCacOff             2000  4096    4  120.24 34.48%     0.125      108.35   0.00000  0.00000   349
R10-RWTD-PdCacOff             2000  4096    8  150.00 78.87%     0.178      183.56   0.00000  0.00000   190

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOff             2000  4096    1    1.07 0.518%     3.662       28.25   0.00000  0.00000   206
R10-RWTD-PdCacOff             2000  4096    2    2.12 3.794%     3.478       32.29   0.00000  0.00000    56
R10-RWTD-PdCacOff             2000  4096    4    3.92 5.926%     3.850       33.38   0.00000  0.00000    66
R10-RWTD-PdCacOff             2000  4096    8    5.40 14.65%     4.195       54.68   0.00000  0.00000    37

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOff             2000  4096    1  211.89 27.29%     0.018      231.49   0.00000  0.00000   776
R10-RWTD-PdCacOff             2000  4096    2  176.41 58.00%     0.042      224.31   0.00000  0.00000   304
R10-RWTD-PdCacOff             2000  4096    4  156.11 97.60%     0.094      261.58   0.00000  0.00000   160
R10-RWTD-PdCacOff             2000  4096    8  137.09 164.4%     0.209      325.76   0.00000  0.00000    83

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOff             2000  4096    1    6.12     0%     0.003        0.02   0.00000  0.00000     0
R10-RWTD-PdCacOff             2000  4096    2    6.24     0%     0.003        0.02   0.00000  0.00000     0
R10-RWTD-PdCacOff             2000  4096    4    6.00     0%     0.003        0.02   0.00000  0.00000     0
R10-RWTD-PdCacOff             2000  4096    8    6.05 4.957%     0.006        0.04   0.00000  0.00000   122

RWTD - Disk cache enabled

tiobench --size 2000 --identifier R10-RWTD-PdCacOn 
Run #1: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOn              2000  4096    1  222.84 14.61%     0.017       20.89   0.00000  0.00000  1524
R10-RWTD-PdCacOn              2000  4096    2  129.22 15.81%     0.060       48.82   0.00000  0.00000   817
R10-RWTD-PdCacOn              2000  4096    4  131.79 38.19%     0.110       83.58   0.00000  0.00000   345
R10-RWTD-PdCacOn              2000  4096    8  141.00 75.24%     0.188      151.67   0.00000  0.00000   187

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOn              2000  4096    1    1.06 0.703%     3.697       27.83   0.00000  0.00000   150
R10-RWTD-PdCacOn              2000  4096    2    1.99 0.970%     3.814       25.99   0.00000  0.00000   206
R10-RWTD-PdCacOn              2000  4096    4    2.88 6.938%     4.034       51.90   0.00000  0.00000    42
R10-RWTD-PdCacOn              2000  4096    8    5.36 14.82%     3.940       54.28   0.00000  0.00000    36

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOn              2000  4096    1  280.53 34.56%     0.013      205.93   0.00000  0.00000   812
R10-RWTD-PdCacOn              2000  4096    2  199.38 62.92%     0.037      165.82   0.00000  0.00000   317
R10-RWTD-PdCacOn              2000  4096    4  191.24 96.84%     0.079      157.06   0.00000  0.00000   197
R10-RWTD-PdCacOn              2000  4096    8  175.74 206.3%     0.170      391.90   0.00000  0.00000    85

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOn              2000  4096    1    7.98 0.204%     0.003        0.01   0.00000  0.00000  3906
R10-RWTD-PdCacOn              2000  4096    2    7.78     0%     0.003        0.01   0.00000  0.00000     0
R10-RWTD-PdCacOn              2000  4096    4    8.24 1.687%     0.003        0.03   0.00000  0.00000   488
R10-RWTD-PdCacOn              2000  4096    8    8.10 11.61%     0.006        0.03   0.00000  0.00000    70

Always Write Back

   storcli /c0/v0 set wrcache=AWB 

RAWBD PdCacOn

storcli /c0/v0 show
----------------------------------------------------------
DG/VD TYPE   State Access Consist Cache sCC     Size Name 
----------------------------------------------------------
0/0   RAID10 Optl  RW     No      RAWBD -   5.454 TB      
----------------------------------------------------------
tiobench --size 2000 --identifier R10-RAWBD-PdCacOn  --numruns 3
Run #3: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Unit information
================
File size = megabytes
Blk Size  = bytes
Rate      = megabytes per second
CPU%      = percentage of CPU used during the test
Latency   = milliseconds
Lat%      = percent of requests that took longer than X seconds
CPU Eff   = Rate divided by CPU% - throughput per cpu load

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBD-PdCacOn             2000  4096    1  146.53 9.671%     0.080     1344.87   0.00000  0.00000  1515
R10-RAWBD-PdCacOn             2000  4096    2  143.45 16.55%     0.158     1364.29   0.00000  0.00000   866
R10-RAWBD-PdCacOn             2000  4096    4  119.89 32.65%     0.362     1940.91   0.00000  0.00000   367
R10-RAWBD-PdCacOn             2000  4096    8  127.14 70.38%     0.668     1781.70   0.00000  0.00000   181

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBD-PdCacOn             2000  4096    1    1.17 0.616%    10.053      117.77   0.00000  0.00000   189
R10-RAWBD-PdCacOn             2000  4096    2    1.94 3.273%    10.045       85.48   0.00000  0.00000    59
R10-RAWBD-PdCacOn             2000  4096    4    3.43 5.976%    10.275      119.88   0.00000  0.00000    57
R10-RAWBD-PdCacOn             2000  4096    8    6.03 18.00%    11.818      185.64   0.00000  0.00000    33

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBD-PdCacOn             2000  4096    1  293.70 42.82%     0.039      252.09   0.00000  0.00000   686
R10-RAWBD-PdCacOn             2000  4096    2  263.84 83.16%     0.086      353.42   0.00000  0.00000   317
R10-RAWBD-PdCacOn             2000  4096    4  275.19 165.3%     0.163      474.69   0.00000  0.00000   166
R10-RAWBD-PdCacOn             2000  4096    8  257.68 351.0%     0.343     1178.33   0.00000  0.00000    73

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBD-PdCacOn             2000  4096    1   14.03     0%     0.009        0.08   0.00000  0.00000     0
R10-RAWBD-PdCacOn             2000  4096    2   15.38 1.312%     0.008        0.04   0.00000  0.00000  1172
R10-RAWBD-PdCacOn             2000  4096    4   14.37     0%     0.010        0.07   0.00000  0.00000     0
R10-RAWBD-PdCacOn             2000  4096    8   18.77 9.612%     0.017        0.10   0.00000  0.00000   195

RAWBC - Disk Cache On

storcli /c0/v0 set iopolicy=cached
storcli /c0/v0 show

----------------------------------------------------------
DG/VD TYPE   State Access Consist Cache sCC     Size Name 
----------------------------------------------------------
0/0   RAID10 Optl  RW     No      RAWBC -   5.454 TB      
----------------------------------------------------------
tiobench --size 2000 --identifier R10-RAWBC-PdCacOn  --numruns 3
Run #3: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOn             2000  4096    1  146.89 9.313%     0.080      986.74   0.00000  0.00000  1577
R10-RAWBC-PdCacOn             2000  4096    2  124.14 14.54%     0.185     1406.61   0.00000  0.00000   853
R10-RAWBC-PdCacOn             2000  4096    4  125.12 33.76%     0.347     1811.97   0.00000  0.00000   371
R10-RAWBC-PdCacOn             2000  4096    8  126.05 70.05%     0.679     1398.69   0.00000  0.00000   180

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOn             2000  4096    1    1.19 0.617%     9.882       96.18   0.00000  0.00000   192
R10-RAWBC-PdCacOn             2000  4096    2    2.02 2.255%    10.182       95.31   0.00000  0.00000    89
R10-RAWBC-PdCacOn             2000  4096    4    3.33 4.808%    10.761      140.33   0.00000  0.00000    69
R10-RAWBC-PdCacOn             2000  4096    8    5.98 17.81%    12.368      173.46   0.00000  0.00000    34

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOn             2000  4096    1  295.63 39.42%     0.038      280.16   0.00000  0.00000   750
R10-RAWBC-PdCacOn             2000  4096    2  258.20 79.84%     0.088      190.60   0.00000  0.00000   323
R10-RAWBC-PdCacOn             2000  4096    4  282.15 140.9%     0.160      848.78   0.00000  0.00000   200
R10-RAWBC-PdCacOn             2000  4096    8  255.79 340.0%     0.342      400.85   0.00000  0.00000    75

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOn             2000  4096    1   16.55     0%     0.009        0.06   0.00000  0.00000     0
R10-RAWBC-PdCacOn             2000  4096    2   18.37     0%     0.008        0.06   0.00000  0.00000     0
R10-RAWBC-PdCacOn             2000  4096    4   13.72 1.873%     0.010        0.07   0.00000  0.00000   732
R10-RAWBC-PdCacOn             2000  4096    8   13.51 12.91%     0.016        0.09   0.00000  0.00000   105

RAWBC - Disk Cache Off

 storcli /c0/v0 set pdcache=Off  
tiobench --size 2000 --identifier R10-RAWBC-PdCacOff  --numruns 1
Run #1: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOff            2000  4096    1  178.90 10.80%     0.022      198.66   0.00000  0.00000  1656
R10-RAWBC-PdCacOff            2000  4096    2  111.23 12.90%     0.069      331.54   0.00000  0.00000   862
R10-RAWBC-PdCacOff            2000  4096    4  173.12 48.85%     0.087      402.51   0.00000  0.00000   354
R10-RAWBC-PdCacOff            2000  4096    8  135.25 71.82%     0.202      400.28   0.00000  0.00000   188

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOff            2000  4096    1    1.14 0.552%     3.440       29.15   0.00000  0.00000   206
R10-RAWBC-PdCacOff            2000  4096    2    2.11 3.455%     3.378       51.59   0.00000  0.00000    61
R10-RAWBC-PdCacOff            2000  4096    4    3.84 3.635%     3.434       31.46   0.00000  0.00000   106
R10-RAWBC-PdCacOff            2000  4096    8    5.95 20.72%     3.958       53.06   0.00000  0.00000    29

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOff            2000  4096    1  277.00 34.90%     0.013      320.74   0.00000  0.00000   794
R10-RAWBC-PdCacOff            2000  4096    2  211.19 67.75%     0.036      133.28   0.00000  0.00000   312
R10-RAWBC-PdCacOff            2000  4096    4  279.79 164.3%     0.053      116.90   0.00000  0.00000   170
R10-RAWBC-PdCacOff            2000  4096    8  212.86 254.1%     0.140      279.91   0.00000  0.00000    84

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOff            2000  4096    1   15.27     0%     0.003        0.02   0.00000  0.00000     0
R10-RAWBC-PdCacOff            2000  4096    2   14.53 0.744%     0.003        0.03   0.00000  0.00000  1953
R10-RAWBC-PdCacOff            2000  4096    4   18.08     0%     0.003        0.03   0.00000  0.00000     0
R10-RAWBC-PdCacOff            2000  4096    8   15.72 9.660%     0.006        0.03   0.00000  0.00000   163

NRAWBC - Disk Cache Off

storcli /c0/v0 set rdcache=nora
storcli /c0/v0 show
-----------------------------------------------------------
DG/VD TYPE   State Access Consist Cache  sCC     Size Name 
-----------------------------------------------------------
0/0   RAID10 Optl  RW     No      NRAWBC -   5.454 TB      
-----------------------------------------------------------
tiobench --size 2000 --identifier R10-NRAWBC-PdCacOff  --numruns 1
Run #1: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-NRAWBC-PdCacOff           2000  4096    1  190.21 11.41%     0.020      433.66   0.00000  0.00000  1667
R10-NRAWBC-PdCacOff           2000  4096    2  109.12 14.42%     0.071      275.32   0.00000  0.00000   756
R10-NRAWBC-PdCacOff           2000  4096    4  123.45 33.13%     0.110      373.31   0.00000  0.00000   373
R10-NRAWBC-PdCacOff           2000  4096    8  128.07 70.72%     0.219      350.40   0.00000  0.00000   181

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-NRAWBC-PdCacOff           2000  4096    1    1.22 0.591%     3.210       61.00   0.00000  0.00000   206
R10-NRAWBC-PdCacOff           2000  4096    2    2.17 3.770%     3.446       48.13   0.00000  0.00000    57
R10-NRAWBC-PdCacOff           2000  4096    4    3.28 4.874%     3.360       34.23   0.00000  0.00000    67
R10-NRAWBC-PdCacOff           2000  4096    8    6.04 17.45%     3.928       48.08   0.00000  0.00000    35

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-NRAWBC-PdCacOff           2000  4096    1  288.05 39.86%     0.013      267.42   0.00000  0.00000   722
R10-NRAWBC-PdCacOff           2000  4096    2  216.66 73.93%     0.035       59.29   0.00000  0.00000   293
R10-NRAWBC-PdCacOff           2000  4096    4  262.35 133.5%     0.057      122.33   0.00000  0.00000   196
R10-NRAWBC-PdCacOff           2000  4096    8  213.97 265.4%     0.138      175.91   0.00000  0.00000    81

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-NRAWBC-PdCacOff           2000  4096    1   14.42     0%     0.003        0.02   0.00000  0.00000     0
R10-NRAWBC-PdCacOff           2000  4096    2   17.76     0%     0.003        0.02   0.00000  0.00000     0
R10-NRAWBC-PdCacOff           2000  4096    4   15.63     0%     0.003        0.03   0.00000  0.00000     0
R10-NRAWBC-PdCacOff           2000  4096    8   15.65 16.42%     0.006        0.03   0.00000  0.00000    95

otrdiena, 2013. gada 18. jūnijs

Testējam RAID 1+0 pret RAID 10 

RAID 1+0 4 diski

[root@bfsa ~]# cat /proc/mdstat 
Personalities : [raid10] [raid1] [raid0] 
md3 : active raid0 md2[1] md1[0]
      83841024 blocks super 1.2 512k chunks
md2 : active raid1 sdj1[1] sdd1[0]
      41920832 blocks super 1.2 [2/2] [UU]     
md1 : active raid1 sdc1[1] sdb1[0]
      41920832 blocks super 1.2 [2/2] [UU]

tiobench --size 8000 --dir /mnt/test1+0/  | tee -a /root/tio.md1+0
================
Sequential Reads
2.6.32-358.2.1.el6.x86_64     8000  4096    1  188.48 16.65%     0.021       79.78   0.00000  0.00000  1132
2.6.32-358.2.1.el6.x86_64     8000  4096    2  297.19 52.23%     0.026       81.99   0.00000  0.00000   569
2.6.32-358.2.1.el6.x86_64     8000  4096    4  246.01 74.94%     0.055      448.62   0.00000  0.00000   328
2.6.32-358.2.1.el6.x86_64     8000  4096    8  205.31 138.7%     0.146      304.43   0.00000  0.00000   148

Random Reads
2.6.32-358.2.1.el6.x86_64     8000  4096    1    0.73 0.354%     5.357       47.57   0.00000  0.00000   206
2.6.32-358.2.1.el6.x86_64     8000  4096    2    1.48 1.384%     5.134       41.11   0.00000  0.00000   107
2.6.32-358.2.1.el6.x86_64     8000  4096    4    1.89 3.478%     6.232       58.14   0.00000  0.00000    54
2.6.32-358.2.1.el6.x86_64     8000  4096    8    3.11 12.86%     8.934       99.02   0.00000  0.00000    24

Sequential Writes
2.6.32-358.2.1.el6.x86_64     8000  4096    1  197.38 40.46%     0.017      102.73   0.00000  0.00000   488
2.6.32-358.2.1.el6.x86_64     8000  4096    2  188.81 87.68%     0.036     2897.20   0.00005  0.00000   215
2.6.32-358.2.1.el6.x86_64     8000  4096    4  144.03 161.3%     0.095     5853.08   0.00059  0.00000    89
2.6.32-358.2.1.el6.x86_64     8000  4096    8  120.28 358.7%     0.229    10946.50   0.00171  0.00015    34

Random Writes
2.6.32-358.2.1.el6.x86_64     8000  4096    1    2.81 1.688%     0.006        0.05   0.00000  0.00000   166
2.6.32-358.2.1.el6.x86_64     8000  4096    2    2.81 3.738%     0.007        0.05   0.00000  0.00000    75
2.6.32-358.2.1.el6.x86_64     8000  4096    4    2.68 9.202%     0.009        0.05   0.00000  0.00000    29
2.6.32-358.2.1.el6.x86_64     8000  4096    8    2.73 18.87%     0.012        4.92   0.00000  0.00000    14

RAID 10 4 diski

mdadm --create --verbose /dev/md10 --level=10 --raid-devices=4 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sdj1

[root@bfsa ~]# cat /proc/mdstat 
Personalities : [raid10] [raid1] [raid0] 
md10 : active raid10 sdj1[3] sdd1[2] sdc1[1] sdb1[0]
      83841024 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
      
mkfs.ext4 /dev/md10
mount /dev/md10 /mnt/test10
      
[root@bfsa ~]# tiobench --size 8000 --dir /mnt/test10  | tee -a /root/tio.md10
Run #1: /usr/bin/tiotest -t 8 -f 1000 -r 500 -b 4096 -d /mnt/test10 -TT

====
Sequential Reads
2.6.32-358.2.1.el6.x86_64     8000  4096    1  115.54 9.296%     0.034       49.51   0.00000  0.00000  1243
2.6.32-358.2.1.el6.x86_64     8000  4096    2  217.76 36.41%     0.036      111.80   0.00000  0.00000   598
2.6.32-358.2.1.el6.x86_64     8000  4096    4  167.97 55.71%     0.092      187.27   0.00000  0.00000   302
2.6.32-358.2.1.el6.x86_64     8000  4096    8  164.52 103.0%     0.176      297.66   0.00000  0.00000   160

Random Reads
2.6.32-358.2.1.el6.x86_64     8000  4096    1    0.76 0.338%     5.164       34.67   0.00000  0.00000   223
2.6.32-358.2.1.el6.x86_64     8000  4096    2    1.54 1.263%     5.027       19.42   0.00000  0.00000   122
2.6.32-358.2.1.el6.x86_64     8000  4096    4    2.38 4.458%     6.453       76.78   0.00000  0.00000    53
2.6.32-358.2.1.el6.x86_64     8000  4096    8    2.77 10.43%     9.030      116.49   0.00000  0.00000    27

Sequential Writes
2.6.32-358.2.1.el6.x86_64     8000  4096    1  198.69 39.37%     0.017      126.57   0.00000  0.00000   505
2.6.32-358.2.1.el6.x86_64     8000  4096    2  140.85 73.33%     0.050     3834.55   0.00039  0.00000   192
2.6.32-358.2.1.el6.x86_64     8000  4096    4  121.15 169.8%     0.114     6588.93   0.00103  0.00000    71
2.6.32-358.2.1.el6.x86_64     8000  4096    8  105.19 501.7%     0.256    11103.32   0.00249  0.00005    21

Random Writes
2.6.32-358.2.1.el6.x86_64     8000  4096    1    2.86 1.776%     0.006        0.04   0.00000  0.00000   161
2.6.32-358.2.1.el6.x86_64     8000  4096    2    2.79 4.039%     0.007        0.06   0.00000  0.00000    69
2.6.32-358.2.1.el6.x86_64     8000  4096    4    2.65 11.11%     0.009        0.05   0.00000  0.00000    24
2.6.32-358.2.1.el6.x86_64     8000  4096    8    2.67 32.15%     0.012        8.04   0.00000  0.00000     8

RAID 1+0 6 diski

mdadm --create --verbose /dev/md1 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1 
mdadm --create --verbose /dev/md2 --level=1 --raid-devices=2 /dev/sdd1 /dev/sdj1 
mdadm --create --verbose /dev/md3 --level=1 --raid-devices=2 /dev/sdk1 /dev/sdn1 
mdadm --create --verbose /dev/md4 --level=stripe --raid-devices=3 /dev/md1 /dev/md2 /dev/md3

mkfs.ext4 /dev/md4
mount /dev/md4 /mnt/test1+0/

[root@bfsa ~]# cat /proc/mdstat 
Personalities : [raid10] [raid1] [raid0] 
md4 : active raid0 md3[2] md2[1] md1[0]
      90173952 blocks super 1.2 512k chunks      
md3 : active raid1 sdn1[1] sdk1[0]
      6333440 blocks super 1.2 [2/2] [UU]      
md2 : active raid1 sdj1[1] sdd1[0]
      41920832 blocks super 1.2 [2/2] [UU]      
md1 : active raid1 sdc1[1] sdb1[0]
      41920832 blocks super 1.2 [2/2] [UU]

[root@bfsa ~]# tiobench --size 8000 --dir /mnt/test1+0/  | tee -a /root/tio.md1+06drv
Run #1: /usr/bin/tiotest -t 8 -f 1000 -r 500 -b 4096 -d /mnt/test1+0/ -TT

================
Sequential Reads
2.6.32-358.2.1.el6.x86_64     8000  4096    1  277.03 23.62%     0.014       48.44   0.00000  0.00000  1173
2.6.32-358.2.1.el6.x86_64     8000  4096    2  392.34 70.15%     0.020      145.79   0.00000  0.00000   559
2.6.32-358.2.1.el6.x86_64     8000  4096    4  266.50 83.11%     0.050      159.51   0.00000  0.00000   321
2.6.32-358.2.1.el6.x86_64     8000  4096    8  220.63 145.7%     0.131      338.48   0.00000  0.00000   151

Random Reads
2.6.32-358.2.1.el6.x86_64     8000  4096    1    0.80 0.402%     4.903       17.29   0.00000  0.00000   198
2.6.32-358.2.1.el6.x86_64     8000  4096    2    1.54 1.694%     5.057       27.96   0.00000  0.00000    91
2.6.32-358.2.1.el6.x86_64     8000  4096    4    1.98 3.818%     5.796       44.83   0.00000  0.00000    52
2.6.32-358.2.1.el6.x86_64     8000  4096    8    2.72 11.67%     9.141      127.06   0.00000  0.00000    23

Sequential Writes
2.6.32-358.2.1.el6.x86_64     8000  4096    1  280.48 60.28%     0.012       85.91   0.00000  0.00000   465
2.6.32-358.2.1.el6.x86_64     8000  4096    2  266.83 131.8%     0.025     1967.06   0.00000  0.00000   202
2.6.32-358.2.1.el6.x86_64     8000  4096    4  147.23 173.3%     0.093     5948.60   0.00054  0.00000    85
2.6.32-358.2.1.el6.x86_64     8000  4096    8  136.94 345.4%     0.205     6689.59   0.00103  0.00000    40

Random Writes
2.6.32-358.2.1.el6.x86_64     8000  4096    1    4.27 2.488%     0.006        0.04   0.00000  0.00000   172
2.6.32-358.2.1.el6.x86_64     8000  4096    2    4.31 5.630%     0.007        0.06   0.00000  0.00000    77
2.6.32-358.2.1.el6.x86_64     8000  4096    4    2.86 9.743%     0.009        0.06   0.00000  0.00000    29
2.6.32-358.2.1.el6.x86_64     8000  4096    8    2.57 18.06%     0.012        5.15   0.00000  0.00000    14

RAID 10 6 diski

mdadm --create --verbose /dev/md10 --level=10 --raid-devices=6 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sdj1 /dev/sdk1 /dev/sdn1

[root@bfsa ~]# cat /proc/mdstat 
Personalities : [raid10] [raid1] [raid0] 
md10 : active raid10 sdn1[5] sdk1[4] sdj1[3] sdd1[2] sdc1[1] sdb1[0]
      19000320 blocks super 1.2 512K chunks 2 near-copies [6/6] [UUUUUU]

mkfs.ext4 /dev/md10
mount /dev/md10 /mnt/test10

[root@bfsa ~]# tiobench --size 8000 --dir /mnt/test10  | tee -a /root/tio.md106drv
Run #1: /usr/bin/tiotest -t 8 -f 1000 -r 500 -b 4096 -d /mnt/test10 -TT

================
Sequential Reads
2.6.32-358.2.1.el6.x86_64     8000  4096    1  166.17 13.41%     0.023       36.49   0.00000  0.00000  1239
2.6.32-358.2.1.el6.x86_64     8000  4096    2  251.41 42.26%     0.031       87.61   0.00000  0.00000   595
2.6.32-358.2.1.el6.x86_64     8000  4096    4  214.33 71.29%     0.072      206.46   0.00000  0.00000   301
2.6.32-358.2.1.el6.x86_64     8000  4096    8  218.76 148.0%     0.141      326.60   0.00000  0.00000   148

Random Reads
2.6.32-358.2.1.el6.x86_64     8000  4096    1    0.78 0.307%     5.033       60.76   0.00000  0.00000   252
2.6.32-358.2.1.el6.x86_64     8000  4096    2    1.45 1.176%     5.183      307.60   0.00000  0.00000   123
2.6.32-358.2.1.el6.x86_64     8000  4096    4    2.42 4.206%     5.972       44.55   0.00000  0.00000    57
2.6.32-358.2.1.el6.x86_64     8000  4096    8    4.10 16.05%     7.377       65.87   0.00000  0.00000    26

Sequential Writes
2.6.32-358.2.1.el6.x86_64     8000  4096    1  273.81 58.18%     0.013       63.96   0.00000  0.00000   471
2.6.32-358.2.1.el6.x86_64     8000  4096    2  160.74 91.67%     0.044     2419.02   0.00063  0.00000   175
2.6.32-358.2.1.el6.x86_64     8000  4096    4  133.57 223.6%     0.108     5177.93   0.00156  0.00000    60
2.6.32-358.2.1.el6.x86_64     8000  4096    8  159.02 646.6%     0.169     7223.63   0.00137  0.00000    25

Random Writes
2.6.32-358.2.1.el6.x86_64     8000  4096    1    4.19 2.684%     0.006        0.03   0.00000  0.00000   156
2.6.32-358.2.1.el6.x86_64     8000  4096    2    4.20 6.282%     0.007        0.06   0.00000  0.00000    67
2.6.32-358.2.1.el6.x86_64     8000  4096    4    4.05 17.82%     0.009        0.08   0.00000  0.00000    23
2.6.32-358.2.1.el6.x86_64     8000  4096    8    3.92 47.87%     0.011        4.62   0.00000  0.00000     8