ceturtdiena, 2013. gada 8. augusts

MegaRAID SAS 2108 - HW RAID 10 (2 + 2 + 2)


Base Board Information
        Manufacturer: Supermicro
        Product Name: X9DRi-LN4+/X9DR3-LN4+
q
RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS 2108 [Liberator] (rev 05)
SAS 9260CV-8i
Sistēma startēta ar parametru "mem = 1G"
FS - EXT4

HW RAID 10 

2 + 2 + 2

RWTD

storcli /c0/v0 show 

----------------------------------------------------------
DG/VD TYPE   State Access Consist Cache sCC     Size Name 
----------------------------------------------------------
0/0   RAID10 Optl  RW     No      RWTD  -   5.454 TB      
----------------------------------------------------------

root@ubuntu:~# tiobench --size 2000 --identifier R10-RWTD --numruns 3 
Run #3: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT
Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD                      2000  4096    1  164.01 10.61%     0.071      110.05   0.00000  0.00000  1545
R10-RWTD                      2000  4096    2  136.11 16.77%     0.168      149.97   0.00000  0.00000   811
R10-RWTD                      2000  4096    4  127.26 36.09%     0.346      310.12   0.00000  0.00000   353
R10-RWTD                      2000  4096    8  143.80 82.40%     0.600      512.41   0.00000  0.00000   174

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD                      2000  4096    1    1.05 0.510%    11.167      103.91   0.00000  0.00000   206
R10-RWTD                      2000  4096    2    1.86 3.086%    10.919      100.91   0.00000  0.00000    60
R10-RWTD                      2000  4096    4    3.40 6.955%    11.365      119.16   0.00000  0.00000    49
R10-RWTD                      2000  4096    8    5.30 20.34%    13.479      241.22   0.00000  0.00000    26

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD                      2000  4096    1  244.78 33.16%     0.046      963.48   0.00000  0.00000   738
R10-RWTD                      2000  4096    2  199.02 62.89%     0.112      758.66   0.00000  0.00000   316
R10-RWTD                      2000  4096    4  186.14 117.4%     0.241      560.05   0.00000  0.00000   159
R10-RWTD                      2000  4096    8  177.05 216.8%     0.508      711.64   0.00000  0.00000    82

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD                      2000  4096    1    8.06     0%     0.009        0.06   0.00000  0.00000     0
R10-RWTD                      2000  4096    2    7.78     0%     0.009        0.04   0.00000  0.00000     0
R10-RWTD                      2000  4096    4    7.82     0%     0.010        0.08   0.00000  0.00000     0
R10-RWTD                      2000  4096    8    7.90 9.166%     0.017        0.09   0.00000  0.00000    86

RWTC

storcli /c0/v0 set iopolicy=cached
storcli /c0/v0 show

----------------------------------------------------------
DG/VD TYPE   State Access Consist Cache sCC     Size Name 
----------------------------------------------------------
0/0   RAID10 Optl  RW     No      RWTC  -   5.454 TB      
----------------------------------------------------------

root@ubuntu:~# tiobench --size 2000 --identifier R10-RWTC --numruns 3 
Run #3: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1  169.42 10.76%     0.069      108.65   0.00000  0.00000  1574
R10-RWTC                      2000  4096    2  139.71 17.76%     0.165      162.62   0.00000  0.00000   787
R10-RWTC                      2000  4096    4  133.68 37.36%     0.325      237.25   0.00000  0.00000   358
R10-RWTC                      2000  4096    8  132.39 75.94%     0.645      637.03   0.00000  0.00000   174

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1    1.04 0.694%    11.234       84.40   0.00000  0.00000   150
R10-RWTC                      2000  4096    2    1.98 3.624%    11.001       82.62   0.00000  0.00000    55
R10-RWTC                      2000  4096    4    3.36 5.441%    11.392      165.62   0.00000  0.00000    62
R10-RWTC                      2000  4096    8    5.25 19.31%    13.533      180.47   0.00000  0.00000    27

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1  247.02 30.99%     0.045      322.56   0.00000  0.00000   797
R10-RWTC                      2000  4096    2  203.29 67.52%     0.110      563.23   0.00000  0.00000   301
R10-RWTC                      2000  4096    4  185.36 116.4%     0.241      526.25   0.00000  0.00000   159
R10-RWTC                      2000  4096    8  177.23 225.7%     0.506      616.67   0.00000  0.00000    78

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1    7.41     0%     0.009        0.06   0.00000  0.00000     0
R10-RWTC                      2000  4096    2    7.98     0%     0.009        0.05   0.00000  0.00000     0
R10-RWTC                      2000  4096    4    7.76 1.059%     0.010        0.08   0.00000  0.00000   732
R10-RWTC                      2000  4096    8    7.92 8.446%     0.017        0.07   0.00000  0.00000    94

RWTC - Disk cache enabled 

tiobench --size 2000 --identifier R10-RWTC 
Run #1: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1  222.69 15.58%     0.017       24.78   0.00000  0.00000  1428
R10-RWTC                      2000  4096    2  126.77 15.46%     0.061       58.48   0.00000  0.00000   820
R10-RWTC                      2000  4096    4  162.10 44.19%     0.087       66.98   0.00000  0.00000   367
R10-RWTC                      2000  4096    8  126.15 72.59%     0.230      208.33   0.00000  0.00000   174

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1    1.09 0.694%     3.596       27.37   0.00000  0.00000   156
R10-RWTC                      2000  4096    2    2.01 1.441%     3.693       25.86   0.00000  0.00000   139
R10-RWTC                      2000  4096    4    2.94 3.841%     3.876       28.29   0.00000  0.00000    77
R10-RWTC                      2000  4096    8    5.25 17.22%     4.559       70.20   0.00000  0.00000    31

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1  281.20 36.22%     0.013      174.99   0.00000  0.00000   776
R10-RWTC                      2000  4096    2  199.49 64.79%     0.037      171.67   0.00000  0.00000   308
R10-RWTC                      2000  4096    4  188.08 119.7%     0.079      175.89   0.00000  0.00000   157
R10-RWTC                      2000  4096    8  176.24 208.0%     0.169      256.70   0.00000  0.00000    85

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1    8.10     0%     0.003        0.03   0.00000  0.00000     0
R10-RWTC                      2000  4096    2    7.89     0%     0.003        0.02   0.00000  0.00000     0
R10-RWTC                      2000  4096    4    7.79     0%     0.003        0.03   0.00000  0.00000     0
R10-RWTC                      2000  4096    8    7.93 6.496%     0.006        0.03   0.00000  0.00000   122

RWTC - Disk cache disabled

# tiobench --size 2000 --identifier R10-RWTC-PdCacOff 
Run #1: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC-PdCacOff             2000  4096    1  221.14 14.06%     0.018       23.75   0.00000  0.00000  1572
R10-RWTC-PdCacOff             2000  4096    2  122.06 15.99%     0.063       50.35   0.00000  0.00000   763
R10-RWTC-PdCacOff             2000  4096    4  131.95 35.99%     0.108       83.59   0.00000  0.00000   367
R10-RWTC-PdCacOff             2000  4096    8  130.39 73.15%     0.219      183.32   0.00000  0.00000   178

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC-PdCacOff             2000  4096    1    1.11 0.509%     3.532       39.69   0.00000  0.00000   217
R10-RWTC-PdCacOff             2000  4096    2    2.11 3.894%     3.485       26.37   0.00000  0.00000    54
R10-RWTC-PdCacOff             2000  4096    4    2.99 3.209%     3.787       32.58   0.00000  0.00000    93
R10-RWTC-PdCacOff             2000  4096    8    4.89 17.40%     4.410       82.89   0.00000  0.00000    28

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC-PdCacOff             2000  4096    1  218.60 29.25%     0.017      237.86   0.00000  0.00000   747
R10-RWTC-PdCacOff             2000  4096    2  171.45 56.99%     0.043      209.19   0.00000  0.00000   301
R10-RWTC-PdCacOff             2000  4096    4  157.12 102.8%     0.095      250.64   0.00000  0.00000   153
R10-RWTC-PdCacOff             2000  4096    8  138.59 166.9%     0.209      340.25   0.00000  0.00000    83

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC-PdCacOff             2000  4096    1    6.02     0%     0.003        0.02   0.00000  0.00000     0
R10-RWTC-PdCacOff             2000  4096    2    5.87     0%     0.003        0.02   0.00000  0.00000     0
R10-RWTC-PdCacOff             2000  4096    4    5.98     0%     0.003        0.03   0.00000  0.00000     0
R10-RWTC-PdCacOff             2000  4096    8    5.99 6.137%     0.006        0.05   0.00000  0.00000    98

RWTD - Disk cache disabled

root@ubuntu:~# tiobench --size 2000 --identifier R10-RWTD-PdCacOff 
Run #1: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOff             2000  4096    1  225.59 13.71%     0.017       23.27   0.00000  0.00000  1645
R10-RWTD-PdCacOff             2000  4096    2  128.78 18.03%     0.060       74.44   0.00000  0.00000   714
R10-RWTD-PdCacOff             2000  4096    4  120.24 34.48%     0.125      108.35   0.00000  0.00000   349
R10-RWTD-PdCacOff             2000  4096    8  150.00 78.87%     0.178      183.56   0.00000  0.00000   190

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOff             2000  4096    1    1.07 0.518%     3.662       28.25   0.00000  0.00000   206
R10-RWTD-PdCacOff             2000  4096    2    2.12 3.794%     3.478       32.29   0.00000  0.00000    56
R10-RWTD-PdCacOff             2000  4096    4    3.92 5.926%     3.850       33.38   0.00000  0.00000    66
R10-RWTD-PdCacOff             2000  4096    8    5.40 14.65%     4.195       54.68   0.00000  0.00000    37

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOff             2000  4096    1  211.89 27.29%     0.018      231.49   0.00000  0.00000   776
R10-RWTD-PdCacOff             2000  4096    2  176.41 58.00%     0.042      224.31   0.00000  0.00000   304
R10-RWTD-PdCacOff             2000  4096    4  156.11 97.60%     0.094      261.58   0.00000  0.00000   160
R10-RWTD-PdCacOff             2000  4096    8  137.09 164.4%     0.209      325.76   0.00000  0.00000    83

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOff             2000  4096    1    6.12     0%     0.003        0.02   0.00000  0.00000     0
R10-RWTD-PdCacOff             2000  4096    2    6.24     0%     0.003        0.02   0.00000  0.00000     0
R10-RWTD-PdCacOff             2000  4096    4    6.00     0%     0.003        0.02   0.00000  0.00000     0
R10-RWTD-PdCacOff             2000  4096    8    6.05 4.957%     0.006        0.04   0.00000  0.00000   122

RWTD - Disk cache enabled

tiobench --size 2000 --identifier R10-RWTD-PdCacOn 
Run #1: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOn              2000  4096    1  222.84 14.61%     0.017       20.89   0.00000  0.00000  1524
R10-RWTD-PdCacOn              2000  4096    2  129.22 15.81%     0.060       48.82   0.00000  0.00000   817
R10-RWTD-PdCacOn              2000  4096    4  131.79 38.19%     0.110       83.58   0.00000  0.00000   345
R10-RWTD-PdCacOn              2000  4096    8  141.00 75.24%     0.188      151.67   0.00000  0.00000   187

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOn              2000  4096    1    1.06 0.703%     3.697       27.83   0.00000  0.00000   150
R10-RWTD-PdCacOn              2000  4096    2    1.99 0.970%     3.814       25.99   0.00000  0.00000   206
R10-RWTD-PdCacOn              2000  4096    4    2.88 6.938%     4.034       51.90   0.00000  0.00000    42
R10-RWTD-PdCacOn              2000  4096    8    5.36 14.82%     3.940       54.28   0.00000  0.00000    36

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOn              2000  4096    1  280.53 34.56%     0.013      205.93   0.00000  0.00000   812
R10-RWTD-PdCacOn              2000  4096    2  199.38 62.92%     0.037      165.82   0.00000  0.00000   317
R10-RWTD-PdCacOn              2000  4096    4  191.24 96.84%     0.079      157.06   0.00000  0.00000   197
R10-RWTD-PdCacOn              2000  4096    8  175.74 206.3%     0.170      391.90   0.00000  0.00000    85

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOn              2000  4096    1    7.98 0.204%     0.003        0.01   0.00000  0.00000  3906
R10-RWTD-PdCacOn              2000  4096    2    7.78     0%     0.003        0.01   0.00000  0.00000     0
R10-RWTD-PdCacOn              2000  4096    4    8.24 1.687%     0.003        0.03   0.00000  0.00000   488
R10-RWTD-PdCacOn              2000  4096    8    8.10 11.61%     0.006        0.03   0.00000  0.00000    70

Always Write Back 

   storcli /c0/v0 set wrcache=AWB

RAWBD PdCacOn 

storcli /c0/v0 show
----------------------------------------------------------
DG/VD TYPE   State Access Consist Cache sCC     Size Name 
----------------------------------------------------------
0/0   RAID10 Optl  RW     No      RAWBD -   5.454 TB      
----------------------------------------------------------

tiobench --size 2000 --identifier R10-RAWBD-PdCacOn  --numruns 3
Run #3: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Unit information
================
File size = megabytes
Blk Size  = bytes
Rate      = megabytes per second
CPU%      = percentage of CPU used during the test
Latency   = milliseconds
Lat%      = percent of requests that took longer than X seconds
CPU Eff   = Rate divided by CPU% - throughput per cpu load

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBD-PdCacOn             2000  4096    1  146.53 9.671%     0.080     1344.87   0.00000  0.00000  1515
R10-RAWBD-PdCacOn             2000  4096    2  143.45 16.55%     0.158     1364.29   0.00000  0.00000   866
R10-RAWBD-PdCacOn             2000  4096    4  119.89 32.65%     0.362     1940.91   0.00000  0.00000   367
R10-RAWBD-PdCacOn             2000  4096    8  127.14 70.38%     0.668     1781.70   0.00000  0.00000   181

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBD-PdCacOn             2000  4096    1    1.17 0.616%    10.053      117.77   0.00000  0.00000   189
R10-RAWBD-PdCacOn             2000  4096    2    1.94 3.273%    10.045       85.48   0.00000  0.00000    59
R10-RAWBD-PdCacOn             2000  4096    4    3.43 5.976%    10.275      119.88   0.00000  0.00000    57
R10-RAWBD-PdCacOn             2000  4096    8    6.03 18.00%    11.818      185.64   0.00000  0.00000    33

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBD-PdCacOn             2000  4096    1  293.70 42.82%     0.039      252.09   0.00000  0.00000   686
R10-RAWBD-PdCacOn             2000  4096    2  263.84 83.16%     0.086      353.42   0.00000  0.00000   317
R10-RAWBD-PdCacOn             2000  4096    4  275.19 165.3%     0.163      474.69   0.00000  0.00000   166
R10-RAWBD-PdCacOn             2000  4096    8  257.68 351.0%     0.343     1178.33   0.00000  0.00000    73

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBD-PdCacOn             2000  4096    1   14.03     0%     0.009        0.08   0.00000  0.00000     0
R10-RAWBD-PdCacOn             2000  4096    2   15.38 1.312%     0.008        0.04   0.00000  0.00000  1172
R10-RAWBD-PdCacOn             2000  4096    4   14.37     0%     0.010        0.07   0.00000  0.00000     0
R10-RAWBD-PdCacOn             2000  4096    8   18.77 9.612%     0.017        0.10   0.00000  0.00000   195

RAWBC - Disk Cache On

storcli /c0/v0 set iopolicy=cached
storcli /c0/v0 show

----------------------------------------------------------
DG/VD TYPE   State Access Consist Cache sCC     Size Name 
----------------------------------------------------------
0/0   RAID10 Optl  RW     No      RAWBC -   5.454 TB      
----------------------------------------------------------

tiobench --size 2000 --identifier R10-RAWBC-PdCacOn  --numruns 3
Run #3: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOn             2000  4096    1  146.89 9.313%     0.080      986.74   0.00000  0.00000  1577
R10-RAWBC-PdCacOn             2000  4096    2  124.14 14.54%     0.185     1406.61   0.00000  0.00000   853
R10-RAWBC-PdCacOn             2000  4096    4  125.12 33.76%     0.347     1811.97   0.00000  0.00000   371
R10-RAWBC-PdCacOn             2000  4096    8  126.05 70.05%     0.679     1398.69   0.00000  0.00000   180

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOn             2000  4096    1    1.19 0.617%     9.882       96.18   0.00000  0.00000   192
R10-RAWBC-PdCacOn             2000  4096    2    2.02 2.255%    10.182       95.31   0.00000  0.00000    89
R10-RAWBC-PdCacOn             2000  4096    4    3.33 4.808%    10.761      140.33   0.00000  0.00000    69
R10-RAWBC-PdCacOn             2000  4096    8    5.98 17.81%    12.368      173.46   0.00000  0.00000    34

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOn             2000  4096    1  295.63 39.42%     0.038      280.16   0.00000  0.00000   750
R10-RAWBC-PdCacOn             2000  4096    2  258.20 79.84%     0.088      190.60   0.00000  0.00000   323
R10-RAWBC-PdCacOn             2000  4096    4  282.15 140.9%     0.160      848.78   0.00000  0.00000   200
R10-RAWBC-PdCacOn             2000  4096    8  255.79 340.0%     0.342      400.85   0.00000  0.00000    75

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOn             2000  4096    1   16.55     0%     0.009        0.06   0.00000  0.00000     0
R10-RAWBC-PdCacOn             2000  4096    2   18.37     0%     0.008        0.06   0.00000  0.00000     0
R10-RAWBC-PdCacOn             2000  4096    4   13.72 1.873%     0.010        0.07   0.00000  0.00000   732
R10-RAWBC-PdCacOn             2000  4096    8   13.51 12.91%     0.016        0.09   0.00000  0.00000   105

RAWBC - Disk Cache Off

 storcli /c0/v0 set pdcache=Off  
tiobench --size 2000 --identifier R10-RAWBC-PdCacOff  --numruns 1
Run #1: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOff            2000  4096    1  178.90 10.80%     0.022      198.66   0.00000  0.00000  1656
R10-RAWBC-PdCacOff            2000  4096    2  111.23 12.90%     0.069      331.54   0.00000  0.00000   862
R10-RAWBC-PdCacOff            2000  4096    4  173.12 48.85%     0.087      402.51   0.00000  0.00000   354
R10-RAWBC-PdCacOff            2000  4096    8  135.25 71.82%     0.202      400.28   0.00000  0.00000   188

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOff            2000  4096    1    1.14 0.552%     3.440       29.15   0.00000  0.00000   206
R10-RAWBC-PdCacOff            2000  4096    2    2.11 3.455%     3.378       51.59   0.00000  0.00000    61
R10-RAWBC-PdCacOff            2000  4096    4    3.84 3.635%     3.434       31.46   0.00000  0.00000   106
R10-RAWBC-PdCacOff            2000  4096    8    5.95 20.72%     3.958       53.06   0.00000  0.00000    29

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOff            2000  4096    1  277.00 34.90%     0.013      320.74   0.00000  0.00000   794
R10-RAWBC-PdCacOff            2000  4096    2  211.19 67.75%     0.036      133.28   0.00000  0.00000   312
R10-RAWBC-PdCacOff            2000  4096    4  279.79 164.3%     0.053      116.90   0.00000  0.00000   170
R10-RAWBC-PdCacOff            2000  4096    8  212.86 254.1%     0.140      279.91   0.00000  0.00000    84

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOff            2000  4096    1   15.27     0%     0.003        0.02   0.00000  0.00000     0
R10-RAWBC-PdCacOff            2000  4096    2   14.53 0.744%     0.003        0.03   0.00000  0.00000  1953
R10-RAWBC-PdCacOff            2000  4096    4   18.08     0%     0.003        0.03   0.00000  0.00000     0
R10-RAWBC-PdCacOff            2000  4096    8   15.72 9.660%     0.006        0.03   0.00000  0.00000   163

NRAWBC - Disk Cache Off

storcli /c0/v0 set rdcache=nora
storcli /c0/v0 show
-----------------------------------------------------------
DG/VD TYPE   State Access Consist Cache  sCC     Size Name 
-----------------------------------------------------------
0/0   RAID10 Optl  RW     No      NRAWBC -   5.454 TB      
-----------------------------------------------------------

tiobench --size 2000 --identifier R10-NRAWBC-PdCacOff  --numruns 1
Run #1: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-NRAWBC-PdCacOff           2000  4096    1  190.21 11.41%     0.020      433.66   0.00000  0.00000  1667
R10-NRAWBC-PdCacOff           2000  4096    2  109.12 14.42%     0.071      275.32   0.00000  0.00000   756
R10-NRAWBC-PdCacOff           2000  4096    4  123.45 33.13%     0.110      373.31   0.00000  0.00000   373
R10-NRAWBC-PdCacOff           2000  4096    8  128.07 70.72%     0.219      350.40   0.00000  0.00000   181

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-NRAWBC-PdCacOff           2000  4096    1    1.22 0.591%     3.210       61.00   0.00000  0.00000   206
R10-NRAWBC-PdCacOff           2000  4096    2    2.17 3.770%     3.446       48.13   0.00000  0.00000    57
R10-NRAWBC-PdCacOff           2000  4096    4    3.28 4.874%     3.360       34.23   0.00000  0.00000    67
R10-NRAWBC-PdCacOff           2000  4096    8    6.04 17.45%     3.928       48.08   0.00000  0.00000    35

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-NRAWBC-PdCacOff           2000  4096    1  288.05 39.86%     0.013      267.42   0.00000  0.00000   722
R10-NRAWBC-PdCacOff           2000  4096    2  216.66 73.93%     0.035       59.29   0.00000  0.00000   293
R10-NRAWBC-PdCacOff           2000  4096    4  262.35 133.5%     0.057      122.33   0.00000  0.00000   196
R10-NRAWBC-PdCacOff           2000  4096    8  213.97 265.4%     0.138      175.91   0.00000  0.00000    81

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-NRAWBC-PdCacOff           2000  4096    1   14.42     0%     0.003        0.02   0.00000  0.00000     0
R10-NRAWBC-PdCacOff           2000  4096    2   17.76     0%     0.003        0.02   0.00000  0.00000     0
R10-NRAWBC-PdCacOff           2000  4096    4   15.63     0%     0.003        0.03   0.00000  0.00000     0
R10-NRAWBC-PdCacOff           2000  4096    8   15.65 16.42%     0.006        0.03   0.00000  0.00000    95
plkst.
Etiķetes:

otrdiena, 2013. gada 18. jūnijs

Testējam RAID 1+0 pret RAID 10 

RAID 1+0 4 diski 

[root@bfsa ~]# cat /proc/mdstat 
Personalities : [raid10] [raid1] [raid0] 
md3 : active raid0 md2[1] md1[0]
      83841024 blocks super 1.2 512k chunks
md2 : active raid1 sdj1[1] sdd1[0]
      41920832 blocks super 1.2 [2/2] [UU]     
md1 : active raid1 sdc1[1] sdb1[0]
      41920832 blocks super 1.2 [2/2] [UU]

tiobench --size 8000 --dir /mnt/test1+0/  | tee -a /root/tio.md1+0
================
Sequential Reads
2.6.32-358.2.1.el6.x86_64     8000  4096    1  188.48 16.65%     0.021       79.78   0.00000  0.00000  1132
2.6.32-358.2.1.el6.x86_64     8000  4096    2  297.19 52.23%     0.026       81.99   0.00000  0.00000   569
2.6.32-358.2.1.el6.x86_64     8000  4096    4  246.01 74.94%     0.055      448.62   0.00000  0.00000   328
2.6.32-358.2.1.el6.x86_64     8000  4096    8  205.31 138.7%     0.146      304.43   0.00000  0.00000   148

Random Reads
2.6.32-358.2.1.el6.x86_64     8000  4096    1    0.73 0.354%     5.357       47.57   0.00000  0.00000   206
2.6.32-358.2.1.el6.x86_64     8000  4096    2    1.48 1.384%     5.134       41.11   0.00000  0.00000   107
2.6.32-358.2.1.el6.x86_64     8000  4096    4    1.89 3.478%     6.232       58.14   0.00000  0.00000    54
2.6.32-358.2.1.el6.x86_64     8000  4096    8    3.11 12.86%     8.934       99.02   0.00000  0.00000    24

Sequential Writes
2.6.32-358.2.1.el6.x86_64     8000  4096    1  197.38 40.46%     0.017      102.73   0.00000  0.00000   488
2.6.32-358.2.1.el6.x86_64     8000  4096    2  188.81 87.68%     0.036     2897.20   0.00005  0.00000   215
2.6.32-358.2.1.el6.x86_64     8000  4096    4  144.03 161.3%     0.095     5853.08   0.00059  0.00000    89
2.6.32-358.2.1.el6.x86_64     8000  4096    8  120.28 358.7%     0.229    10946.50   0.00171  0.00015    34

Random Writes
2.6.32-358.2.1.el6.x86_64     8000  4096    1    2.81 1.688%     0.006        0.05   0.00000  0.00000   166
2.6.32-358.2.1.el6.x86_64     8000  4096    2    2.81 3.738%     0.007        0.05   0.00000  0.00000    75
2.6.32-358.2.1.el6.x86_64     8000  4096    4    2.68 9.202%     0.009        0.05   0.00000  0.00000    29
2.6.32-358.2.1.el6.x86_64     8000  4096    8    2.73 18.87%     0.012        4.92   0.00000  0.00000    14

RAID 10 4 diski 

mdadm --create --verbose /dev/md10 --level=10 --raid-devices=4 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sdj1

[root@bfsa ~]# cat /proc/mdstat 
Personalities : [raid10] [raid1] [raid0] 
md10 : active raid10 sdj1[3] sdd1[2] sdc1[1] sdb1[0]
      83841024 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
      
mkfs.ext4 /dev/md10
mount /dev/md10 /mnt/test10
      
[root@bfsa ~]# tiobench --size 8000 --dir /mnt/test10  | tee -a /root/tio.md10
Run #1: /usr/bin/tiotest -t 8 -f 1000 -r 500 -b 4096 -d /mnt/test10 -TT

====
Sequential Reads
2.6.32-358.2.1.el6.x86_64     8000  4096    1  115.54 9.296%     0.034       49.51   0.00000  0.00000  1243
2.6.32-358.2.1.el6.x86_64     8000  4096    2  217.76 36.41%     0.036      111.80   0.00000  0.00000   598
2.6.32-358.2.1.el6.x86_64     8000  4096    4  167.97 55.71%     0.092      187.27   0.00000  0.00000   302
2.6.32-358.2.1.el6.x86_64     8000  4096    8  164.52 103.0%     0.176      297.66   0.00000  0.00000   160

Random Reads
2.6.32-358.2.1.el6.x86_64     8000  4096    1    0.76 0.338%     5.164       34.67   0.00000  0.00000   223
2.6.32-358.2.1.el6.x86_64     8000  4096    2    1.54 1.263%     5.027       19.42   0.00000  0.00000   122
2.6.32-358.2.1.el6.x86_64     8000  4096    4    2.38 4.458%     6.453       76.78   0.00000  0.00000    53
2.6.32-358.2.1.el6.x86_64     8000  4096    8    2.77 10.43%     9.030      116.49   0.00000  0.00000    27

Sequential Writes
2.6.32-358.2.1.el6.x86_64     8000  4096    1  198.69 39.37%     0.017      126.57   0.00000  0.00000   505
2.6.32-358.2.1.el6.x86_64     8000  4096    2  140.85 73.33%     0.050     3834.55   0.00039  0.00000   192
2.6.32-358.2.1.el6.x86_64     8000  4096    4  121.15 169.8%     0.114     6588.93   0.00103  0.00000    71
2.6.32-358.2.1.el6.x86_64     8000  4096    8  105.19 501.7%     0.256    11103.32   0.00249  0.00005    21

Random Writes
2.6.32-358.2.1.el6.x86_64     8000  4096    1    2.86 1.776%     0.006        0.04   0.00000  0.00000   161
2.6.32-358.2.1.el6.x86_64     8000  4096    2    2.79 4.039%     0.007        0.06   0.00000  0.00000    69
2.6.32-358.2.1.el6.x86_64     8000  4096    4    2.65 11.11%     0.009        0.05   0.00000  0.00000    24
2.6.32-358.2.1.el6.x86_64     8000  4096    8    2.67 32.15%     0.012        8.04   0.00000  0.00000     8

RAID 1+0 6 diski 

mdadm --create --verbose /dev/md1 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1 
mdadm --create --verbose /dev/md2 --level=1 --raid-devices=2 /dev/sdd1 /dev/sdj1 
mdadm --create --verbose /dev/md3 --level=1 --raid-devices=2 /dev/sdk1 /dev/sdn1 
mdadm --create --verbose /dev/md4 --level=stripe --raid-devices=3 /dev/md1 /dev/md2 /dev/md3

mkfs.ext4 /dev/md4
mount /dev/md4 /mnt/test1+0/

[root@bfsa ~]# cat /proc/mdstat 
Personalities : [raid10] [raid1] [raid0] 
md4 : active raid0 md3[2] md2[1] md1[0]
      90173952 blocks super 1.2 512k chunks      
md3 : active raid1 sdn1[1] sdk1[0]
      6333440 blocks super 1.2 [2/2] [UU]      
md2 : active raid1 sdj1[1] sdd1[0]
      41920832 blocks super 1.2 [2/2] [UU]      
md1 : active raid1 sdc1[1] sdb1[0]
      41920832 blocks super 1.2 [2/2] [UU]

[root@bfsa ~]# tiobench --size 8000 --dir /mnt/test1+0/  | tee -a /root/tio.md1+06drv
Run #1: /usr/bin/tiotest -t 8 -f 1000 -r 500 -b 4096 -d /mnt/test1+0/ -TT

================
Sequential Reads
2.6.32-358.2.1.el6.x86_64     8000  4096    1  277.03 23.62%     0.014       48.44   0.00000  0.00000  1173
2.6.32-358.2.1.el6.x86_64     8000  4096    2  392.34 70.15%     0.020      145.79   0.00000  0.00000   559
2.6.32-358.2.1.el6.x86_64     8000  4096    4  266.50 83.11%     0.050      159.51   0.00000  0.00000   321
2.6.32-358.2.1.el6.x86_64     8000  4096    8  220.63 145.7%     0.131      338.48   0.00000  0.00000   151

Random Reads
2.6.32-358.2.1.el6.x86_64     8000  4096    1    0.80 0.402%     4.903       17.29   0.00000  0.00000   198
2.6.32-358.2.1.el6.x86_64     8000  4096    2    1.54 1.694%     5.057       27.96   0.00000  0.00000    91
2.6.32-358.2.1.el6.x86_64     8000  4096    4    1.98 3.818%     5.796       44.83   0.00000  0.00000    52
2.6.32-358.2.1.el6.x86_64     8000  4096    8    2.72 11.67%     9.141      127.06   0.00000  0.00000    23

Sequential Writes
2.6.32-358.2.1.el6.x86_64     8000  4096    1  280.48 60.28%     0.012       85.91   0.00000  0.00000   465
2.6.32-358.2.1.el6.x86_64     8000  4096    2  266.83 131.8%     0.025     1967.06   0.00000  0.00000   202
2.6.32-358.2.1.el6.x86_64     8000  4096    4  147.23 173.3%     0.093     5948.60   0.00054  0.00000    85
2.6.32-358.2.1.el6.x86_64     8000  4096    8  136.94 345.4%     0.205     6689.59   0.00103  0.00000    40

Random Writes
2.6.32-358.2.1.el6.x86_64     8000  4096    1    4.27 2.488%     0.006        0.04   0.00000  0.00000   172
2.6.32-358.2.1.el6.x86_64     8000  4096    2    4.31 5.630%     0.007        0.06   0.00000  0.00000    77
2.6.32-358.2.1.el6.x86_64     8000  4096    4    2.86 9.743%     0.009        0.06   0.00000  0.00000    29
2.6.32-358.2.1.el6.x86_64     8000  4096    8    2.57 18.06%     0.012        5.15   0.00000  0.00000    14

RAID 10 6 diski 

mdadm --create --verbose /dev/md10 --level=10 --raid-devices=6 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sdj1 /dev/sdk1 /dev/sdn1

[root@bfsa ~]# cat /proc/mdstat 
Personalities : [raid10] [raid1] [raid0] 
md10 : active raid10 sdn1[5] sdk1[4] sdj1[3] sdd1[2] sdc1[1] sdb1[0]
      19000320 blocks super 1.2 512K chunks 2 near-copies [6/6] [UUUUUU]

mkfs.ext4 /dev/md10
mount /dev/md10 /mnt/test10

[root@bfsa ~]# tiobench --size 8000 --dir /mnt/test10  | tee -a /root/tio.md106drv
Run #1: /usr/bin/tiotest -t 8 -f 1000 -r 500 -b 4096 -d /mnt/test10 -TT

================
Sequential Reads
2.6.32-358.2.1.el6.x86_64     8000  4096    1  166.17 13.41%     0.023       36.49   0.00000  0.00000  1239
2.6.32-358.2.1.el6.x86_64     8000  4096    2  251.41 42.26%     0.031       87.61   0.00000  0.00000   595
2.6.32-358.2.1.el6.x86_64     8000  4096    4  214.33 71.29%     0.072      206.46   0.00000  0.00000   301
2.6.32-358.2.1.el6.x86_64     8000  4096    8  218.76 148.0%     0.141      326.60   0.00000  0.00000   148

Random Reads
2.6.32-358.2.1.el6.x86_64     8000  4096    1    0.78 0.307%     5.033       60.76   0.00000  0.00000   252
2.6.32-358.2.1.el6.x86_64     8000  4096    2    1.45 1.176%     5.183      307.60   0.00000  0.00000   123
2.6.32-358.2.1.el6.x86_64     8000  4096    4    2.42 4.206%     5.972       44.55   0.00000  0.00000    57
2.6.32-358.2.1.el6.x86_64     8000  4096    8    4.10 16.05%     7.377       65.87   0.00000  0.00000    26

Sequential Writes
2.6.32-358.2.1.el6.x86_64     8000  4096    1  273.81 58.18%     0.013       63.96   0.00000  0.00000   471
2.6.32-358.2.1.el6.x86_64     8000  4096    2  160.74 91.67%     0.044     2419.02   0.00063  0.00000   175
2.6.32-358.2.1.el6.x86_64     8000  4096    4  133.57 223.6%     0.108     5177.93   0.00156  0.00000    60
2.6.32-358.2.1.el6.x86_64     8000  4096    8  159.02 646.6%     0.169     7223.63   0.00137  0.00000    25

Random Writes
2.6.32-358.2.1.el6.x86_64     8000  4096    1    4.19 2.684%     0.006        0.03   0.00000  0.00000   156
2.6.32-358.2.1.el6.x86_64     8000  4096    2    4.20 6.282%     0.007        0.06   0.00000  0.00000    67
2.6.32-358.2.1.el6.x86_64     8000  4096    4    4.05 17.82%     0.009        0.08   0.00000  0.00000    23
2.6.32-358.2.1.el6.x86_64     8000  4096    8    3.92 47.87%     0.011        4.62   0.00000  0.00000     8
plkst.
Etiķetes:

piektdiena, 2012. gada 2. novembris

Melnraksts - pykota


Melnraksts - liekam WINBIND/PYKOTA/CUPS

apt-get install winbind
apt-get install krb5-config krb5-user
scp -r root@druka:/etc/krb5.conf /etc/
scp -r root@druka:/etc/cups /etc/
scp -r root@druka:/etc/samba /etc/
net ads join -S vaserv -U admin_vea
vim /etc/nsswitch.conf
/etc/init.d/winbind restart




apt-get install subversion
cd /usr/local/src/
svn co http://svn.pykota.com/pykota/trunk pykota
cd pykota
/* # python -V
Python 2.7.3 */
python checkdeps.py
apt-get install python-pygresql python-jaxml python-reportlab python-imaging pkpgcounter python-pam
cd /usr/local/src
svn co http://svn.pykota.com/pkipplib/trunk pkipplib
cd pkipplib/
python setup.py install
cd /usr/local/src/pykota
python checkdeps.py
python setup.py install -v -f | tee -a install.log

adduser --system --group --home /etc/pykota --gecos PyKota pykota
cp /usr/local/share/pykota/cupspykota /usr/lib/cups/backend/cupspykota
cp /usr/local/share/pykota/conf/pykota.conf.sample /etc/pykota/pykota.conf
cp /usr/local/share/pykota/conf/pykotadmin.conf.sample /etc/pykota/pykotadmin.conf
chmod 644 /etc/pykota/pykota.conf
chmod 640 /etc/pykota/pykotadmin.conf
chown pykota.pykota /etc/pykota/pykota.conf /etc/pykota/pykotadmin.conf


/*
[global]
Debug: Yes
config_charset : UTF-8
storagebackend: pgstorage
storageserver: stende
storagename: XXX
storageuser : XXX
storageuserpw : XXX
storagecaching: No
disablehistory: No
logger: system
logourl : http://www.pykota.com/pykota.png
logolink : http://www.pykota.com/
smtpserver: localhost
maildomain: venta.lv
usernamecase: lower
privacy : no
onbackenderror : nocharge
keepfiles : no
accounter: software()
skipinitialwait : no
preaccounter: software()
onaccountererror: stop
admin: John Doe
adminmail: root@localhost
mailto : both
balancezero: -0.5
gracedelay : 7
poorman : 1.0
poorwarn : Your Print Quota account balance is low.
 Soon you'll not be allowed to print anymore.
softwarn: Your Print Quota Soft Limit is reached.
 This means that you may still be allowed to print for some
 time, but you must contact your administrator to purchase
 more print quota.
hardwarn: Your Print Quota Hard Limit is reached.
 This means that you are not allowed to print anymore.
 Please contact your administrator at root@localhost
 as soon as possible to solve the problem.
policy: external(/usr/local/bin/pkusers --add --limitby balance --skipexisting  %(username)s > /dev/null &&  /usr/local/bin/edpykota --add --skipexisting -n %(username)s)
maxdenybanners: 0
enforcement : strict
trustjobsize : yes
denyduplicates : no
duplicatesdelay : 0
noprintingmaxdelay : 60
statusstabilizationloops : 5
statusstabilizationdelay : 4.0
snmperrormask : 0x4FCC
*/

plkst.

piektdiena, 2012. gada 26. oktobris

Veidojam LTSP ar rdesktop klientu


Būvējam LTSP tīkla ielādi priekš rdesktop

Izmantots:

  • OpenVZ serveris priekš tīkla attēla
  • Kaut kāds PXE spējīgs dators
  • Kaut kāds Windows 2008R2 terminālserveris


  1. Uzliekam OpenVZ Ubuntu 12.04 konteineri +tīkls u.t.t.
  2. Novācam visu nost no OpenVZ konteinera (saslauth, apache, sendmail, bind u.t.t.)
  3. Apdeitojam, apgreidojam
  4. Lejupielādējam ltsp-server pakotni (lai nebūtu jāliek ~600 MB līdzi nākošās ieteiktās pakas)
    apt-cache show ltsp-server | grep Filename (???)
    wget http://archive.ubuntu.com/ubuntu/pool/main/l/ltsp/ltsp-server_5.3.7-0ubuntu2.2_all.deb
    dpkg --unpack ltsp-server_5.3.7-0ubuntu2.2_all.deb
    ltsp-build-image (izmet kļūdu)
    apt-get -f install (pienstalē debootstrap nbd-server liblzo2-2 squashfs-tools  tcpd ltsp-server libc-bin )
    apt-get install tftpd-hpa
  5. Uztaisa klientu
    ltsp-build-client 
  6. Apdeito, apgreido
    ltsp-chroot
    apt-get update && apt-get upgrade -y --force-yes && apt-get clean
    apt-get install ltspfs
    exit
  7. Pievieno rdesktop-1.7.1 - tas nepieciešams lai Windows2008R2 varētu licencēt "Per Device" modē.
    ltsp-chroot
    wget http://launchpadlibrarian.net/103630514/rdesktop_1.7.1-1ubuntu1_i386.deb
    dpkg -i rdesktop_1.7.1-1ubuntu1_i386.deb
    apt-get -f install
    dpkg -i rdesktop_1.7.1-1ubuntu1_i386.deb
    exit
  8. Atjaunojam ielādes attēlu
    ltsp-update-sshkeys
    ltsp-update-kernels
    ltsp-update-image
    service nbd restart 
  9. Veidojam /var/lib/tftpboot/ltsp/i386/pxelinux.cfg failu
  10. Veidojam /var/lib/tftpboot/ltsp/i386/lts.conf ar saturu:
    [default]
    SEARCH_DOMAIN = skola.venta.lv || Change this for your own network
    DNS_SERVER = 10.0.0.50  || Change this for your own network
    LOCALDEV=True
    SOUND=True
    XKBLAYOUT=en
    SCREEN_03=shell
    SCREEN_07="rdesktop -a 24 -x b -f -k en-us -d SKOLA  -r sound:local -r disk:Drives=/media/root w2k8r2RDS"
  11. Startējam klientu - uz 7 konsoles jābūt ielogošanās ekrānam

Kas nav pārbaudīts:
  • vai strādā USB flashi
  • vai strādā skaņa
  • vai nerāda melnu peles kursoru
Kāpēc gan ikonas uz Desktopa var atvērt ar vienu klikšķi?!
plkst.
Etiķetes: , , ,

trešdiena, 2012. gada 22. augusts

Pilnīgā bremze ar LVM nospiedumiem

Nesekmīgi cīnos ar RAID1, LVM sējumiem un nospiedumiem.
Tātad ir sataisīti 4 gab. mdadm RAID1 sējumi (katrs no diviem 2TB diskiem pie SRC16HI kontroliera)  un uz tiem tiek taisīts LVM sējums ar 4 straipiem (opcija pie veidošanas " lvcreate ... -i 4 ... "). Ja šādam sējumam tiek izveidots  nospiedums, vai divi, tad disku ātrums būtiski krīt - taisot rsync esošiem datiem, average IO access time pat reizēm sasniedz 80 sek. un rsync nomirst ar hung task.
Tātad sējums BKPA:(
lvs -o +devices
  LV          VG   Attr     LSize   Pool Origin Data%  Move Log Copy%  Convert Devices                                                                                   
  bkpa        data owi-aos-   2,00t                                            /dev/md126(196608),/dev/md124(196608),/dev/md125(196608),/dev/md127(196608)


[root@bfsa clilin]# tiotest -f 2000 -t 4 -d /data2/ -k 1 -k 3
Tiotest results for 4 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write        8000 MBs |   64.1 s | 124.868 MB/s |   4.8 %  | 159.7 % |
| Read         8000 MBs |   25.4 s | 314.513 MB/s |   7.1 %  | 125.1 % |
`----------------------------------------------------------------------'
Ja izveidojam nospiedumu: 
lvcreate -s /dev/data/bkpa -n aa -L 100G -i 4
...
lvs -o +devices
  LV          VG   Attr     LSize   Pool Origin Data%  Move Log Copy%  Convert Devices                                                                    
  aa          data swi-a-s- 100,00g      bkpa     7,84                         /dev/md126(327680),/dev/md124(327680),/dev/md125(327680),/dev/md127(327680)
...
Tad tiotest ātrums ir sekojošs: 
tiotest -f 2000 -t 4 -d /data2/ -k 1 -k 3
Tiotest results for 4 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write        8000 MBs |  652.6 s |  12.259 MB/s |   0.3 %  |  54.0 % |
| Read         8000 MBs |   28.8 s | 278.213 MB/s |   7.1 %  | 336.3 % |
`----------------------------------------------------------------------'
Pievienojot vēl vienu nospiedumu, ātrums vēl vairāk nokrīt. Adaptera kontroliera parametu mainīšana (WT-WB, Direct-Cached, EnDskCache-DisDiskCache) neko neuzlaboja. Šie rezultāti ir pie sekojošiem rādītājiem:
CmdTool2 -ldinfo -lall -aall
...
Current Cache Policy: WriteBack, ReadAheadNone, Direct, No Write Cache if Bad BBU
..
Disk Cache Policy   : Enabled
..

Darbojos uz Centos 6.3, jo Ubuntu 12.04 vispār pie slodzes kontrolierim diski aizgāja offlainā.
plkst.

trešdiena, 2012. gada 8. augusts

NFS OpenVZ uz Debian

Kā piedabūt darboties NFS sējumus uz OpenVZ klientiem. 

Darbs notiek uz Debian saimnieka (kodols 2.6.32-5-openvz-amd64) - NFS klients darbojas konteinerī arī ar šo Debian kodolu, bet, ja nepieciešams konteinerī startēt arī NFS-kernel-server, tad saimnieksistēmā jālieto RHEL openvz kodols ( http://forum.openvz.org/index.php?t=msg&goto=43097 ).
Veidojot virtuālo mašīnu, gan jānorāda opcijas, ka klients izmantos NFS (ja to nedara, tad klientā tiek saņemts ziņojums 
# mount -t nfs SERVER:/DIR /mnt
mount.nfs: No such device
): 
vzctl create 4001 --ostemplate debian-6.0-x86 --hostname ovz-4001-nfstest
vzctl set 4001 --features nfs:on --save
vzctl set 4001 --ipadd 10.1.4.1 --save
vzctl start 4001
vzctl enter 4001

Nākamās grūtības sagādāja nfs-common uzstādīšana klientā. Instalējot pēc noklusējuma: 
apt-get install nfs-common

tika saņemti kļūdas ziņojumi 
Setting up nfs-common (1:1.2.2-4squeeze2) ...
insserv: Service portmap has to be enabled to start service nfs-common
insserv: exiting now!
update-rc.d: error: insserv rejected the script header
dpkg: error processing nfs-common (--configure):
 subprocess installed post-installation script returned error exit status 1
configured to not write apport reports
                                      Errors were encountered while processing:
 nfs-common
E: Sub-process /usr/bin/dpkg returned an error code (1)

Ja tika papildus pieinstalēts un startēts portmap serviss kā arī manuāli - /sbin/rpc.statd, tad NFS klients darbojās (mount -t nfs IP:/home/nfsout /mnt), bet pēc pārstartēšanas rpc.statd ir jāstartē ar roku.
Mūsdienās gan portmap vietā gan izmantojot rpcbind
nfs-common uzstādīšanu atrisināja sekojošā secība:
apt-get remove --purge portmap
apt-get remove --purge rpcbind

apt-get remove --purge nfs-common
apt-get install portmap
apt-get install nfs-common


Pēc tam no klienta varēja piemontēt NFS sējumus.











plkst.






















Etiķetes:

















          

        

          

        

piektdiena, 2012. gada 3. augusts


          

        









RAIDi un LVMi










Testējamies un meklējam optimālos piegājienus darbam ar RAID'iem.



DOTS:

Serveris ar SRCSASPH16i adapteri, 8 diski SATA diski  2TB 7200rpm 32MB Cache.



Kā labāk lietot - RAID6, RAID10, vai 4 x RAID1 un uzlikt LVM. Kā vienmēr  ir gan bojājumpiecietība, gan ātrums.

Galu galā nosliecos uz 4 RAID1 sējumiem, kas katrs ir no diviem diskiem, un tam virsū - LVM ar straipu. Domāju, ka kāda atsevišķa diska nobrukšanas gadījumā, daudz vieglāk un ātrāk būs ar LVM līdzekļiem norādīt kurus masīvus izmantot, nekā cerēt ka rezerves disks sinhronizēsies RAID10 vai RAID6 gadījumos.



Diski RAID kontrolierī konfigurēti kā atsevišķi JBOD, ko OS rāda kā:

/dev/sd{e,f,g,h,m,n,o,p}





1. RAID10



mdadm --create --verbose /dev/md6 --level=raid6  --raid-devices=8 /dev/sd{e,f,g,h,m,n,o,p}

Gaidam, kad pabeidzas sinhronizācija (>10h)

mkfs.ext4 /dev/md6

mount /dev/md6 /mnt/



TIOTEST skritps (CentOS6.3 tiobench kārās ar kļūdas ziņojumu par dalīšanu ar nulli)




for i in 1 2 4 8 ; do echo ================================= ; echo THREADS  $i ; echo SIZE PER THREAD $((8000/$i)) ; tiotest -d /mnt/ -f $((8000/$i))  -t $i  ; done



=================================
THREADS 1
SIZE PER THREAD 8000
Tiotest results for 1 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write        8000 MBs |   46.2 s | 173.178 MB/s |   1.3 %  |  32.2 % |
| Random Write    4 MBs |    2.5 s |   1.536 MB/s |   0.0 %  |   0.9 % |
| Read         8000 MBs |   10.3 s | 774.033 MB/s |   4.1 %  |  59.6 % |
| Random Read     4 MBs |    3.6 s |   1.090 MB/s |   0.0 %  |   0.2 % |
`----------------------------------------------------------------------'
=================================
THREADS 2
SIZE PER THREAD 4000
Tiotest results for 2 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write        8000 MBs |   62.0 s | 129.005 MB/s |   2.0 %  |  70.6 % |
| Random Write    8 MBs |    5.5 s |   1.415 MB/s |   0.0 %  |   0.4 % |
| Read         8000 MBs |   15.9 s | 504.067 MB/s |   5.5 %  |  76.5 % |
| Random Read     8 MBs |    4.0 s |   1.933 MB/s |   0.1 %  |   0.0 % |
`----------------------------------------------------------------------'
=================================
THREADS 4
SIZE PER THREAD 2000
Tiotest results for 4 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write        8000 MBs |  158.1 s |  50.591 MB/s |   2.6 %  | 136.7 % |
| Random Write   16 MBs |   11.7 s |   1.333 MB/s |   0.1 %  |   0.0 % |
| Read         8000 MBs |   16.5 s | 484.436 MB/s |   9.8 %  | 143.8 % |
| Random Read    16 MBs |    4.6 s |   3.421 MB/s |   0.0 %  |   0.0 % |
`----------------------------------------------------------------------'

=================================
THREADS 8
SIZE PER THREAD 1000
Tiotest results for 8 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write        8000 MBs |  208.1 s |  38.449 MB/s |   4.4 %  | 351.5 % |
| Random Write   31 MBs |   21.5 s |   1.454 MB/s |   0.2 %  |   9.2 % |
| Read         8000 MBs |   16.3 s | 491.984 MB/s |  21.2 %  | 284.4 % |
| Random Read    31 MBs |    5.5 s |   5.714 MB/s |   0.4 %  |   0.0 % |
`----------------------------------------------------------------------'

========
========
========

time  for D in `seq 1000 1999` ; do echo $D ; mkdir $D ; for F in `seq 1000 1234 1000000` ; do echo $D $F ; dd if=/dev/zero  bs=$F count=1 of=/mnt/A/$D/$F.txt ; done ; done
real    50m29.980s
user    4m2.111s
sys     36m16.997s

========
========
========


rsync -a --stats /mnt/A /mnt/C
..
sent 405221629673 bytes  received 15394018 bytes  52844366.39 bytes/sec
..

========
========
========

time cp -al /mnt/A /mnt/B 

real    1m0.912s
user    0m3.396s
sys     0m43.982s

========
========
========

time rm -rf /mnt/A  ; time rm -rf /mnt/B ; time rm -rf /mnt/C

real    0m49.380s
user    0m0.330s
sys     0m8.737s

real    0m59.611s
user    0m0.517s
sys     0m27.437s

real    1m15.621s
user    0m0.545s
sys     0m35.994s
=========================





RAID 10 

Sagatavojam:




umount /mnt
mdadm -S /dev/md6
mdadm --remove /dev/md6
mdadm --zero-superblock /dev/sd{e,f,g,h,m,n,o,p}
mdadm --create --verbose /dev/md6 --level=raid10  --raid-devices=8 /dev/sd{e,f,g,h,m,n,o,p}
## Gaidām, kad sinhronizējas
mkfs.ext4 /dev/md6
mount /dev/md6 /mnt




Testējam:






=================================
THREADS 1
SIZE PER THREAD 8000
Tiotest results for 1 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write        8000 MBs |   56.3 s | 142.088 MB/s |   1.0 %  |  26.9 % |
| Random Write    4 MBs |    0.2 s |  18.616 MB/s |   0.5 %  |   6.7 % |
| Read         8000 MBs |   17.2 s | 466.319 MB/s |   2.7 %  |  37.4 % |
| Random Read     4 MBs |    4.0 s |   0.976 MB/s |   0.0 %  |   0.2 % |
`----------------------------------------------------------------------'
=================================
THREADS 2
SIZE PER THREAD 4000
Tiotest results for 2 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write        8000 MBs |   54.0 s | 148.037 MB/s |   2.4 %  |  67.7 % |
| Random Write    8 MBs |    0.7 s |  11.882 MB/s |   0.3 %  |   0.0 % |
| Read         8000 MBs |   14.8 s | 540.108 MB/s |   5.5 %  |  88.9 % |
| Random Read     8 MBs |    4.2 s |   1.882 MB/s |   0.0 %  |   0.0 % |
`----------------------------------------------------------------------'
=================================
THREADS 4
SIZE PER THREAD 2000
Tiotest results for 4 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write        8000 MBs |   54.2 s | 147.615 MB/s |   8.0 %  | 188.9 % |
| Random Write   16 MBs |    3.5 s |   4.501 MB/s |   0.5 %  |   0.0 % |
| Read         8000 MBs |   16.1 s | 496.289 MB/s |   7.2 %  | 158.6 % |
| Random Read    16 MBs |    4.8 s |   3.236 MB/s |   0.0 %  |   0.0 % |
`----------------------------------------------------------------------'
=================================
THREADS 8
SIZE PER THREAD 1000
Tiotest results for 8 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write        8000 MBs |   56.7 s | 141.097 MB/s |  20.4 %  | 490.5 % |
| Random Write   31 MBs |    5.6 s |   5.541 MB/s |   1.3 %  |   0.0 % |
| Read         8000 MBs |   16.3 s | 492.223 MB/s |  12.2 %  | 301.0 % |
| Random Read    31 MBs |    5.6 s |   5.559 MB/s |   1.4 %  |   0.0 % |
`----------------------------------------------------------------------'

Failu veidošana:
========
========
========

time  for D in `seq 1000 1999` ; do echo $D ; mkdir $D ; for F in `seq 1000 1234 1000000` ; do echo $D $F ; dd if=/dev/zero  bs=$F count=1 of=/mnt/A/$D/$F.txt ; done ; done

real    42m58.039s
user    3m52.933s
sys     26m18.912s

========
========
========
rsync -a --stats /mnt/A /mnt/C
...
sent 405221480087 bytes  received 15394018 bytes  51606096.67 bytes/sec
...

========
========
========
time cp -al /mnt/A /mnt/B 

real    1m21.329s
user    0m3.292s
sys     1m8.058s

========
========
========

for i in A B C ; do time rm -rf /mnt/$i ; done 

real    0m12.508s
user    0m0.362s
sys     0m7.445s

real    0m36.581s
user    0m0.463s
sys     0m25.834s

real    1m1.690s
user    0m0.570s
sys     0m47.764s






LVM uz RAID1 masīviem:


mdadm -S /dev/md6
mdadm --remove /dev/md6
mdadm --zero-superblock /dev/sd{e,f,g,h,m,n,o,p}
mdadm --create --verbose /dev/md11 --level=raid1 --raid-devices=2 /dev/sd{e,m}
mdadm --create --verbose /dev/md12 --level=raid1 --raid-devices=2 /dev/sd{f,n}
mdadm --create --verbose /dev/md13 --level=raid1 --raid-devices=2 /dev/sd{g,o}
mdadm --create --verbose /dev/md14 --level=raid1 --raid-devices=2 /dev/sd{h,p}



Izveidojam PV un VG:

pvcreate /dev/md11
pvcreate /dev/md12
pvcreate /dev/md13
pvcreate /dev/md14

vgcreate test /dev/md11 /dev/md12 /dev/md13 /dev/md14



Izveidojam 3 LV sējumus - bez un ar straipiem:

lvcreate      -L1T -n tests test
lvcreate -i 3 -L1T -n tests3 test
lvcreate -i 4 -L1T -n tests4 test



Testējam:

Bez straipa:

mkfs.ext4 /dev/test/tests 

real    8m52.703s
user    0m1.248s
sys     0m17.725s


Unit information
================
File size = megabytes
Blk Size  = bytes
Rate      = megabytes per second
CPU%      = percentage of CPU used during the test
Latency   = milliseconds
Lat%      = percent of requests that took longer than X seconds
CPU Eff   = Rate divided by CPU% - throughput per cpu load

Sequential Reads
2.6.32-279.2.1.el6.x86_64     8000  4096    1  133.91 12.61%     0.029     1308.61   0.00000  0.00000  1062
2.6.32-279.2.1.el6.x86_64     8000  4096    2  307.13 60.66%     0.025     1069.12   0.00000  0.00000   506
2.6.32-279.2.1.el6.x86_64     8000  4096    4  178.61 58.89%     0.080     1439.95   0.00000  0.00000   303
2.6.32-279.2.1.el6.x86_64     8000  4096    8  159.53 82.49%     0.159     2918.88   0.00005  0.00000   193

Random Reads
2.6.32-279.2.1.el6.x86_64     8000  4096    1    0.96 0.263%     4.076       17.82   0.00000  0.00000   363
2.6.32-279.2.1.el6.x86_64     8000  4096    2    1.94 0.074%     4.013       19.58   0.00000  0.00000  2604
2.6.32-279.2.1.el6.x86_64     8000  4096    4    2.31 0.340%     6.088       89.15   0.00000  0.00000   679
2.6.32-279.2.1.el6.x86_64     8000  4096    8    2.64 0.135%     8.743      144.99   0.00000  0.00000  1953

Sequential Writes
2.6.32-279.2.1.el6.x86_64     8000  4096    1   38.24 7.624%     0.095     1826.67   0.00000  0.00000   502
2.6.32-279.2.1.el6.x86_64     8000  4096    2   42.03 22.86%     0.172     8895.27   0.00093  0.00000   184
2.6.32-279.2.1.el6.x86_64     8000  4096    4   50.89 84.20%     0.268    12996.51   0.00200  0.00034    60
2.6.32-279.2.1.el6.x86_64     8000  4096    8   49.56 254.4%     0.541    18589.47   0.00474  0.00142    19

Random Writes
2.6.32-279.2.1.el6.x86_64     8000  4096    1    1.29 0.733%     0.005        0.03   0.00000  0.00000   176
2.6.32-279.2.1.el6.x86_64     8000  4096    2    1.44 0.312%     0.007        0.04   0.00000  0.00000   460
2.6.32-279.2.1.el6.x86_64     8000  4096    4    1.27 0.146%     0.009        0.05   0.00000  0.00000   868
2.6.32-279.2.1.el6.x86_64     8000  4096    8    1.46 0.223%     0.012       10.15   0.00000  0.00000   651



3 straipi:

time  mkfs.ext4 /dev/test/tests3

real    3m19.238s
user    0m1.241s
sys     0m17.049s


Unit information
================
File size = megabytes
Blk Size  = bytes
Rate      = megabytes per second
CPU%      = percentage of CPU used during the test
Latency   = milliseconds
Lat%      = percent of requests that took longer than X seconds
CPU Eff   = Rate divided by CPU% - throughput per cpu load

Sequential Reads
2.6.32-279.2.1.el6.x86_64     8000  4096    1  287.61 28.51%     0.013      472.71   0.00000  0.00000  1009
2.6.32-279.2.1.el6.x86_64     8000  4096    2  292.54 61.72%     0.026      788.40   0.00000  0.00000   474
2.6.32-279.2.1.el6.x86_64     8000  4096    4  245.74 101.4%     0.063     1150.85   0.00000  0.00000   242
2.6.32-279.2.1.el6.x86_64     8000  4096    8  239.19 195.8%     0.125      597.14   0.00000  0.00000   122

Random Reads
2.6.32-279.2.1.el6.x86_64     8000  4096    1    1.04 0.434%     3.742       35.00   0.00000  0.00000   240
2.6.32-279.2.1.el6.x86_64     8000  4096    2    2.06 1.239%     3.668       30.53   0.00000  0.00000   166
2.6.32-279.2.1.el6.x86_64     8000  4096    4    3.68 0.895%     4.156      156.84   0.00000  0.00000   411
2.6.32-279.2.1.el6.x86_64     8000  4096    8    5.31 0.271%     5.207       60.89   0.00000  0.00000  1953

Sequential Writes
2.6.32-279.2.1.el6.x86_64     8000  4096    1  108.41 21.86%     0.033     2525.84   0.00015  0.00000   496
2.6.32-279.2.1.el6.x86_64     8000  4096    2   97.02 45.85%     0.073     3191.88   0.00015  0.00000   212
2.6.32-279.2.1.el6.x86_64     8000  4096    4  104.68 125.0%     0.134     6294.07   0.00103  0.00000    84
2.6.32-279.2.1.el6.x86_64     8000  4096    8   96.39 268.3%     0.272     9232.73   0.00229  0.00000    36

Random Writes
2.6.32-279.2.1.el6.x86_64     8000  4096    1    5.35 2.907%     0.005        0.03   0.00000  0.00000   184
2.6.32-279.2.1.el6.x86_64     8000  4096    2    5.31 5.636%     0.007        0.05   0.00000  0.00000    94
2.6.32-279.2.1.el6.x86_64     8000  4096    4    4.54 3.838%     0.009        0.06   0.00000  0.00000   118
2.6.32-279.2.1.el6.x86_64     8000  4096    8    5.40 1.106%     0.011        6.28   0.00000  0.00000   488



4 straipi

[root@bfsa ~]# cat t.4
mkfs.ext4 /dev/test/tests4

real    2m28.422s
user    0m1.211s
sys     0m17.627s


Unit information
================
File size = megabytes
Blk Size  = bytes
Rate      = megabytes per second
CPU%      = percentage of CPU used during the test
Latency   = milliseconds
Lat%      = percent of requests that took longer than X seconds
CPU Eff   = Rate divided by CPU% - throughput per cpu load

Sequential Reads
2.6.32-279.2.1.el6.x86_64     8000  4096    1  352.56 34.89%     0.011      361.91   0.00000  0.00000  1010
2.6.32-279.2.1.el6.x86_64     8000  4096    2  300.66 61.63%     0.025      550.33   0.00000  0.00000   488
2.6.32-279.2.1.el6.x86_64     8000  4096    4  297.59 121.8%     0.052      594.57   0.00000  0.00000   244
2.6.32-279.2.1.el6.x86_64     8000  4096    8  297.59 242.5%     0.103      498.79   0.00000  0.00000   123

Random Reads
2.6.32-279.2.1.el6.x86_64     8000  4096    1    1.07 0.431%     3.651       37.60   0.00000  0.00000   248
2.6.32-279.2.1.el6.x86_64     8000  4096    2    2.02 0.710%     3.767       31.34   0.00000  0.00000   284
2.6.32-279.2.1.el6.x86_64     8000  4096    4    3.77 0.289%     3.991       38.33   0.00000  0.00000  1302
2.6.32-279.2.1.el6.x86_64     8000  4096    8    5.74 1.614%     5.079       68.38   0.00000  0.00000   355

Sequential Writes
2.6.32-279.2.1.el6.x86_64     8000  4096    1  140.81 28.79%     0.026     1226.49   0.00000  0.00000   489
2.6.32-279.2.1.el6.x86_64     8000  4096    2  160.22 76.74%     0.045     1535.99   0.00000  0.00000   209
2.6.32-279.2.1.el6.x86_64     8000  4096    4  160.26 208.7%     0.087     3242.12   0.00068  0.00000    77
2.6.32-279.2.1.el6.x86_64     8000  4096    8  143.09 462.8%     0.179     5060.46   0.00161  0.00000    31

Random Writes
2.6.32-279.2.1.el6.x86_64     8000  4096    1    7.26 3.252%     0.005        0.04   0.00000  0.00000   223
2.6.32-279.2.1.el6.x86_64     8000  4096    2    6.93 2.839%     0.007        0.04   0.00000  0.00000   244
2.6.32-279.2.1.el6.x86_64     8000  4096    4    7.16 0.549%     0.009        0.05   0.00000  0.00000  1302
2.6.32-279.2.1.el6.x86_64     8000  4096    8    5.77 0.590%     0.008        0.06   0.00000  0.00000   977



Failu veidošanas tests (veidojam 2x mazāk nekā iepriekšējos testos!):

for i in 4 3 1 ; do echo  ===== VEIDOJAM FAILUS $i ==== ; time  for D in `seq 1000 1499` ; do  mkdir /mnt/$i/A/$D ; for F in `seq 1000 1234 1000000` ; do  dd  status=noxfer if=/dev/zero  bs=$F count=1 of=/mnt/$i/A/$D/$F.txt 2>/dev/null ; done ; done ; done

===== VEIDOJAM FAILUS 4 (split 4) ====
real    22m9.756s
user    1m55.964s
sys     14m11.965s

===== VEIDOJAM FAILUS 3 (split 3) ====
real    30m37.752s
user    1m48.219s
sys     13m24.232s

===== VEIDOJAM FAILUS 1 (no split) ====
real    78m11.036s
user    1m45.851s
sys     12m59.008s



OK - šķiet LVM ar 3 vai 4 straipiem derētu.

Pašreizējais sējumu izkārtojums:

lvs -o +seg_pe_ranges --segments 
  LV     VG   Attr     #Str Type    SSize PE Ranges                                                                              
  tests  test -wi-a---    1 linear  1,00t /dev/md11:0-262143                                                                     
  tests3 test -wi-a---    3 striped 1,00t /dev/md11:262144-349525 /dev/md12:0-87381 /dev/md13:0-87381                            
  tests4 test -wi-a---    4 striped 1,00t /dev/md11:349526-415061 /dev/md12:87382-152917 /dev/md13:87382-152917 /dev/md14:0-65535



Tātad provējam no VG izņemt vienu masīvu:



# vgreduce -d -v test /dev/md14
    Finding volume group "test"
    Using physical volume(s) on command line
  Physical volume "/dev/md14" still in use

:(



Skaidrs, seejums test/tests4 lieto /dev/md14

Provējam pārvietot:

# pvmove -v /dev/md14
    Finding volume group "test"
    Archiving volume group "test" metadata (seqno 4).
    Creating logical volume pvmove0
    Moving 65536 extents of logical volume test/tests4
  Insufficient suitable allocatable extents for logical volume : 65536 more required
  Unable to allocate mirror extents for pvmove0.
  Failed to convert pvmove LV to mirrored



Spriežot pēc visa, tā ir nevis kļūda, ka vairākus stripus nevar uzlikt uz viena PV, bet gan fīča, kas izlabota RHEL6/Centos6 ( https://bugzilla.redhat.com/show_bug.cgi?id=580155 )

Tātad ir jāmēģina samazināt LV sējumam tests4 straipus no 4 uz 3:


## Uzzinam lielumu extentos
lvdisplay /dev/test/tests4 | grep LE
  Current LE             262144

## Mainam stripu skaitu, saglabājot sējuma lielumu ?varbūt var uzreiz norādīt izmantojamos PV??
time lvextend -v -i 3 -l 262144 /dev/test/tests4 
    Finding volume group test
  New size (262144 extents) matches existing size (262144 extents)
  Run `lvextend --help' for more information.

real    0m2.186s
user    0m0.127s
sys     0m0.024s
[root@bfsa ~]# time lvextend -v -i 3 -l 262146 /dev/test/tests4 
    Finding volume group test
  Using stripesize of last segment 64,00 KiB
  Rounding size (262146 extents) up to stripe boundary size for segment (262147 extents)
    Archiving volume group "test" metadata (seqno 4).
  Extending logical volume tests4 to 1,00 TiB
    Found volume group "test"
    Found volume group "test"
    Loading test-tests4 table (253:2)
    Suspending test-tests4 (253:2) with device flush
    Found volume group "test"
    Resuming test-tests4 (253:2)
    Creating volume group backup "/etc/lvm/backup/test" (seqno 5).
  Logical volume tests4 successfully resized

real    0m2.626s
user    0m0.133s
sys     0m0.047s



Provējam pārvietot disku

pvmove -v /dev/md14 /dev/md13
    Finding volume group "test"
    Archiving volume group "test" metadata (seqno 5).
    Creating logical volume pvmove0
    Moving 65536 extents of logical volume test/tests4
  Insufficient suitable allocatable extents for logical volume : 65536 more required
  Unable to allocate mirror extents for pvmove0.
  Failed to convert pvmove LV to mirrored


:(
# Skatamies, kas notiek:
lvs -o +seg_pe_ranges --segments
  LV     VG   Attr     #Str Type    SSize  PE Ranges                                                                              
  tests  test -wi-a---    1 linear   1,00t /dev/md11:0-262143                                                                     
  tests3 test -wi-a---    3 striped  1,00t /dev/md11:262144-349525 /dev/md12:0-87381 /dev/md13:0-87381                            
  tests4 test -wi-a---    4 striped  1,00t /dev/md11:349526-415061 /dev/md12:87382-152917 /dev/md13:87382-152917 /dev/md14:0-65535
  tests4 test -wi-a---    3 striped 12,00m /dev/md11:415062-415062 /dev/md12:152918-152918 /dev/md13:152918-152918                



















plkst.






































        

      









Abonēt:
Ziņas (Atom)