ceturtdiena, 2013. gada 8. augusts

MegaRAID SAS 2108 - HW RAID 10 (2 + 2 + 2)


Base Board Information
        Manufacturer: Supermicro
        Product Name: X9DRi-LN4+/X9DR3-LN4+
q
RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS 2108 [Liberator] (rev 05)
SAS 9260CV-8i
Sistēma startēta ar parametru "mem = 1G"
FS - EXT4

HW RAID 10 

2 + 2 + 2

RWTD

storcli /c0/v0 show 

----------------------------------------------------------
DG/VD TYPE   State Access Consist Cache sCC     Size Name 
----------------------------------------------------------
0/0   RAID10 Optl  RW     No      RWTD  -   5.454 TB      
----------------------------------------------------------
root@ubuntu:~# tiobench --size 2000 --identifier R10-RWTD --numruns 3 
Run #3: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT
Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD                      2000  4096    1  164.01 10.61%     0.071      110.05   0.00000  0.00000  1545
R10-RWTD                      2000  4096    2  136.11 16.77%     0.168      149.97   0.00000  0.00000   811
R10-RWTD                      2000  4096    4  127.26 36.09%     0.346      310.12   0.00000  0.00000   353
R10-RWTD                      2000  4096    8  143.80 82.40%     0.600      512.41   0.00000  0.00000   174

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD                      2000  4096    1    1.05 0.510%    11.167      103.91   0.00000  0.00000   206
R10-RWTD                      2000  4096    2    1.86 3.086%    10.919      100.91   0.00000  0.00000    60
R10-RWTD                      2000  4096    4    3.40 6.955%    11.365      119.16   0.00000  0.00000    49
R10-RWTD                      2000  4096    8    5.30 20.34%    13.479      241.22   0.00000  0.00000    26

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD                      2000  4096    1  244.78 33.16%     0.046      963.48   0.00000  0.00000   738
R10-RWTD                      2000  4096    2  199.02 62.89%     0.112      758.66   0.00000  0.00000   316
R10-RWTD                      2000  4096    4  186.14 117.4%     0.241      560.05   0.00000  0.00000   159
R10-RWTD                      2000  4096    8  177.05 216.8%     0.508      711.64   0.00000  0.00000    82

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD                      2000  4096    1    8.06     0%     0.009        0.06   0.00000  0.00000     0
R10-RWTD                      2000  4096    2    7.78     0%     0.009        0.04   0.00000  0.00000     0
R10-RWTD                      2000  4096    4    7.82     0%     0.010        0.08   0.00000  0.00000     0
R10-RWTD                      2000  4096    8    7.90 9.166%     0.017        0.09   0.00000  0.00000    86

RWTC

storcli /c0/v0 set iopolicy=cached
storcli /c0/v0 show

----------------------------------------------------------
DG/VD TYPE   State Access Consist Cache sCC     Size Name 
----------------------------------------------------------
0/0   RAID10 Optl  RW     No      RWTC  -   5.454 TB      
----------------------------------------------------------
root@ubuntu:~# tiobench --size 2000 --identifier R10-RWTC --numruns 3 
Run #3: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1  169.42 10.76%     0.069      108.65   0.00000  0.00000  1574
R10-RWTC                      2000  4096    2  139.71 17.76%     0.165      162.62   0.00000  0.00000   787
R10-RWTC                      2000  4096    4  133.68 37.36%     0.325      237.25   0.00000  0.00000   358
R10-RWTC                      2000  4096    8  132.39 75.94%     0.645      637.03   0.00000  0.00000   174

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1    1.04 0.694%    11.234       84.40   0.00000  0.00000   150
R10-RWTC                      2000  4096    2    1.98 3.624%    11.001       82.62   0.00000  0.00000    55
R10-RWTC                      2000  4096    4    3.36 5.441%    11.392      165.62   0.00000  0.00000    62
R10-RWTC                      2000  4096    8    5.25 19.31%    13.533      180.47   0.00000  0.00000    27

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1  247.02 30.99%     0.045      322.56   0.00000  0.00000   797
R10-RWTC                      2000  4096    2  203.29 67.52%     0.110      563.23   0.00000  0.00000   301
R10-RWTC                      2000  4096    4  185.36 116.4%     0.241      526.25   0.00000  0.00000   159
R10-RWTC                      2000  4096    8  177.23 225.7%     0.506      616.67   0.00000  0.00000    78

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1    7.41     0%     0.009        0.06   0.00000  0.00000     0
R10-RWTC                      2000  4096    2    7.98     0%     0.009        0.05   0.00000  0.00000     0
R10-RWTC                      2000  4096    4    7.76 1.059%     0.010        0.08   0.00000  0.00000   732
R10-RWTC                      2000  4096    8    7.92 8.446%     0.017        0.07   0.00000  0.00000    94

RWTC - Disk cache enabled

tiobench --size 2000 --identifier R10-RWTC 
Run #1: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1  222.69 15.58%     0.017       24.78   0.00000  0.00000  1428
R10-RWTC                      2000  4096    2  126.77 15.46%     0.061       58.48   0.00000  0.00000   820
R10-RWTC                      2000  4096    4  162.10 44.19%     0.087       66.98   0.00000  0.00000   367
R10-RWTC                      2000  4096    8  126.15 72.59%     0.230      208.33   0.00000  0.00000   174

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1    1.09 0.694%     3.596       27.37   0.00000  0.00000   156
R10-RWTC                      2000  4096    2    2.01 1.441%     3.693       25.86   0.00000  0.00000   139
R10-RWTC                      2000  4096    4    2.94 3.841%     3.876       28.29   0.00000  0.00000    77
R10-RWTC                      2000  4096    8    5.25 17.22%     4.559       70.20   0.00000  0.00000    31

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1  281.20 36.22%     0.013      174.99   0.00000  0.00000   776
R10-RWTC                      2000  4096    2  199.49 64.79%     0.037      171.67   0.00000  0.00000   308
R10-RWTC                      2000  4096    4  188.08 119.7%     0.079      175.89   0.00000  0.00000   157
R10-RWTC                      2000  4096    8  176.24 208.0%     0.169      256.70   0.00000  0.00000    85

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC                      2000  4096    1    8.10     0%     0.003        0.03   0.00000  0.00000     0
R10-RWTC                      2000  4096    2    7.89     0%     0.003        0.02   0.00000  0.00000     0
R10-RWTC                      2000  4096    4    7.79     0%     0.003        0.03   0.00000  0.00000     0
R10-RWTC                      2000  4096    8    7.93 6.496%     0.006        0.03   0.00000  0.00000   122

RWTC - Disk cache disabled

# tiobench --size 2000 --identifier R10-RWTC-PdCacOff 
Run #1: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC-PdCacOff             2000  4096    1  221.14 14.06%     0.018       23.75   0.00000  0.00000  1572
R10-RWTC-PdCacOff             2000  4096    2  122.06 15.99%     0.063       50.35   0.00000  0.00000   763
R10-RWTC-PdCacOff             2000  4096    4  131.95 35.99%     0.108       83.59   0.00000  0.00000   367
R10-RWTC-PdCacOff             2000  4096    8  130.39 73.15%     0.219      183.32   0.00000  0.00000   178

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC-PdCacOff             2000  4096    1    1.11 0.509%     3.532       39.69   0.00000  0.00000   217
R10-RWTC-PdCacOff             2000  4096    2    2.11 3.894%     3.485       26.37   0.00000  0.00000    54
R10-RWTC-PdCacOff             2000  4096    4    2.99 3.209%     3.787       32.58   0.00000  0.00000    93
R10-RWTC-PdCacOff             2000  4096    8    4.89 17.40%     4.410       82.89   0.00000  0.00000    28

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC-PdCacOff             2000  4096    1  218.60 29.25%     0.017      237.86   0.00000  0.00000   747
R10-RWTC-PdCacOff             2000  4096    2  171.45 56.99%     0.043      209.19   0.00000  0.00000   301
R10-RWTC-PdCacOff             2000  4096    4  157.12 102.8%     0.095      250.64   0.00000  0.00000   153
R10-RWTC-PdCacOff             2000  4096    8  138.59 166.9%     0.209      340.25   0.00000  0.00000    83

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTC-PdCacOff             2000  4096    1    6.02     0%     0.003        0.02   0.00000  0.00000     0
R10-RWTC-PdCacOff             2000  4096    2    5.87     0%     0.003        0.02   0.00000  0.00000     0
R10-RWTC-PdCacOff             2000  4096    4    5.98     0%     0.003        0.03   0.00000  0.00000     0
R10-RWTC-PdCacOff             2000  4096    8    5.99 6.137%     0.006        0.05   0.00000  0.00000    98

RWTD - Disk cache disabled

root@ubuntu:~# tiobench --size 2000 --identifier R10-RWTD-PdCacOff 
Run #1: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOff             2000  4096    1  225.59 13.71%     0.017       23.27   0.00000  0.00000  1645
R10-RWTD-PdCacOff             2000  4096    2  128.78 18.03%     0.060       74.44   0.00000  0.00000   714
R10-RWTD-PdCacOff             2000  4096    4  120.24 34.48%     0.125      108.35   0.00000  0.00000   349
R10-RWTD-PdCacOff             2000  4096    8  150.00 78.87%     0.178      183.56   0.00000  0.00000   190

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOff             2000  4096    1    1.07 0.518%     3.662       28.25   0.00000  0.00000   206
R10-RWTD-PdCacOff             2000  4096    2    2.12 3.794%     3.478       32.29   0.00000  0.00000    56
R10-RWTD-PdCacOff             2000  4096    4    3.92 5.926%     3.850       33.38   0.00000  0.00000    66
R10-RWTD-PdCacOff             2000  4096    8    5.40 14.65%     4.195       54.68   0.00000  0.00000    37

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOff             2000  4096    1  211.89 27.29%     0.018      231.49   0.00000  0.00000   776
R10-RWTD-PdCacOff             2000  4096    2  176.41 58.00%     0.042      224.31   0.00000  0.00000   304
R10-RWTD-PdCacOff             2000  4096    4  156.11 97.60%     0.094      261.58   0.00000  0.00000   160
R10-RWTD-PdCacOff             2000  4096    8  137.09 164.4%     0.209      325.76   0.00000  0.00000    83

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOff             2000  4096    1    6.12     0%     0.003        0.02   0.00000  0.00000     0
R10-RWTD-PdCacOff             2000  4096    2    6.24     0%     0.003        0.02   0.00000  0.00000     0
R10-RWTD-PdCacOff             2000  4096    4    6.00     0%     0.003        0.02   0.00000  0.00000     0
R10-RWTD-PdCacOff             2000  4096    8    6.05 4.957%     0.006        0.04   0.00000  0.00000   122

RWTD - Disk cache enabled

tiobench --size 2000 --identifier R10-RWTD-PdCacOn 
Run #1: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOn              2000  4096    1  222.84 14.61%     0.017       20.89   0.00000  0.00000  1524
R10-RWTD-PdCacOn              2000  4096    2  129.22 15.81%     0.060       48.82   0.00000  0.00000   817
R10-RWTD-PdCacOn              2000  4096    4  131.79 38.19%     0.110       83.58   0.00000  0.00000   345
R10-RWTD-PdCacOn              2000  4096    8  141.00 75.24%     0.188      151.67   0.00000  0.00000   187

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOn              2000  4096    1    1.06 0.703%     3.697       27.83   0.00000  0.00000   150
R10-RWTD-PdCacOn              2000  4096    2    1.99 0.970%     3.814       25.99   0.00000  0.00000   206
R10-RWTD-PdCacOn              2000  4096    4    2.88 6.938%     4.034       51.90   0.00000  0.00000    42
R10-RWTD-PdCacOn              2000  4096    8    5.36 14.82%     3.940       54.28   0.00000  0.00000    36

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOn              2000  4096    1  280.53 34.56%     0.013      205.93   0.00000  0.00000   812
R10-RWTD-PdCacOn              2000  4096    2  199.38 62.92%     0.037      165.82   0.00000  0.00000   317
R10-RWTD-PdCacOn              2000  4096    4  191.24 96.84%     0.079      157.06   0.00000  0.00000   197
R10-RWTD-PdCacOn              2000  4096    8  175.74 206.3%     0.170      391.90   0.00000  0.00000    85

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RWTD-PdCacOn              2000  4096    1    7.98 0.204%     0.003        0.01   0.00000  0.00000  3906
R10-RWTD-PdCacOn              2000  4096    2    7.78     0%     0.003        0.01   0.00000  0.00000     0
R10-RWTD-PdCacOn              2000  4096    4    8.24 1.687%     0.003        0.03   0.00000  0.00000   488
R10-RWTD-PdCacOn              2000  4096    8    8.10 11.61%     0.006        0.03   0.00000  0.00000    70

Always Write Back

   storcli /c0/v0 set wrcache=AWB 

RAWBD PdCacOn

storcli /c0/v0 show
----------------------------------------------------------
DG/VD TYPE   State Access Consist Cache sCC     Size Name 
----------------------------------------------------------
0/0   RAID10 Optl  RW     No      RAWBD -   5.454 TB      
----------------------------------------------------------
tiobench --size 2000 --identifier R10-RAWBD-PdCacOn  --numruns 3
Run #3: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Unit information
================
File size = megabytes
Blk Size  = bytes
Rate      = megabytes per second
CPU%      = percentage of CPU used during the test
Latency   = milliseconds
Lat%      = percent of requests that took longer than X seconds
CPU Eff   = Rate divided by CPU% - throughput per cpu load

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBD-PdCacOn             2000  4096    1  146.53 9.671%     0.080     1344.87   0.00000  0.00000  1515
R10-RAWBD-PdCacOn             2000  4096    2  143.45 16.55%     0.158     1364.29   0.00000  0.00000   866
R10-RAWBD-PdCacOn             2000  4096    4  119.89 32.65%     0.362     1940.91   0.00000  0.00000   367
R10-RAWBD-PdCacOn             2000  4096    8  127.14 70.38%     0.668     1781.70   0.00000  0.00000   181

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBD-PdCacOn             2000  4096    1    1.17 0.616%    10.053      117.77   0.00000  0.00000   189
R10-RAWBD-PdCacOn             2000  4096    2    1.94 3.273%    10.045       85.48   0.00000  0.00000    59
R10-RAWBD-PdCacOn             2000  4096    4    3.43 5.976%    10.275      119.88   0.00000  0.00000    57
R10-RAWBD-PdCacOn             2000  4096    8    6.03 18.00%    11.818      185.64   0.00000  0.00000    33

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBD-PdCacOn             2000  4096    1  293.70 42.82%     0.039      252.09   0.00000  0.00000   686
R10-RAWBD-PdCacOn             2000  4096    2  263.84 83.16%     0.086      353.42   0.00000  0.00000   317
R10-RAWBD-PdCacOn             2000  4096    4  275.19 165.3%     0.163      474.69   0.00000  0.00000   166
R10-RAWBD-PdCacOn             2000  4096    8  257.68 351.0%     0.343     1178.33   0.00000  0.00000    73

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBD-PdCacOn             2000  4096    1   14.03     0%     0.009        0.08   0.00000  0.00000     0
R10-RAWBD-PdCacOn             2000  4096    2   15.38 1.312%     0.008        0.04   0.00000  0.00000  1172
R10-RAWBD-PdCacOn             2000  4096    4   14.37     0%     0.010        0.07   0.00000  0.00000     0
R10-RAWBD-PdCacOn             2000  4096    8   18.77 9.612%     0.017        0.10   0.00000  0.00000   195

RAWBC - Disk Cache On

storcli /c0/v0 set iopolicy=cached
storcli /c0/v0 show

----------------------------------------------------------
DG/VD TYPE   State Access Consist Cache sCC     Size Name 
----------------------------------------------------------
0/0   RAID10 Optl  RW     No      RAWBC -   5.454 TB      
----------------------------------------------------------
tiobench --size 2000 --identifier R10-RAWBC-PdCacOn  --numruns 3
Run #3: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOn             2000  4096    1  146.89 9.313%     0.080      986.74   0.00000  0.00000  1577
R10-RAWBC-PdCacOn             2000  4096    2  124.14 14.54%     0.185     1406.61   0.00000  0.00000   853
R10-RAWBC-PdCacOn             2000  4096    4  125.12 33.76%     0.347     1811.97   0.00000  0.00000   371
R10-RAWBC-PdCacOn             2000  4096    8  126.05 70.05%     0.679     1398.69   0.00000  0.00000   180

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOn             2000  4096    1    1.19 0.617%     9.882       96.18   0.00000  0.00000   192
R10-RAWBC-PdCacOn             2000  4096    2    2.02 2.255%    10.182       95.31   0.00000  0.00000    89
R10-RAWBC-PdCacOn             2000  4096    4    3.33 4.808%    10.761      140.33   0.00000  0.00000    69
R10-RAWBC-PdCacOn             2000  4096    8    5.98 17.81%    12.368      173.46   0.00000  0.00000    34

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOn             2000  4096    1  295.63 39.42%     0.038      280.16   0.00000  0.00000   750
R10-RAWBC-PdCacOn             2000  4096    2  258.20 79.84%     0.088      190.60   0.00000  0.00000   323
R10-RAWBC-PdCacOn             2000  4096    4  282.15 140.9%     0.160      848.78   0.00000  0.00000   200
R10-RAWBC-PdCacOn             2000  4096    8  255.79 340.0%     0.342      400.85   0.00000  0.00000    75

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOn             2000  4096    1   16.55     0%     0.009        0.06   0.00000  0.00000     0
R10-RAWBC-PdCacOn             2000  4096    2   18.37     0%     0.008        0.06   0.00000  0.00000     0
R10-RAWBC-PdCacOn             2000  4096    4   13.72 1.873%     0.010        0.07   0.00000  0.00000   732
R10-RAWBC-PdCacOn             2000  4096    8   13.51 12.91%     0.016        0.09   0.00000  0.00000   105

RAWBC - Disk Cache Off

 storcli /c0/v0 set pdcache=Off  
tiobench --size 2000 --identifier R10-RAWBC-PdCacOff  --numruns 1
Run #1: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOff            2000  4096    1  178.90 10.80%     0.022      198.66   0.00000  0.00000  1656
R10-RAWBC-PdCacOff            2000  4096    2  111.23 12.90%     0.069      331.54   0.00000  0.00000   862
R10-RAWBC-PdCacOff            2000  4096    4  173.12 48.85%     0.087      402.51   0.00000  0.00000   354
R10-RAWBC-PdCacOff            2000  4096    8  135.25 71.82%     0.202      400.28   0.00000  0.00000   188

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOff            2000  4096    1    1.14 0.552%     3.440       29.15   0.00000  0.00000   206
R10-RAWBC-PdCacOff            2000  4096    2    2.11 3.455%     3.378       51.59   0.00000  0.00000    61
R10-RAWBC-PdCacOff            2000  4096    4    3.84 3.635%     3.434       31.46   0.00000  0.00000   106
R10-RAWBC-PdCacOff            2000  4096    8    5.95 20.72%     3.958       53.06   0.00000  0.00000    29

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOff            2000  4096    1  277.00 34.90%     0.013      320.74   0.00000  0.00000   794
R10-RAWBC-PdCacOff            2000  4096    2  211.19 67.75%     0.036      133.28   0.00000  0.00000   312
R10-RAWBC-PdCacOff            2000  4096    4  279.79 164.3%     0.053      116.90   0.00000  0.00000   170
R10-RAWBC-PdCacOff            2000  4096    8  212.86 254.1%     0.140      279.91   0.00000  0.00000    84

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-RAWBC-PdCacOff            2000  4096    1   15.27     0%     0.003        0.02   0.00000  0.00000     0
R10-RAWBC-PdCacOff            2000  4096    2   14.53 0.744%     0.003        0.03   0.00000  0.00000  1953
R10-RAWBC-PdCacOff            2000  4096    4   18.08     0%     0.003        0.03   0.00000  0.00000     0
R10-RAWBC-PdCacOff            2000  4096    8   15.72 9.660%     0.006        0.03   0.00000  0.00000   163

NRAWBC - Disk Cache Off

storcli /c0/v0 set rdcache=nora
storcli /c0/v0 show
-----------------------------------------------------------
DG/VD TYPE   State Access Consist Cache  sCC     Size Name 
-----------------------------------------------------------
0/0   RAID10 Optl  RW     No      NRAWBC -   5.454 TB      
-----------------------------------------------------------
tiobench --size 2000 --identifier R10-NRAWBC-PdCacOff  --numruns 1
Run #1: /usr/bin/tiotest -t 8 -f 250 -r 500 -b 4096 -d . -TTT

Sequential Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-NRAWBC-PdCacOff           2000  4096    1  190.21 11.41%     0.020      433.66   0.00000  0.00000  1667
R10-NRAWBC-PdCacOff           2000  4096    2  109.12 14.42%     0.071      275.32   0.00000  0.00000   756
R10-NRAWBC-PdCacOff           2000  4096    4  123.45 33.13%     0.110      373.31   0.00000  0.00000   373
R10-NRAWBC-PdCacOff           2000  4096    8  128.07 70.72%     0.219      350.40   0.00000  0.00000   181

Random Reads
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-NRAWBC-PdCacOff           2000  4096    1    1.22 0.591%     3.210       61.00   0.00000  0.00000   206
R10-NRAWBC-PdCacOff           2000  4096    2    2.17 3.770%     3.446       48.13   0.00000  0.00000    57
R10-NRAWBC-PdCacOff           2000  4096    4    3.28 4.874%     3.360       34.23   0.00000  0.00000    67
R10-NRAWBC-PdCacOff           2000  4096    8    6.04 17.45%     3.928       48.08   0.00000  0.00000    35

Sequential Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-NRAWBC-PdCacOff           2000  4096    1  288.05 39.86%     0.013      267.42   0.00000  0.00000   722
R10-NRAWBC-PdCacOff           2000  4096    2  216.66 73.93%     0.035       59.29   0.00000  0.00000   293
R10-NRAWBC-PdCacOff           2000  4096    4  262.35 133.5%     0.057      122.33   0.00000  0.00000   196
R10-NRAWBC-PdCacOff           2000  4096    8  213.97 265.4%     0.138      175.91   0.00000  0.00000    81

Random Writes
                              File  Blk   Num                   Avg      Maximum      Lat%     Lat%    CPU
Identifier                    Size  Size  Thr   Rate  (CPU%)  Latency    Latency      >2s      >10s    Eff
---------------------------- ------ ----- ---  ------ ------ --------- -----------  -------- -------- -----
R10-NRAWBC-PdCacOff           2000  4096    1   14.42     0%     0.003        0.02   0.00000  0.00000     0
R10-NRAWBC-PdCacOff           2000  4096    2   17.76     0%     0.003        0.02   0.00000  0.00000     0
R10-NRAWBC-PdCacOff           2000  4096    4   15.63     0%     0.003        0.03   0.00000  0.00000     0
R10-NRAWBC-PdCacOff           2000  4096    8   15.65 16.42%     0.006        0.03   0.00000  0.00000    95