虚拟机性能测试数据
性能测试的详细数据
CPU 测试
输出结果:
- 物理机:
Benchmark Run: Mon Aug 09 2021 17:19:07 - 17:48:22
48 CPUs in system; running 1 parallel copy of tests
Dhrystone 2 using register variables 34839728.4 lps (10.0 s, 7 samples)
Double-Precision Whetstone 4368.3 MWIPS (8.8 s, 7 samples)
Execl Throughput 1291.3 lps (29.1 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks 517296.7 KBps (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks 139166.0 KBps (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks 1825548.4 KBps (30.0 s, 2 samples)
Pipe Throughput 670895.6 lps (10.0 s, 7 samples)
Pipe-based Context Switching 111871.6 lps (10.0 s, 7 samples)
Process Creation 3276.8 lps (30.0 s, 2 samples)
Shell Scripts (1 concurrent) 3604.5 lpm (60.0 s, 2 samples)
Shell Scripts (8 concurrent) 2900.6 lpm (60.0 s, 2 samples)
System Call Overhead 476273.4 lps (10.0 s, 7 samples)
- 虚拟机:
Benchmark Run: Mon Aug 09 2021 10:03:04 - 10:31:06
8 CPUs in system; running 1 parallel copy of tests
Dhrystone 2 using register variables 33964545.0 lps (10.0 s, 7 samples)
Double-Precision Whetstone 4244.3 MWIPS (9.9 s, 7 samples)
Execl Throughput 2348.2 lps (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks 644436.7 KBps (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks 169199.0 KBps (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks 1788048.3 KBps (30.0 s, 2 samples)
Pipe Throughput 933768.1 lps (10.0 s, 7 samples)
Pipe-based Context Switching 139898.5 lps (10.0 s, 7 samples)
Process Creation 5991.6 lps (30.0 s, 2 samples)
Shell Scripts (1 concurrent) 5698.6 lpm (60.0 s, 2 samples)
Shell Scripts (8 concurrent) 2662.4 lpm (60.0 s, 2 samples)
System Call Overhead 920352.7 lps (10.0 s, 7 samples)
内存测试
输出结果：
- 物理机：
[root@baremetal-test mbw]# ./mbw 512
Long uses 8 bytes. Allocating 2*67108864 elements = 1073741824 bytes of memory.
Using 262144 bytes as blocks for memcpy block copy test.
Getting down to business... Doing 10 runs per test.
0 Method: MEMCPY Elapsed: 0.07964 MiB: 512.00000 Copy: 6429.172 MiB/s
1 Method: MEMCPY Elapsed: 0.06827 MiB: 512.00000 Copy: 7500.183 MiB/s
2 Method: MEMCPY Elapsed: 0.06628 MiB: 512.00000 Copy: 7724.338 MiB/s
3 Method: MEMCPY Elapsed: 0.06668 MiB: 512.00000 Copy: 7677.889 MiB/s
4 Method: MEMCPY Elapsed: 0.06568 MiB: 512.00000 Copy: 7795.253 MiB/s
5 Method: MEMCPY Elapsed: 0.06569 MiB: 512.00000 Copy: 7794.541 MiB/s
6 Method: MEMCPY Elapsed: 0.06578 MiB: 512.00000 Copy: 7783.994 MiB/s
7 Method: MEMCPY Elapsed: 0.06656 MiB: 512.00000 Copy: 7692.539 MiB/s
8 Method: MEMCPY Elapsed: 0.06555 MiB: 512.00000 Copy: 7811.189 MiB/s
9 Method: MEMCPY Elapsed: 0.06549 MiB: 512.00000 Copy: 7818.107 MiB/s
AVG Method: MEMCPY Elapsed: 0.06756 MiB: 512.00000 Copy: 7578.348 MiB/s
0 Method: DUMB Elapsed: 0.17209 MiB: 512.00000 Copy: 2975.239 MiB/s
1 Method: DUMB Elapsed: 0.17363 MiB: 512.00000 Copy: 2948.833 MiB/s
2 Method: DUMB Elapsed: 0.16853 MiB: 512.00000 Copy: 3038.053 MiB/s
3 Method: DUMB Elapsed: 0.17081 MiB: 512.00000 Copy: 2997.447 MiB/s
4 Method: DUMB Elapsed: 0.17639 MiB: 512.00000 Copy: 2902.642 MiB/s
5 Method: DUMB Elapsed: 0.16661 MiB: 512.00000 Copy: 3073.137 MiB/s
6 Method: DUMB Elapsed: 0.17822 MiB: 512.00000 Copy: 2872.773 MiB/s
7 Method: DUMB Elapsed: 0.16628 MiB: 512.00000 Copy: 3079.181 MiB/s
8 Method: DUMB Elapsed: 0.16490 MiB: 512.00000 Copy: 3104.818 MiB/s
9 Method: DUMB Elapsed: 0.17354 MiB: 512.00000 Copy: 2950.345 MiB/s
AVG Method: DUMB Elapsed: 0.17110 MiB: 512.00000 Copy: 2992.404 MiB/s
0 Method: MCBLOCK Elapsed: 0.09400 MiB: 512.00000 Copy: 5447.040 MiB/s
1 Method: MCBLOCK Elapsed: 0.09566 MiB: 512.00000 Copy: 5352.177 MiB/s
2 Method: MCBLOCK Elapsed: 0.09422 MiB: 512.00000 Copy: 5434.321 MiB/s
3 Method: MCBLOCK Elapsed: 0.09393 MiB: 512.00000 Copy: 5450.926 MiB/s
4 Method: MCBLOCK Elapsed: 0.09398 MiB: 512.00000 Copy: 5447.678 MiB/s
5 Method: MCBLOCK Elapsed: 0.09439 MiB: 512.00000 Copy: 5424.533 MiB/s
6 Method: MCBLOCK Elapsed: 0.09482 MiB: 512.00000 Copy: 5399.933 MiB/s
7 Method: MCBLOCK Elapsed: 0.09395 MiB: 512.00000 Copy: 5449.533 MiB/s
8 Method: MCBLOCK Elapsed: 0.09476 MiB: 512.00000 Copy: 5402.953 MiB/s
9 Method: MCBLOCK Elapsed: 0.09401 MiB: 512.00000 Copy: 5446.403 MiB/s
AVG Method: MCBLOCK Elapsed: 0.09437 MiB: 512.00000 Copy: 5425.378 MiB/s
- 虚拟机:
[root@guest-test mbw]# ./mbw 512
Long uses 8 bytes. Allocating 2*67108864 elements = 1073741824 bytes of memory.
Using 262144 bytes as blocks for memcpy block copy test.
Getting down to business... Doing 10 runs per test.
0 Method: MEMCPY Elapsed: 0.08215 MiB: 512.00000 Copy: 6232.350 MiB/s
1 Method: MEMCPY Elapsed: 0.08157 MiB: 512.00000 Copy: 6276.433 MiB/s
2 Method: MEMCPY Elapsed: 0.08259 MiB: 512.00000 Copy: 6198.998 MiB/s
3 Method: MEMCPY Elapsed: 0.08155 MiB: 512.00000 Copy: 6278.357 MiB/s
4 Method: MEMCPY Elapsed: 0.08166 MiB: 512.00000 Copy: 6269.746 MiB/s
5 Method: MEMCPY Elapsed: 0.08159 MiB: 512.00000 Copy: 6275.125 MiB/s
6 Method: MEMCPY Elapsed: 0.08167 MiB: 512.00000 Copy: 6268.978 MiB/s
7 Method: MEMCPY Elapsed: 0.08157 MiB: 512.00000 Copy: 6276.587 MiB/s
8 Method: MEMCPY Elapsed: 0.08261 MiB: 512.00000 Copy: 6198.022 MiB/s
9 Method: MEMCPY Elapsed: 0.08163 MiB: 512.00000 Copy: 6272.127 MiB/s
AVG Method: MEMCPY Elapsed: 0.08186 MiB: 512.00000 Copy: 6254.520 MiB/s
0 Method: DUMB Elapsed: 0.20884 MiB: 512.00000 Copy: 2451.649 MiB/s
1 Method: DUMB Elapsed: 0.20017 MiB: 512.00000 Copy: 2557.864 MiB/s
2 Method: DUMB Elapsed: 0.17721 MiB: 512.00000 Copy: 2889.179 MiB/s
3 Method: DUMB Elapsed: 0.17734 MiB: 512.00000 Copy: 2887.110 MiB/s
4 Method: DUMB Elapsed: 0.17337 MiB: 512.00000 Copy: 2953.238 MiB/s
5 Method: DUMB Elapsed: 0.17051 MiB: 512.00000 Copy: 3002.704 MiB/s
6 Method: DUMB Elapsed: 0.17720 MiB: 512.00000 Copy: 2889.358 MiB/s
7 Method: DUMB Elapsed: 0.17516 MiB: 512.00000 Copy: 2923.008 MiB/s
8 Method: DUMB Elapsed: 0.17527 MiB: 512.00000 Copy: 2921.174 MiB/s
9 Method: DUMB Elapsed: 0.17683 MiB: 512.00000 Copy: 2895.354 MiB/s
AVG Method: DUMB Elapsed: 0.18119 MiB: 512.00000 Copy: 2825.744 MiB/s
0 Method: MCBLOCK Elapsed: 0.09484 MiB: 512.00000 Copy: 5398.338 MiB/s
1 Method: MCBLOCK Elapsed: 0.09387 MiB: 512.00000 Copy: 5454.294 MiB/s
2 Method: MCBLOCK Elapsed: 0.09394 MiB: 512.00000 Copy: 5450.055 MiB/s
3 Method: MCBLOCK Elapsed: 0.09392 MiB: 512.00000 Copy: 5451.390 MiB/s
4 Method: MCBLOCK Elapsed: 0.09394 MiB: 512.00000 Copy: 5450.403 MiB/s
5 Method: MCBLOCK Elapsed: 0.09391 MiB: 512.00000 Copy: 5451.912 MiB/s
6 Method: MCBLOCK Elapsed: 0.09496 MiB: 512.00000 Copy: 5391.914 MiB/s
7 Method: MCBLOCK Elapsed: 0.09400 MiB: 512.00000 Copy: 5446.866 MiB/s
8 Method: MCBLOCK Elapsed: 0.09388 MiB: 512.00000 Copy: 5454.003 MiB/s
9 Method: MCBLOCK Elapsed: 0.09381 MiB: 512.00000 Copy: 5457.549 MiB/s
AVG Method: MCBLOCK Elapsed: 0.09411 MiB: 512.00000 Copy: 5440.575 MiB/s
本地存储IO测试
测试数据
- 物理机
# 4KB随机读
random-read: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [r(1)][100.0%][r=828KiB/s,w=0KiB/s][r=207,w=0 IOPS][eta 00m:00s]
random-read: (groupid=0, jobs=1): err= 0: pid=24797: Sat Oct 30 00:06:20 2021
read: IOPS=195, BW=782KiB/s (801kB/s)(45.9MiB/60039msec)
slat (nsec): min=105, max=109517, avg=820.31, stdev=1377.18
clat (msec): min=41, max=184, avg=81.76, stdev=15.04
lat (msec): min=41, max=184, avg=81.76, stdev=15.04
clat percentiles (msec):
| 1.00th=[ 53], 5.00th=[ 61], 10.00th=[ 65], 20.00th=[ 70],
| 30.00th=[ 74], 40.00th=[ 78], 50.00th=[ 81], 60.00th=[ 84],
| 70.00th=[ 88], 80.00th=[ 92], 90.00th=[ 100], 95.00th=[ 109],
| 99.00th=[ 130], 99.50th=[ 133], 99.90th=[ 165], 99.95th=[ 165],
| 99.99th=[ 182]
bw ( KiB/s): min= 560, max= 992, per=99.97%, avg=781.78, stdev=75.68, samples=120
iops : min= 140, max= 248, avg=195.42, stdev=18.92, samples=120
lat (msec) : 50=0.65%, 100=90.14%, 250=9.21%
cpu : usr=0.15%, sys=0.06%, ctx=5878, majf=0, minf=54
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.0%, 16=49.9%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.8%, 8=0.1%, 16=4.2%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=11744,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=782KiB/s (801kB/s), 782KiB/s-782KiB/s (801kB/s-801kB/s), io=45.9MiB (48.1MB), run=60039-60039msec
# 4KB随机写
Disk stats (read/write):
sda: ios=11821/13626, merge=0/68, ticks=60458/1750, in_queue=62206, util=99.61%
random-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=30.4MiB/s][r=0,w=7786 IOPS][eta 00m:00s]
random-write: (groupid=0, jobs=1): err= 0: pid=25911: Sat Oct 30 00:07:21 2021
write: IOPS=7349, BW=28.7MiB/s (30.1MB/s)(1723MiB/60002msec)
slat (nsec): min=106, max=126819, avg=490.79, stdev=527.24
clat (usec): min=421, max=35306, avg=2173.06, stdev=1510.06
lat (usec): min=422, max=35308, avg=2173.55, stdev=1510.07
clat percentiles (usec):
| 1.00th=[ 701], 5.00th=[ 816], 10.00th=[ 906], 20.00th=[ 1045],
| 30.00th=[ 1188], 40.00th=[ 1352], 50.00th=[ 1614], 60.00th=[ 1991],
| 70.00th=[ 2507], 80.00th=[ 3195], 90.00th=[ 4228], 95.00th=[ 5211],
| 99.00th=[ 7046], 99.50th=[ 7963], 99.90th=[11338], 99.95th=[13960],
| 99.99th=[22152]
bw ( KiB/s): min=22784, max=73576, per=99.99%, avg=29392.35, stdev=5074.51, samples=120
iops : min= 5696, max=18394, avg=7348.06, stdev=1268.63, samples=120
lat (usec) : 500=0.01%, 750=2.56%, 1000=14.13%
lat (msec) : 2=43.57%, 4=27.87%, 10=11.73%, 20=0.13%, 50=0.01%
cpu : usr=2.60%, sys=1.30%, ctx=220949, majf=0, minf=199
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.1%, 16=49.9%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=96.1%, 8=3.4%, 16=0.5%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,440960,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=28.7MiB/s (30.1MB/s), 28.7MiB/s-28.7MiB/s (30.1MB/s-30.1MB/s), io=1723MiB (1806MB), run=60002-60002msec
# 4KB顺序读
Disk stats (read/write):
sda: ios=366/454672, merge=0/66, ticks=8513/72093, in_queue=80441, util=91.29%
random-read: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [R(1)][100.0%][r=102MiB/s,w=0KiB/s][r=26.2k,w=0 IOPS][eta 00m:00s]
random-read: (groupid=0, jobs=1): err= 0: pid=27058: Sat Oct 30 00:08:22 2021
read: IOPS=22.1k, BW=86.3MiB/s (90.5MB/s)(5177MiB/60001msec)
slat (nsec): min=85, max=514542, avg=279.98, stdev=729.38
clat (usec): min=100, max=63893, avg=722.15, stdev=770.18
lat (usec): min=101, max=63893, avg=722.43, stdev=770.20
clat percentiles (usec):
| 1.00th=[ 523], 5.00th=[ 537], 10.00th=[ 553], 20.00th=[ 578],
| 30.00th=[ 586], 40.00th=[ 603], 50.00th=[ 619], 60.00th=[ 644],
| 70.00th=[ 676], 80.00th=[ 717], 90.00th=[ 816], 95.00th=[ 1319],
| 99.00th=[ 1647], 99.50th=[ 3130], 99.90th=[10028], 99.95th=[13698],
| 99.99th=[36963]
bw ( KiB/s): min=48192, max=108624, per=99.99%, avg=88337.98, stdev=16408.38, samples=120
iops : min=12048, max=27156, avg=22084.47, stdev=4102.10, samples=120
lat (usec) : 250=0.01%, 500=0.12%, 750=85.27%, 1000=7.30%
lat (msec) : 2=6.59%, 4=0.29%, 10=0.32%, 20=0.08%, 50=0.02%
lat (msec) : 100=0.01%
cpu : usr=5.11%, sys=2.44%, ctx=668513, majf=0, minf=89
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.1%, 16=49.8%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=96.5%, 8=3.0%, 16=0.5%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=1325257,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=86.3MiB/s (90.5MB/s), 86.3MiB/s-86.3MiB/s (90.5MB/s-90.5MB/s), io=5177MiB (5428MB), run=60001-60001msec
# 4KB顺序写
Disk stats (read/write):
sda: ios=1322430/18311, merge=0/55, ticks=46838/2900, in_queue=49343, util=77.26%
random-write: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [W(1)][100.0%][r=0KiB/s,w=112MiB/s][r=0,w=28.7k IOPS][eta 00m:00s]
random-write: (groupid=0, jobs=1): err= 0: pid=28222: Sat Oct 30 00:09:22 2021
write: IOPS=24.5k, BW=95.8MiB/s (100MB/s)(5749MiB/60003msec)
slat (nsec): min=104, max=265343, avg=332.75, stdev=542.34
clat (usec): min=172, max=21303, avg=650.00, stdev=199.53
lat (usec): min=172, max=21304, avg=650.34, stdev=199.55
clat percentiles (usec):
| 1.00th=[ 474], 5.00th=[ 490], 10.00th=[ 506], 20.00th=[ 545],
| 30.00th=[ 578], 40.00th=[ 603], 50.00th=[ 619], 60.00th=[ 644],
| 70.00th=[ 668], 80.00th=[ 709], 90.00th=[ 783], 95.00th=[ 848],
| 99.00th=[ 1680], 99.50th=[ 1795], 99.90th=[ 1958], 99.95th=[ 2147],
| 99.99th=[ 3458]
bw ( KiB/s): min=71848, max=116968, per=99.84%, avg=97955.39, stdev=9230.84, samples=119
iops : min=17962, max=29242, avg=24488.86, stdev=2307.70, samples=119
lat (usec) : 250=0.01%, 500=7.71%, 750=78.94%, 1000=10.97%
lat (msec) : 2=2.31%, 4=0.07%, 10=0.01%, 20=0.01%, 50=0.01%
cpu : usr=5.70%, sys=2.47%, ctx=734099, majf=0, minf=106
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.1%, 16=49.9%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=96.5%, 8=2.9%, 16=0.6%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,1471719,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=95.8MiB/s (100MB/s), 95.8MiB/s-95.8MiB/s (100MB/s-100MB/s), io=5749MiB (6028MB), run=60003-60003msec
# 1MB顺序读
Disk stats (read/write):
sda: ios=194/1485684, merge=0/68, ticks=781/41242, in_queue=41581, util=64.75%
seq-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=posixaio, iodepth=1
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [R(1)][100.0%][r=1012MiB/s,w=0KiB/s][r=1012,w=0 IOPS][eta 00m:00s]
seq-read: (groupid=0, jobs=1): err= 0: pid=29329: Sat Oct 30 00:10:23 2021
read: IOPS=1055, BW=1056MiB/s (1107MB/s)(61.9GiB/60001msec)
slat (nsec): min=1044, max=107499, avg=3986.03, stdev=1721.44
clat (usec): min=322, max=57357, avg=941.21, stdev=1327.27
lat (usec): min=327, max=57360, avg=945.20, stdev=1327.38
clat percentiles (usec):
| 1.00th=[ 502], 5.00th=[ 562], 10.00th=[ 594], 20.00th=[ 685],
| 30.00th=[ 750], 40.00th=[ 807], 50.00th=[ 840], 60.00th=[ 865],
| 70.00th=[ 906], 80.00th=[ 971], 90.00th=[ 1139], 95.00th=[ 1303],
| 99.00th=[ 2540], 99.50th=[ 3621], 99.90th=[21365], 99.95th=[29754],
| 99.99th=[47973]
bw ( MiB/s): min= 411, max= 1234, per=99.99%, avg=1055.62, stdev=134.87, samples=120
iops : min= 411, max= 1234, avg=1055.60, stdev=134.86, samples=120
lat (usec) : 500=0.99%, 750=29.18%, 1000=52.75%
lat (msec) : 2=15.42%, 4=1.17%, 10=0.15%, 20=0.22%, 50=0.10%
lat (msec) : 100=0.01%
cpu : usr=0.94%, sys=1.04%, ctx=63587, majf=0, minf=88
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=63342,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=1056MiB/s (1107MB/s), 1056MiB/s-1056MiB/s (1107MB/s-1107MB/s), io=61.9GiB (66.4GB), run=60001-60001msec
# 1MB顺序写
Disk stats (read/write):
sda: ios=253323/13507, merge=0/75, ticks=132267/1870, in_queue=128284, util=93.64%
seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=posixaio, iodepth=1
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [W(1)][100.0%][r=0KiB/s,w=859MiB/s][r=0,w=859 IOPS][eta 00m:00s]
seq-write: (groupid=0, jobs=1): err= 0: pid=30516: Sat Oct 30 00:11:24 2021
write: IOPS=1117, BW=1118MiB/s (1172MB/s)(65.5GiB/60001msec)
slat (usec): min=9, max=6753, avg=24.51, stdev=29.94
clat (usec): min=271, max=44824, avg=867.94, stdev=330.79
lat (usec): min=290, max=44838, avg=892.45, stdev=333.21
clat percentiles (usec):
| 1.00th=[ 420], 5.00th=[ 635], 10.00th=[ 685], 20.00th=[ 734],
| 30.00th=[ 758], 40.00th=[ 791], 50.00th=[ 824], 60.00th=[ 857],
| 70.00th=[ 898], 80.00th=[ 971], 90.00th=[ 1090], 95.00th=[ 1221],
| 99.00th=[ 1696], 99.50th=[ 2089], 99.90th=[ 3326], 99.95th=[ 4424],
| 99.99th=[10421]
bw ( MiB/s): min= 400, max= 2020, per=99.99%, avg=1117.69, stdev=155.96, samples=120
iops : min= 400, max= 2020, avg=1117.67, stdev=155.96, samples=120
lat (usec) : 500=1.61%, 750=25.22%, 1000=56.22%
lat (msec) : 2=16.38%, 4=0.52%, 10=0.04%, 20=0.01%, 50=0.01%
cpu : usr=3.48%, sys=0.91%, ctx=67526, majf=0, minf=1991
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,67068,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=1118MiB/s (1172MB/s), 1118MiB/s-1118MiB/s (1172MB/s-1172MB/s), io=65.5GiB (70.3GB), run=60001-60001msec
Disk stats (read/write):
sda: ios=313/283336, merge=0/65, ticks=95351/143804, in_queue=232888, util=93.03%
- Linux虚拟机（cache=none）
# 4KB随机读
random-read: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
random-read: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [r(1)][100.0%][r=972KiB/s,w=0KiB/s][r=243,w=0 IOPS][eta 00m:00s]
random-read: (groupid=0, jobs=1): err= 0: pid=4810: Sat Oct 30 01:16:42 2021
read: IOPS=245, BW=981KiB/s (1005kB/s)(57.6MiB/60069msec)
slat (nsec): min=454, max=34511k, avg=3635.31, stdev=284257.42
clat (msec): min=18, max=196, avg=65.14, stdev=20.14
lat (msec): min=18, max=196, avg=65.14, stdev=20.14
clat percentiles (msec):
| 1.00th=[ 32], 5.00th=[ 40], 10.00th=[ 44], 20.00th=[ 50],
| 30.00th=[ 54], 40.00th=[ 58], 50.00th=[ 63], 60.00th=[ 67],
| 70.00th=[ 72], 80.00th=[ 79], 90.00th=[ 89], 95.00th=[ 104],
| 99.00th=[ 136], 99.50th=[ 150], 99.90th=[ 178], 99.95th=[ 180],
| 99.99th=[ 192]
bw ( KiB/s): min= 552, max= 1280, per=100.00%, avg=981.51, stdev=150.99, samples=120
iops : min= 138, max= 320, avg=245.37, stdev=37.74, samples=120
lat (msec) : 20=0.02%, 50=20.54%, 100=73.80%, 250=5.63%
cpu : usr=0.41%, sys=0.29%, ctx=7373, majf=1, minf=61
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.0%, 16=49.9%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.8%, 8=3.0%, 16=1.2%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=14739,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=981KiB/s (1005kB/s), 981KiB/s-981KiB/s (1005kB/s-1005kB/s), io=57.6MiB (60.4MB), run=60069-60069msec
# 4KB随机写
Disk stats (read/write):
sdb: ios=14725/7, merge=0/5, ticks=59419/12, in_queue=59422, util=98.99%
random-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
random-write: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=24.2MiB/s][r=0,w=6195 IOPS][eta 00m:00s]
random-write: (groupid=0, jobs=1): err= 0: pid=4815: Sat Oct 30 01:17:43 2021
write: IOPS=6096, BW=23.8MiB/s (24.0MB/s)(1430MiB/60030msec)
slat (nsec): min=463, max=4844.2k, avg=1024.87, stdev=9101.74
clat (usec): min=822, max=18757, avg=2612.45, stdev=539.56
lat (usec): min=824, max=18759, avg=2613.48, stdev=539.70
clat percentiles (usec):
| 1.00th=[ 2245], 5.00th=[ 2278], 10.00th=[ 2311], 20.00th=[ 2376],
| 30.00th=[ 2409], 40.00th=[ 2442], 50.00th=[ 2507], 60.00th=[ 2540],
| 70.00th=[ 2638], 80.00th=[ 2737], 90.00th=[ 2933], 95.00th=[ 3195],
| 99.00th=[ 4293], 99.50th=[ 5866], 99.90th=[ 9372], 99.95th=[10290],
| 99.99th=[14877]
bw ( KiB/s): min=20040, max=25856, per=100.00%, avg=24392.81, stdev=846.07, samples=120
iops : min= 5010, max= 6464, avg=6098.17, stdev=211.52, samples=120
lat (usec) : 1000=0.01%
lat (msec) : 2=0.05%, 4=98.69%, 10=1.20%, 20=0.05%
cpu : usr=5.74%, sys=4.52%, ctx=182607, majf=0, minf=58
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.1%, 16=49.9%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.9%, 8=2.5%, 16=1.6%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,365981,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=23.8MiB/s (24.0MB/s), 23.8MiB/s-23.8MiB/s (24.0MB/s-24.0MB/s), io=1430MiB (1499MB), run=60030-60030msec
# 4KB顺序读
Disk stats (read/write):
sdb: ios=0/371909, merge=0/0, ticks=0/52257, in_queue=52136, util=78.35%
random-read: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [R(1)][100.0%][r=35.7MiB/s,w=0KiB/s][r=9130,w=0 IOPS][eta 00m:00s]
random-read: (groupid=0, jobs=1): err= 0: pid=4821: Sat Oct 30 01:18:44 2021
read: IOPS=9458, BW=36.9MiB/s (38.7MB/s)(2217MiB/60002msec)
slat (nsec): min=432, max=1340.5k, avg=807.74, stdev=2574.66
clat (usec): min=137, max=84747, avg=1682.97, stdev=808.61
lat (usec): min=209, max=84748, avg=1683.77, stdev=808.62
clat percentiles (usec):
| 1.00th=[ 1401], 5.00th=[ 1434], 10.00th=[ 1450], 20.00th=[ 1500],
| 30.00th=[ 1532], 40.00th=[ 1565], 50.00th=[ 1614], 60.00th=[ 1647],
| 70.00th=[ 1696], 80.00th=[ 1778], 90.00th=[ 1909], 95.00th=[ 2073],
| 99.00th=[ 2540], 99.50th=[ 3556], 99.90th=[ 7832], 99.95th=[12911],
| 99.99th=[42206]
bw ( KiB/s): min=29016, max=41285, per=99.99%, avg=37829.40, stdev=2266.77, samples=120
iops : min= 7254, max=10321, avg=9457.33, stdev=566.68, samples=120
lat (usec) : 250=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=93.50%, 4=6.03%, 10=0.39%, 20=0.04%, 50=0.02%
lat (msec) : 100=0.01%
cpu : usr=6.26%, sys=6.74%, ctx=283036, majf=0, minf=62
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.1%, 16=49.9%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=96.1%, 8=2.0%, 16=1.9%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=567555,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=36.9MiB/s (38.7MB/s), 36.9MiB/s-36.9MiB/s (38.7MB/s-38.7MB/s), io=2217MiB (2325MB), run=60002-60002msec
# 4KB顺序写
Disk stats (read/write):
sdb: ios=566497/24, merge=0/2019, ticks=51963/54, in_queue=51908, util=86.49%
random-write: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [W(1)][100.0%][r=0KiB/s,w=25.1MiB/s][r=0,w=6428 IOPS][eta 00m:00s]
random-write: (groupid=0, jobs=1): err= 0: pid=4825: Sat Oct 30 01:19:44 2021
write: IOPS=6278, BW=24.5MiB/s (25.7MB/s)(1472MiB/60006msec)
slat (nsec): min=465, max=588878, avg=1013.68, stdev=1704.99
clat (usec): min=1292, max=43347, avg=2539.67, stdev=513.14
lat (usec): min=1302, max=43349, avg=2540.68, stdev=513.18
clat percentiles (usec):
| 1.00th=[ 2114], 5.00th=[ 2180], 10.00th=[ 2212], 20.00th=[ 2278],
| 30.00th=[ 2311], 40.00th=[ 2343], 50.00th=[ 2409], 60.00th=[ 2474],
| 70.00th=[ 2573], 80.00th=[ 2704], 90.00th=[ 3130], 95.00th=[ 3359],
| 99.00th=[ 3884], 99.50th=[ 4293], 99.90th=[ 5997], 99.95th=[ 8225],
| 99.99th=[15533]
bw ( KiB/s): min=18850, max=26064, per=99.92%, avg=25095.27, stdev=851.44, samples=120
iops : min= 4712, max= 6516, avg=6273.70, stdev=212.93, samples=120
lat (msec) : 2=0.06%, 4=99.15%, 10=0.75%, 20=0.03%, 50=0.01%
cpu : usr=5.66%, sys=4.92%, ctx=188032, majf=0, minf=57
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.1%, 16=49.9%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=96.4%, 8=3.1%, 16=0.5%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,376770,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=24.5MiB/s (25.7MB/s), 24.5MiB/s-24.5MiB/s (25.7MB/s-25.7MB/s), io=1472MiB (1543MB), run=60006-60006msec
# 1MB顺序读
Disk stats (read/write):
sdb: ios=0/376777, merge=0/0, ticks=0/50320, in_queue=50180, util=83.54%
seq-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=posixaio, iodepth=1
fio-3.7
Starting 1 process
seq-read: Laying out IO file (1 file / 16384MiB)
Jobs: 1 (f=1): [R(1)][100.0%][r=1070MiB/s,w=0KiB/s][r=1070,w=0 IOPS][eta 00m:00s]
seq-read: (groupid=0, jobs=1): err= 0: pid=4831: Sat Oct 30 01:21:02 2021
read: IOPS=979, BW=980MiB/s (1027MB/s)(57.4GiB/60001msec)
slat (nsec): min=1520, max=5174.5k, avg=10653.72, stdev=22099.10
clat (usec): min=468, max=66949, avg=1005.92, stdev=1405.26
lat (usec): min=477, max=66958, avg=1016.57, stdev=1405.59
clat percentiles (usec):
| 1.00th=[ 578], 5.00th=[ 635], 10.00th=[ 668], 20.00th=[ 725],
| 30.00th=[ 791], 40.00th=[ 857], 50.00th=[ 906], 60.00th=[ 938],
| 70.00th=[ 971], 80.00th=[ 1020], 90.00th=[ 1156], 95.00th=[ 1385],
| 99.00th=[ 3032], 99.50th=[ 7373], 99.90th=[21890], 99.95th=[33817],
| 99.99th=[52691]
bw ( KiB/s): min=362496, max=1384448, per=99.90%, avg=1002091.21, stdev=157173.29, samples=119
iops : min= 354, max= 1352, avg=978.56, stdev=153.49, samples=119
lat (usec) : 500=0.05%, 750=24.05%, 1000=52.55%
lat (msec) : 2=21.68%, 4=0.91%, 10=0.46%, 20=0.17%, 50=0.11%
lat (msec) : 100=0.01%
cpu : usr=1.90%, sys=3.52%, ctx=58790, majf=0, minf=60
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=58776,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=980MiB/s (1027MB/s), 980MiB/s-980MiB/s (1027MB/s-1027MB/s), io=57.4GiB (61.6GB), run=60001-60001msec
# 1MB顺序写
Disk stats (read/write):
sdb: ios=175957/5, merge=0/3, ticks=124683/0, in_queue=123610, util=88.13%
seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=posixaio, iodepth=1
fio-3.7
Starting 1 process
seq-write: Laying out IO file (1 file / 16384MiB)
Jobs: 1 (f=1): [W(1)][100.0%][r=0KiB/s,w=1036MiB/s][r=0,w=1036 IOPS][eta 00m:00s]
seq-write: (groupid=0, jobs=1): err= 0: pid=4835: Sat Oct 30 01:22:03 2021
write: IOPS=1058, BW=1059MiB/s (1110MB/s)(62.0GiB/60001msec)
slat (usec): min=12, max=1522, avg=31.91, stdev=12.80
clat (usec): min=493, max=48198, avg=908.03, stdev=391.90
lat (usec): min=528, max=48234, avg=939.94, stdev=392.47
clat percentiles (usec):
| 1.00th=[ 627], 5.00th=[ 709], 10.00th=[ 742], 20.00th=[ 783],
| 30.00th=[ 807], 40.00th=[ 840], 50.00th=[ 865], 60.00th=[ 889],
| 70.00th=[ 930], 80.00th=[ 979], 90.00th=[ 1074], 95.00th=[ 1205],
| 99.00th=[ 1795], 99.50th=[ 2245], 99.90th=[ 4080], 99.95th=[ 4817],
| 99.99th=[11076]
bw ( MiB/s): min= 486, max= 1276, per=100.00%, avg=1058.92, stdev=112.74, samples=119
iops : min= 486, max= 1276, avg=1058.89, stdev=112.74, samples=119
lat (usec) : 500=0.01%, 750=11.46%, 1000=71.77%
lat (msec) : 2=16.05%, 4=0.61%, 10=0.09%, 20=0.01%, 50=0.01%
cpu : usr=4.91%, sys=2.88%, ctx=63546, majf=0, minf=58
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,63538,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=1059MiB/s (1110MB/s), 1059MiB/s-1059MiB/s (1110MB/s-1110MB/s), io=62.0GiB (66.6GB), run=60001-60001msec
Disk stats (read/write):
sdb: ios=0/190623, merge=0/1, ticks=0/134765, in_queue=133681, util=85.12%
- Linux虚拟机（cache=none,queues=4)
# 4KB随机读
random-read: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
random-read: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [r(1)][100.0%][r=896KiB/s,w=0KiB/s][r=224,w=0 IOPS][eta 00m:00s]
random-read: (groupid=0, jobs=1): err= 0: pid=4785: Sat Oct 30 01:52:12 2021
read: IOPS=243, BW=973KiB/s (996kB/s)(57.1MiB/60049msec)
slat (nsec): min=445, max=8814.3k, avg=1814.66, stdev=72928.10
clat (msec): min=23, max=178, avg=65.72, stdev=19.18
lat (msec): min=23, max=178, avg=65.73, stdev=19.18
clat percentiles (msec):
| 1.00th=[ 32], 5.00th=[ 41], 10.00th=[ 46], 20.00th=[ 52],
| 30.00th=[ 56], 40.00th=[ 59], 50.00th=[ 64], 60.00th=[ 67],
| 70.00th=[ 72], 80.00th=[ 78], 90.00th=[ 89], 95.00th=[ 102],
| 99.00th=[ 133], 99.50th=[ 144], 99.90th=[ 174], 99.95th=[ 178],
| 99.99th=[ 178]
bw ( KiB/s): min= 440, max= 1256, per=99.95%, avg=972.56, stdev=137.15, samples=120
iops : min= 110, max= 314, avg=243.09, stdev=34.28, samples=120
lat (msec) : 50=17.26%, 100=77.44%, 250=5.30%
cpu : usr=0.41%, sys=0.26%, ctx=7302, majf=1, minf=61
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.0%, 16=49.9%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.8%, 8=0.1%, 16=4.2%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=14608,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=973KiB/s (996kB/s), 973KiB/s-973KiB/s (996kB/s-996kB/s), io=57.1MiB (59.8MB), run=60049-60049msec
# 4KB随机写
Disk stats (read/write):
sdb: ios=14568/7, merge=0/5, ticks=59340/180, in_queue=59514, util=98.95%
random-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
random-write: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=25.0MiB/s][r=0,w=6411 IOPS][eta 00m:00s]
random-write: (groupid=0, jobs=1): err= 0: pid=4789: Sat Oct 30 01:53:13 2021
write: IOPS=7483, BW=29.2MiB/s (30.7MB/s)(1755MiB/60019msec)
slat (nsec): min=447, max=6054.0k, avg=980.93, stdev=12417.10
clat (usec): min=240, max=67969, avg=2127.42, stdev=729.10
lat (usec): min=253, max=67971, avg=2128.40, stdev=729.22
clat percentiles (usec):
| 1.00th=[ 1729], 5.00th=[ 1795], 10.00th=[ 1827], 20.00th=[ 1860],
| 30.00th=[ 1909], 40.00th=[ 1942], 50.00th=[ 1991], 60.00th=[ 2040],
| 70.00th=[ 2114], 80.00th=[ 2245], 90.00th=[ 2474], 95.00th=[ 2769],
| 99.00th=[ 4293], 99.50th=[ 6849], 99.90th=[ 9503], 99.95th=[11207],
| 99.99th=[16581]
bw ( KiB/s): min=22296, max=32512, per=100.00%, avg=29941.92, stdev=1923.92, samples=120
iops : min= 5574, max= 8128, avg=7485.46, stdev=480.97, samples=120
lat (usec) : 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=52.88%, 4=45.95%, 10=1.10%, 20=0.06%, 50=0.01%
lat (msec) : 100=0.01%
cpu : usr=6.32%, sys=5.68%, ctx=223660, majf=0, minf=58
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.1%, 16=49.8%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=96.1%, 8=2.5%, 16=1.3%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,449182,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=29.2MiB/s (30.7MB/s), 29.2MiB/s-29.2MiB/s (30.7MB/s-30.7MB/s), io=1755MiB (1840MB), run=60019-60019msec
# 4KB顺序读
Disk stats (read/write):
sdb: ios=0/459554, merge=0/178, ticks=0/54640, in_queue=54482, util=75.42%
random-read: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [R(1)][100.0%][r=38.0MiB/s,w=0KiB/s][r=9729,w=0 IOPS][eta 00m:00s]
random-read: (groupid=0, jobs=1): err= 0: pid=4794: Sat Oct 30 01:54:14 2021
read: IOPS=11.5k, BW=44.9MiB/s (47.0MB/s)(2691MiB/60002msec)
slat (nsec): min=419, max=2605.9k, avg=854.55, stdev=5320.53
clat (usec): min=29, max=24208, avg=1384.52, stdev=491.41
lat (usec): min=209, max=24209, avg=1385.38, stdev=491.47
clat percentiles (usec):
| 1.00th=[ 963], 5.00th=[ 1012], 10.00th=[ 1057], 20.00th=[ 1106],
| 30.00th=[ 1172], 40.00th=[ 1237], 50.00th=[ 1303], 60.00th=[ 1385],
| 70.00th=[ 1467], 80.00th=[ 1598], 90.00th=[ 1729], 95.00th=[ 1860],
| 99.00th=[ 2802], 99.50th=[ 3654], 99.90th=[ 6915], 99.95th=[ 9241],
| 99.99th=[15926]
bw ( KiB/s): min=30728, max=54144, per=99.99%, avg=45925.91, stdev=4851.27, samples=120
iops : min= 7682, max=13536, avg=11481.45, stdev=1212.83, samples=120
lat (usec) : 50=0.01%, 250=0.01%, 500=0.01%, 750=0.02%, 1000=3.54%
lat (msec) : 2=92.99%, 4=3.04%, 10=0.37%, 20=0.04%, 50=0.01%
cpu : usr=8.11%, sys=8.32%, ctx=340402, majf=0, minf=61
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.3%, 16=49.6%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.8%, 8=2.6%, 16=1.6%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=688976,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=44.9MiB/s (47.0MB/s), 44.9MiB/s-44.9MiB/s (47.0MB/s-47.0MB/s), io=2691MiB (2822MB), run=60002-60002msec
# 4KB顺序写
Disk stats (read/write):
sdb: ios=687927/27, merge=0/2076, ticks=49392/28, in_queue=49238, util=82.09%
random-write: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [W(1)][100.0%][r=0KiB/s,w=29.9MiB/s][r=0,w=7662 IOPS][eta 00m:00s]
random-write: (groupid=0, jobs=1): err= 0: pid=4798: Sat Oct 30 01:55:14 2021
write: IOPS=7271, BW=28.4MiB/s (29.8MB/s)(1704MiB/60004msec)
slat (nsec): min=442, max=1140.2k, avg=1040.46, stdev=3941.55
clat (usec): min=529, max=47390, avg=2190.82, stdev=663.74
lat (usec): min=531, max=47392, avg=2191.86, stdev=663.82
clat percentiles (usec):
| 1.00th=[ 1663], 5.00th=[ 1729], 10.00th=[ 1762], 20.00th=[ 1827],
| 30.00th=[ 1876], 40.00th=[ 1926], 50.00th=[ 2008], 60.00th=[ 2114],
| 70.00th=[ 2245], 80.00th=[ 2474], 90.00th=[ 2868], 95.00th=[ 3163],
| 99.00th=[ 4146], 99.50th=[ 4817], 99.90th=[ 7767], 99.95th=[10028],
| 99.99th=[16450]
bw ( KiB/s): min=23160, max=32184, per=99.99%, avg=29082.50, stdev=2032.53, samples=120
iops : min= 5790, max= 8046, avg=7270.61, stdev=508.13, samples=120
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=49.30%, 4=49.50%, 10=1.15%, 20=0.04%, 50=0.01%
cpu : usr=6.57%, sys=5.99%, ctx=216527, majf=0, minf=57
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.2%, 16=49.7%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=96.1%, 8=2.8%, 16=1.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,436332,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=28.4MiB/s (29.8MB/s), 28.4MiB/s-28.4MiB/s (29.8MB/s-29.8MB/s), io=1704MiB (1787MB), run=60004-60004msec
# 1MB顺序读
Disk stats (read/write):
sdb: ios=0/436339, merge=0/0, ticks=0/48476, in_queue=48334, util=80.45%
seq-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=posixaio, iodepth=1
fio-3.7
Starting 1 process
seq-read: Laying out IO file (1 file / 16384MiB)
Jobs: 1 (f=1): [R(1)][100.0%][r=986MiB/s,w=0KiB/s][r=986,w=0 IOPS][eta 00m:00s]
seq-read: (groupid=0, jobs=1): err= 0: pid=4803: Sat Oct 30 01:56:32 2021
read: IOPS=958, BW=959MiB/s (1005MB/s)(56.2GiB/60001msec)
slat (usec): min=2, max=7236, avg=10.69, stdev=32.07
clat (usec): min=480, max=61289, avg=1027.94, stdev=1508.77
lat (usec): min=485, max=61300, avg=1038.63, stdev=1509.28
clat percentiles (usec):
| 1.00th=[ 603], 5.00th=[ 644], 10.00th=[ 676], 20.00th=[ 742],
| 30.00th=[ 799], 40.00th=[ 857], 50.00th=[ 906], 60.00th=[ 938],
| 70.00th=[ 971], 80.00th=[ 1020], 90.00th=[ 1172], 95.00th=[ 1385],
| 99.00th=[ 3195], 99.50th=[ 8717], 99.90th=[22938], 99.95th=[38536],
| 99.99th=[51119]
bw ( KiB/s): min=471040, max=1140736, per=99.97%, avg=981562.49, stdev=142088.97, samples=119
iops : min= 460, max= 1114, avg=958.54, stdev=138.75, samples=119
lat (usec) : 500=0.01%, 750=21.52%, 1000=54.93%
lat (msec) : 2=21.71%, 4=0.96%, 10=0.48%, 20=0.26%, 50=0.11%
lat (msec) : 100=0.01%
cpu : usr=2.16%, sys=3.59%, ctx=57553, majf=0, minf=61
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=57529,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=959MiB/s (1005MB/s), 959MiB/s-959MiB/s (1005MB/s-1005MB/s), io=56.2GiB (60.3GB), run=60001-60001msec
# 1MB顺序写
Disk stats (read/write):
sdb: ios=172183/6, merge=0/3, ticks=123760/4, in_queue=122711, util=88.26%
seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=posixaio, iodepth=1
fio-3.7
Starting 1 process
seq-write: Laying out IO file (1 file / 16384MiB)
Jobs: 1 (f=1): [W(1)][100.0%][r=0KiB/s,w=1102MiB/s][r=0,w=1102 IOPS][eta 00m:00s]
seq-write: (groupid=0, jobs=1): err= 0: pid=4808: Sat Oct 30 01:57:33 2021
write: IOPS=1052, BW=1052MiB/s (1104MB/s)(61.7GiB/60001msec)
slat (usec): min=12, max=1388, avg=35.04, stdev=16.39
clat (usec): min=528, max=50673, avg=910.62, stdev=506.20
lat (usec): min=557, max=50700, avg=945.66, stdev=506.89
clat percentiles (usec):
| 1.00th=[ 619], 5.00th=[ 676], 10.00th=[ 717], 20.00th=[ 766],
| 30.00th=[ 799], 40.00th=[ 832], 50.00th=[ 857], 60.00th=[ 889],
| 70.00th=[ 930], 80.00th=[ 979], 90.00th=[ 1106], 95.00th=[ 1254],
| 99.00th=[ 1811], 99.50th=[ 2278], 99.90th=[ 4686], 99.95th=[ 6521],
| 99.99th=[19006]
bw ( MiB/s): min= 594, max= 1310, per=99.96%, avg=1052.04, stdev=111.44, samples=119
iops : min= 594, max= 1310, avg=1052.02, stdev=111.43, samples=119
lat (usec) : 750=16.12%, 1000=66.01%
lat (msec) : 2=17.16%, 4=0.57%, 10=0.11%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%
cpu : usr=5.45%, sys=2.77%, ctx=63159, majf=0, minf=58
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,63147,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=1052MiB/s (1104MB/s), 1052MiB/s-1052MiB/s (1104MB/s-1104MB/s), io=61.7GiB (66.2GB), run=60001-60001msec
Disk stats (read/write):
sdb: ios=0/189450, merge=0/1, ticks=0/133484, in_queue=132230, util=84.23%
- Linux虚拟机(cache=writeback)
# 4KB随机读
random-read: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
random-read: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [r(1)][100.0%][r=46.7MiB/s,w=0KiB/s][r=11.0k,w=0 IOPS][eta 00m:00s]
random-read: (groupid=0, jobs=1): err= 0: pid=5154: Sat Oct 30 00:37:09 2021
read: IOPS=13.0k, BW=54.5MiB/s (57.2MB/s)(3273MiB/60002msec)
slat (nsec): min=424, max=2322.4k, avg=784.96, stdev=3164.24
clat (usec): min=155, max=48546, avg=1136.72, stdev=397.48
lat (usec): min=156, max=48548, avg=1137.50, stdev=397.53
clat percentiles (usec):
| 1.00th=[ 865], 5.00th=[ 898], 10.00th=[ 914], 20.00th=[ 947],
| 30.00th=[ 979], 40.00th=[ 1012], 50.00th=[ 1057], 60.00th=[ 1090],
| 70.00th=[ 1156], 80.00th=[ 1303], 90.00th=[ 1467], 95.00th=[ 1532],
| 99.00th=[ 2008], 99.50th=[ 2507], 99.90th=[ 5080], 99.95th=[ 7308],
| 99.99th=[12256]
bw ( KiB/s): min=39928, max=67048, per=99.99%, avg=55844.02, stdev=5142.90, samples=120
iops : min= 9982, max=16762, avg=13960.98, stdev=1285.72, samples=120
lat (usec) : 250=0.01%, 500=0.01%, 750=0.04%, 1000=36.06%
lat (msec) : 2=62.89%, 4=0.82%, 10=0.16%, 20=0.02%, 50=0.01%
cpu : usr=8.97%, sys=8.57%, ctx=415284, majf=1, minf=61
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.1%, 16=49.8%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.9%, 8=2.5%, 16=1.5%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=837810,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=54.5MiB/s (57.2MB/s), 54.5MiB/s-54.5MiB/s (57.2MB/s-57.2MB/s), io=3273MiB (3432MB), run=60002-60002msec
# 4KB随机写
Disk stats (read/write):
sdb: ios=836086/11, merge=0/7, ticks=47986/0, in_queue=47831, util=79.80%
random-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
random-write: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=29.8MiB/s][r=0,w=7624 IOPS][eta 00m:00s]
random-write: (groupid=0, jobs=1): err= 0: pid=5158: Sat Oct 30 00:38:10 2021
write: IOPS=8360, BW=32.7MiB/s (34.2MB/s)(1961MiB/60049msec)
slat (nsec): min=446, max=10314k, avg=1031.87, stdev=15008.45
clat (usec): min=158, max=72950, avg=1901.29, stdev=819.44
lat (usec): min=160, max=72951, avg=1902.32, stdev=819.63
clat percentiles (usec):
| 1.00th=[ 1237], 5.00th=[ 1319], 10.00th=[ 1385], 20.00th=[ 1483],
| 30.00th=[ 1598], 40.00th=[ 1729], 50.00th=[ 1860], 60.00th=[ 1926],
| 70.00th=[ 1975], 80.00th=[ 2040], 90.00th=[ 2278], 95.00th=[ 2737],
| 99.00th=[ 4817], 99.50th=[ 6063], 99.90th=[10683], 99.95th=[12911],
| 99.99th=[16909]
bw ( KiB/s): min=24336, max=41536, per=100.00%, avg=33463.48, stdev=3564.38, samples=120
iops : min= 6084, max=10384, avg=8365.84, stdev=891.10, samples=120
lat (usec) : 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.02%
lat (msec) : 2=74.38%, 4=23.96%, 10=1.53%, 20=0.11%, 50=0.01%
lat (msec) : 100=0.01%
cpu : usr=8.01%, sys=6.53%, ctx=248268, majf=0, minf=58
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.3%, 16=49.6%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=96.0%, 8=2.6%, 16=1.4%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,502034,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=32.7MiB/s (34.2MB/s), 32.7MiB/s-32.7MiB/s (34.2MB/s-34.2MB/s), io=1961MiB (2056MB), run=60049-60049msec
# 4KB顺序读
Disk stats (read/write):
sdb: ios=0/515160, merge=0/4, ticks=0/116065, in_queue=115875, util=72.02%
random-read: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [R(1)][100.0%][r=54.4MiB/s,w=0KiB/s][r=13.9k,w=0 IOPS][eta 00m:00s]
random-read: (groupid=0, jobs=1): err= 0: pid=5163: Sat Oct 30 00:39:11 2021
read: IOPS=13.1k, BW=51.3MiB/s (53.8MB/s)(3076MiB/60001msec)
slat (nsec): min=471, max=2159.4k, avg=875.32, stdev=4000.31
clat (usec): min=46, max=43071, avg=1210.09, stdev=539.79
lat (usec): min=211, max=43072, avg=1210.96, stdev=539.85
clat percentiles (usec):
| 1.00th=[ 857], 5.00th=[ 889], 10.00th=[ 914], 20.00th=[ 955],
| 30.00th=[ 996], 40.00th=[ 1037], 50.00th=[ 1090], 60.00th=[ 1172],
| 70.00th=[ 1303], 80.00th=[ 1434], 90.00th=[ 1516], 95.00th=[ 1647],
| 99.00th=[ 2671], 99.50th=[ 3654], 99.90th=[ 7570], 99.95th=[10028],
| 99.99th=[17171]
bw ( KiB/s): min=34760, max=65712, per=99.97%, avg=52480.80, stdev=6353.68, samples=120
iops : min= 8690, max=16428, avg=13120.16, stdev=1588.42, samples=120
lat (usec) : 50=0.01%, 100=0.01%, 250=0.01%, 500=0.01%, 750=0.07%
lat (usec) : 1000=30.60%
lat (msec) : 2=67.08%, 4=1.84%, 10=0.36%, 20=0.05%, 50=0.01%
cpu : usr=9.01%, sys=8.95%, ctx=388111, majf=0, minf=61
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.3%, 16=49.5%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.8%, 8=2.6%, 16=1.6%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=787495,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=51.3MiB/s (53.8MB/s), 51.3MiB/s-51.3MiB/s (53.8MB/s-53.8MB/s), io=3076MiB (3226MB), run=60001-60001msec
# 4KB顺序写
Disk stats (read/write):
sdb: ios=785914/24, merge=0/2088, ticks=47770/54, in_queue=47625, util=79.36%
random-write: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [F(1)][100.0%][r=0KiB/s,w=0KiB/s][r=0,w=0 IOPS][eta 00m:00s]
random-write: (groupid=0, jobs=1): err= 0: pid=5167: Sat Oct 30 00:40:16 2021
write: IOPS=8976, BW=35.1MiB/s (36.8MB/s)(2263MiB/64549msec)
slat (nsec): min=446, max=2998.0k, avg=925.01, stdev=4567.39
clat (usec): min=212, max=7198.2k, avg=1647.56, stdev=37821.31
lat (usec): min=224, max=7198.2k, avg=1648.49, stdev=37821.31
clat percentiles (usec):
| 1.00th=[ 1012], 5.00th=[ 1074], 10.00th=[ 1106], 20.00th=[ 1156],
| 30.00th=[ 1205], 40.00th=[ 1254], 50.00th=[ 1319], 60.00th=[ 1401],
| 70.00th=[ 1582], 80.00th=[ 1713], 90.00th=[ 1795], 95.00th=[ 1942],
| 99.00th=[ 3064], 99.50th=[ 4015], 99.90th=[ 9110], 99.95th=[12518],
| 99.99th=[24773]
bw ( KiB/s): min= 7025, max=53776, per=100.00%, avg=43316.14, stdev=7568.27, samples=107
iops : min= 1756, max=13444, avg=10829.04, stdev=1892.09, samples=107
lat (usec) : 250=0.01%, 750=0.01%, 1000=0.63%
lat (msec) : 2=95.48%, 4=3.38%, 10=0.42%, 20=0.06%, 50=0.02%
cpu : usr=6.19%, sys=5.83%, ctx=288082, majf=0, minf=59
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.1%, 16=49.8%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.7%, 8=2.2%, 16=2.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,579449,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=35.1MiB/s (36.8MB/s), 35.1MiB/s-35.1MiB/s (36.8MB/s-36.8MB/s), io=2263MiB (2373MB), run=64549-64549msec
# 1MB顺序读
Disk stats (read/write):
sdb: ios=0/579459, merge=0/0, ticks=0/123683, in_queue=123488, util=80.62%
seq-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=posixaio, iodepth=1
fio-3.7
Starting 1 process
seq-read: Laying out IO file (1 file / 16384MiB)
Jobs: 1 (f=1): [R(1)][100.0%][r=2463MiB/s,w=0KiB/s][r=2463,w=0 IOPS][eta 00m:00s]
seq-read: (groupid=0, jobs=1): err= 0: pid=5171: Sat Oct 30 00:41:46 2021
read: IOPS=2550, BW=2551MiB/s (2675MB/s)(149GiB/60001msec)
slat (usec): min=2, max=6584, avg= 8.53, stdev=22.98
clat (usec): min=242, max=29625, avg=379.48, stdev=271.90
lat (usec): min=252, max=29647, avg=388.01, stdev=273.60
clat percentiles (usec):
| 1.00th=[ 269], 5.00th=[ 285], 10.00th=[ 293], 20.00th=[ 310],
| 30.00th=[ 326], 40.00th=[ 338], 50.00th=[ 351], 60.00th=[ 367],
| 70.00th=[ 379], 80.00th=[ 400], 90.00th=[ 437], 95.00th=[ 502],
| 99.00th=[ 898], 99.50th=[ 1336], 99.90th=[ 4228], 99.95th=[ 6194],
| 99.99th=[10421]
bw ( MiB/s): min= 1748, max= 3012, per=99.98%, avg=2550.44, stdev=232.06, samples=120
iops : min= 1748, max= 3012, avg=2550.39, stdev=232.05, samples=120
lat (usec) : 250=0.01%, 500=94.89%, 750=3.62%, 1000=0.70%
lat (msec) : 2=0.48%, 4=0.19%, 10=0.10%, 20=0.01%, 50=0.01%
cpu : usr=4.57%, sys=8.60%, ctx=153106, majf=0, minf=61
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=153059,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=2551MiB/s (2675MB/s), 2551MiB/s-2551MiB/s (2675MB/s-2675MB/s), io=149GiB (160GB), run=60001-60001msec
# 1MB顺序写
Disk stats (read/write):
sdb: ios=459043/6, merge=0/2, ticks=107748/1, in_queue=105258, util=74.40%
seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=posixaio, iodepth=1
fio-3.7
Starting 1 process
seq-write: Laying out IO file (1 file / 16384MiB)
Jobs: 1 (f=1): [F(1)][100.0%][r=0KiB/s,w=0KiB/s][r=0,w=0 IOPS][eta 00m:00s]
seq-write: (groupid=0, jobs=1): err= 0: pid=5175: Sat Oct 30 00:42:58 2021
write: IOPS=1045, BW=1046MiB/s (1096MB/s)(73.3GiB/71818msec)
slat (usec): min=14, max=8756, avg=35.70, stdev=57.72
clat (usec): min=7, max=139859, avg=758.68, stdev=1543.97
lat (usec): min=428, max=139891, avg=794.38, stdev=1545.57
clat percentiles (usec):
| 1.00th=[ 449], 5.00th=[ 482], 10.00th=[ 498], 20.00th=[ 529],
| 30.00th=[ 553], 40.00th=[ 578], 50.00th=[ 611], 60.00th=[ 644],
| 70.00th=[ 685], 80.00th=[ 750], 90.00th=[ 873], 95.00th=[ 1057],
| 99.00th=[ 3589], 99.50th=[ 5538], 99.90th=[18220], 99.95th=[23200],
| 99.99th=[68682]
bw ( MiB/s): min= 542, max= 1852, per=100.00%, avg=1250.35, stdev=317.10, samples=119
iops : min= 542, max= 1852, avg=1250.30, stdev=317.12, samples=119
lat (usec) : 10=0.01%, 500=10.44%, 750=69.25%, 1000=14.47%
lat (msec) : 2=3.90%, 4=1.11%, 10=0.55%, 20=0.21%, 50=0.05%
lat (msec) : 100=0.01%, 250=0.01%
cpu : usr=6.34%, sys=3.38%, ctx=75158, majf=0, minf=59
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,75097,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=1046MiB/s (1096MB/s), 1046MiB/s-1046MiB/s (1096MB/s-1096MB/s), io=73.3GiB (78.7GB), run=71818-71818msec
Disk stats (read/write):
sdb: ios=0/225300, merge=0/1, ticks=0/212470, in_queue=211023, util=90.99%
- Linux虚拟机(cache=writeback,queues=4)
# 4KB随机读
random-read: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
random-read: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [r(1)][100.0%][r=126MiB/s,w=0KiB/s][r=32.2k,w=0 IOPS][eta 00m:00s]
random-read: (groupid=0, jobs=1): err= 0: pid=4808: Sat Oct 30 02:09:21 2021
read: IOPS=28.7k, BW=112MiB/s (117MB/s)(6717MiB/60001msec)
slat (nsec): min=418, max=774286, avg=648.17, stdev=1117.03
clat (usec): min=45, max=39467, avg=551.68, stdev=262.35
lat (usec): min=102, max=39469, avg=552.33, stdev=262.39
clat percentiles (usec):
| 1.00th=[ 412], 5.00th=[ 429], 10.00th=[ 437], 20.00th=[ 449],
| 30.00th=[ 461], 40.00th=[ 478], 50.00th=[ 494], 60.00th=[ 515],
| 70.00th=[ 545], 80.00th=[ 594], 90.00th=[ 742], 95.00th=[ 898],
| 99.00th=[ 1139], 99.50th=[ 1369], 99.90th=[ 2802], 99.95th=[ 4293],
| 99.99th=[ 8586]
bw ( KiB/s): min=82672, max=136023, per=99.98%, avg=114605.57, stdev=12051.84, samples=120
iops : min=20668, max=34005, avg=28651.37, stdev=3012.97, samples=120
lat (usec) : 50=0.01%, 250=0.01%, 500=53.60%, 750=36.73%, 1000=7.48%
lat (msec) : 2=2.00%, 4=0.14%, 10=0.05%, 20=0.01%, 50=0.01%
cpu : usr=10.62%, sys=13.94%, ctx=820177, majf=1, minf=61
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=49.9%, 16=50.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=96.2%, 8=3.0%, 16=0.8%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=1719435,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=112MiB/s (117MB/s), 112MiB/s-112MiB/s (117MB/s-117MB/s), io=6717MiB (7043MB), run=60001-60001msec
# 4KB随机写
Disk stats (read/write):
sdb: ios=1717107/7, merge=0/5, ticks=39989/0, in_queue=39689, util=66.18%
random-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
random-write: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=50.0MiB/s][r=0,w=12.8k IOPS][eta 00m:00s]
random-write: (groupid=0, jobs=1): err= 0: pid=4812: Sat Oct 30 02:10:22 2021
write: IOPS=12.2k, BW=47.5MiB/s (49.8MB/s)(2851MiB/60047msec)
slat (nsec): min=446, max=890642, avg=868.68, stdev=1701.53
clat (usec): min=101, max=209944, avg=1306.04, stdev=1963.58
lat (usec): min=447, max=209945, avg=1306.91, stdev=1963.60
clat percentiles (usec):
| 1.00th=[ 848], 5.00th=[ 906], 10.00th=[ 947], 20.00th=[ 996],
| 30.00th=[ 1045], 40.00th=[ 1106], 50.00th=[ 1172], 60.00th=[ 1303],
| 70.00th=[ 1418], 80.00th=[ 1467], 90.00th=[ 1549], 95.00th=[ 1680],
| 99.00th=[ 2900], 99.50th=[ 4359], 99.90th=[ 9634], 99.95th=[ 21103],
| 99.99th=[111674]
bw ( KiB/s): min=10072, max=61056, per=100.00%, avg=48655.44, stdev=7643.55, samples=120
iops : min= 2518, max=15264, avg=12163.83, stdev=1910.88, samples=120
lat (usec) : 250=0.01%, 500=0.01%, 750=0.03%, 1000=20.64%
lat (msec) : 2=76.81%, 4=1.91%, 10=0.51%, 20=0.04%, 50=0.02%
lat (msec) : 100=0.02%, 250=0.01%
cpu : usr=8.05%, sys=7.93%, ctx=362299, majf=0, minf=58
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.1%, 16=49.9%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.8%, 8=2.6%, 16=1.5%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,729934,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=47.5MiB/s (49.8MB/s), 47.5MiB/s-47.5MiB/s (49.8MB/s-49.8MB/s), io=2851MiB (2990MB), run=60047-60047msec
# 4KB顺序读
Disk stats (read/write):
sdb: ios=0/753722, merge=0/301, ticks=0/156690, in_queue=156482, util=69.53%
random-read: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [R(1)][100.0%][r=125MiB/s,w=0KiB/s][r=31.9k,w=0 IOPS][eta 00m:00s]
random-read: (groupid=0, jobs=1): err= 0: pid=4817: Sat Oct 30 02:11:22 2021
read: IOPS=29.6k, BW=116MiB/s (121MB/s)(6938MiB/60001msec)
slat (nsec): min=419, max=547504, avg=636.55, stdev=1095.32
clat (usec): min=28, max=48101, avg=534.43, stdev=269.89
lat (usec): min=39, max=48102, avg=535.07, stdev=269.94
clat percentiles (usec):
| 1.00th=[ 400], 5.00th=[ 412], 10.00th=[ 420], 20.00th=[ 433],
| 30.00th=[ 445], 40.00th=[ 457], 50.00th=[ 474], 60.00th=[ 494],
| 70.00th=[ 529], 80.00th=[ 578], 90.00th=[ 734], 95.00th=[ 873],
| 99.00th=[ 1090], 99.50th=[ 1369], 99.90th=[ 2999], 99.95th=[ 4146],
| 99.99th=[ 8160]
bw ( KiB/s): min=82568, max=143200, per=99.98%, avg=118380.02, stdev=13225.77, samples=119
iops : min=20642, max=35800, avg=29594.97, stdev=3306.45, samples=119
lat (usec) : 50=0.01%, 250=0.01%, 500=62.00%, 750=28.61%, 1000=7.84%
lat (msec) : 2=1.33%, 4=0.16%, 10=0.05%, 20=0.01%, 50=0.01%
cpu : usr=9.65%, sys=14.23%, ctx=852125, majf=0, minf=61
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.0%, 16=50.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=96.0%, 8=3.2%, 16=0.8%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=1776117,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=116MiB/s (121MB/s), 116MiB/s-116MiB/s (121MB/s-121MB/s), io=6938MiB (7275MB), run=60001-60001msec
# 4KB顺序写
Disk stats (read/write):
sdb: ios=1772977/62, merge=0/2004, ticks=39158/47, in_queue=38891, util=64.78%
random-write: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [F(1)][100.0%][r=0KiB/s,w=0KiB/s][r=0,w=0 IOPS][eta 00m:00s]
random-write: (groupid=0, jobs=1): err= 0: pid=4821: Sat Oct 30 02:12:43 2021
write: IOPS=11.2k, BW=43.8MiB/s (45.9MB/s)(3503MiB/79939msec)
slat (nsec): min=446, max=656366, avg=816.76, stdev=1500.08
clat (usec): min=163, max=14400k, avg=1062.44, stdev=60832.15
lat (usec): min=173, max=14400k, avg=1063.26, stdev=60832.15
clat percentiles (usec):
| 1.00th=[ 461], 5.00th=[ 502], 10.00th=[ 537], 20.00th=[ 578],
| 30.00th=[ 619], 40.00th=[ 660], 50.00th=[ 701], 60.00th=[ 758],
| 70.00th=[ 832], 80.00th=[ 971], 90.00th=[ 1172], 95.00th=[ 1287],
| 99.00th=[ 2114], 99.50th=[ 2671], 99.90th=[ 5211], 99.95th=[ 6915],
| 99.99th=[16909]
bw ( KiB/s): min=26360, max=106264, per=100.00%, avg=77976.51, stdev=15679.69, samples=92
iops : min= 6590, max=26566, avg=19494.12, stdev=3919.92, samples=92
lat (usec) : 250=0.01%, 500=4.65%, 750=54.58%, 1000=21.87%
lat (msec) : 2=17.75%, 4=0.96%, 10=0.16%, 20=0.02%, 50=0.01%
lat (msec) : 500=0.01%
cpu : usr=5.37%, sys=6.70%, ctx=429626, majf=0, minf=58
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.0%, 16=49.9%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.7%, 8=2.4%, 16=1.9%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,896763,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=43.8MiB/s (45.9MB/s), 43.8MiB/s-43.8MiB/s (45.9MB/s-45.9MB/s), io=3503MiB (3673MB), run=79939-79939msec
# 1MB顺序读
Disk stats (read/write):
sdb: ios=0/896774, merge=0/0, ticks=0/186488, in_queue=186261, util=79.67%
seq-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=posixaio, iodepth=1
fio-3.7
Starting 1 process
seq-read: Laying out IO file (1 file / 16384MiB)
Jobs: 1 (f=1): [R(1)][100.0%][r=2731MiB/s,w=0KiB/s][r=2730,w=0 IOPS][eta 00m:00s]
seq-read: (groupid=0, jobs=1): err= 0: pid=4827: Sat Oct 30 02:14:09 2021
read: IOPS=2931, BW=2932MiB/s (3074MB/s)(172GiB/60001msec)
slat (nsec): min=1939, max=3778.2k, avg=7671.60, stdev=10036.59
clat (usec): min=126, max=53500, avg=329.76, stdev=204.26
lat (usec): min=243, max=53517, avg=337.43, stdev=204.80
clat percentiles (usec):
| 1.00th=[ 258], 5.00th=[ 269], 10.00th=[ 277], 20.00th=[ 285],
| 30.00th=[ 297], 40.00th=[ 306], 50.00th=[ 318], 60.00th=[ 330],
| 70.00th=[ 343], 80.00th=[ 359], 90.00th=[ 383], 95.00th=[ 408],
| 99.00th=[ 515], 99.50th=[ 652], 99.90th=[ 1631], 99.95th=[ 2999],
| 99.99th=[ 5866]
bw ( MiB/s): min= 2198, max= 3214, per=99.98%, avg=2931.44, stdev=179.13, samples=120
iops : min= 2198, max= 3214, avg=2931.39, stdev=179.11, samples=120
lat (usec) : 250=0.21%, 500=98.67%, 750=0.75%, 1000=0.18%
lat (msec) : 2=0.11%, 4=0.05%, 10=0.02%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%
cpu : usr=4.24%, sys=7.87%, ctx=175932, majf=0, minf=61
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=175919,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=2932MiB/s (3074MB/s), 2932MiB/s-2932MiB/s (3074MB/s-3074MB/s), io=172GiB (184GB), run=60001-60001msec
# 1MB顺序写
Disk stats (read/write):
sdb: ios=526776/5, merge=0/3, ticks=104637/0, in_queue=101736, util=74.41%
seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=posixaio, iodepth=1
fio-3.7
Starting 1 process
seq-write: Laying out IO file (1 file / 16384MiB)
Jobs: 1 (f=1): [F(1)][100.0%][r=0KiB/s,w=0KiB/s][r=0,w=0 IOPS][eta 00m:00s]
seq-write: (groupid=0, jobs=1): err= 0: pid=4832: Sat Oct 30 02:15:25 2021
write: IOPS=1220, BW=1220MiB/s (1280MB/s)(90.0GiB/75522msec)
slat (usec): min=12, max=6901, avg=33.17, stdev=29.64
clat (usec): min=4, max=84036, avg=613.76, stdev=935.28
lat (usec): min=368, max=84056, avg=646.93, stdev=935.93
clat percentiles (usec):
| 1.00th=[ 388], 5.00th=[ 412], 10.00th=[ 433], 20.00th=[ 461],
| 30.00th=[ 482], 40.00th=[ 502], 50.00th=[ 523], 60.00th=[ 545],
| 70.00th=[ 586], 80.00th=[ 668], 90.00th=[ 758], 95.00th=[ 840],
| 99.00th=[ 1893], 99.50th=[ 3359], 99.90th=[13698], 99.95th=[20579],
| 99.99th=[36439]
bw ( MiB/s): min= 672, max= 2184, per=100.00%, avg=1536.11, stdev=367.42, samples=120
iops : min= 672, max= 2184, avg=1536.09, stdev=367.42, samples=120
lat (usec) : 10=0.01%, 500=39.34%, 750=50.19%, 1000=8.44%
lat (msec) : 2=1.08%, 4=0.60%, 10=0.22%, 20=0.07%, 50=0.05%
lat (msec) : 100=0.01%
cpu : usr=5.75%, sys=3.13%, ctx=92204, majf=0, minf=59
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,92174,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=1220MiB/s (1280MB/s), 1220MiB/s-1220MiB/s (1280MB/s-1280MB/s), io=90.0GiB (96.7GB), run=75522-75522msec
Disk stats (read/write):
sdb: ios=0/276525, merge=0/0, ticks=0/182473, in_queue=181017, util=96.13%
- Windows虚拟机(cache=none)
# 4KB随机读
D:\>c:\fio\fio\fio --name=random-read --ioengine=windowsaio --rw=randread --bs=4k --size=4g --numjobs=1 --iodepth=16 --runtime=60 --time_based --direct=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
random-read: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=windowsaio, iodepth=16
fio-3.27
Starting 1 thread
random-read: (groupid=0, jobs=1): err= 0: pid=2072: Sat Oct 30 02:40:25 2021
read: IOPS=2811, BW=11.0MiB/s (11.5MB/s)(659MiB/60015msec)
slat (usec): min=10, max=5373, avg=25.68, stdev=25.09
clat (usec): min=11, max=73724, avg=5621.21, stdev=5224.92
lat (usec): min=103, max=73749, avg=5646.90, stdev=5224.95
clat percentiles (usec):
| 1.00th=[ 128], 5.00th=[ 163], 10.00th=[ 206], 20.00th=[ 404],
| 30.00th=[ 2409], 40.00th=[ 3687], 50.00th=[ 4883], 60.00th=[ 6128],
| 70.00th=[ 7373], 80.00th=[ 8586], 90.00th=[11076], 95.00th=[15533],
| 99.00th=[24511], 99.50th=[28967], 99.90th=[40109], 99.95th=[43779],
| 99.99th=[56361]
bw ( KiB/s): min= 9635, max=12200, per=100.00%, avg=11257.24, stdev=457.67, samples=119
iops : min= 2408, max= 3050, avg=2813.85, stdev=114.34, samples=119
lat (usec) : 20=0.01%, 50=0.01%, 100=0.10%, 250=14.68%, 500=5.68%
lat (usec) : 750=0.51%, 1000=0.37%
lat (msec) : 2=5.78%, 4=15.49%, 10=45.38%, 20=9.97%, 50=2.01%
lat (msec) : 100=0.02%
cpu : usr=0.00%, sys=16.66%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=16.1%, 16=83.8%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=168728,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=11.0MiB/s (11.5MB/s), 11.0MiB/s-11.0MiB/s (11.5MB/s-11.5MB/s), io=659MiB (691MB), run=60015-60015msec
# 4KB随机写
D:\>c:\fio\fio\fio --name=random-write --ioengine=windowsaio --rw=randwrite --bs=4k --size=4g --numjobs=1 --iodepth=16 --runtime=60 --time_based --direct=1 --end_fsync=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
random-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=windowsaio, iodepth=16
fio-3.27
Starting 1 thread
random-write: Laying out IO file (1 file / 4096MiB)
random-write: (groupid=0, jobs=1): err= 0: pid=2456: Sat Oct 30 02:41:26 2021
write: IOPS=6002, BW=23.4MiB/s (24.6MB/s)(1407MiB/60002msec); 0 zone resets
slat (usec): min=9, max=3977.2k, avg=46.96, stdev=7329.99
clat (usec): min=11, max=10631k, avg=1894.52, stdev=19237.01
lat (usec): min=126, max=11336k, avg=1953.41, stdev=26987.94
clat percentiles (usec):
| 1.00th=[ 219], 5.00th=[ 404], 10.00th=[ 586], 20.00th=[ 832],
| 30.00th=[ 1012], 40.00th=[ 1156], 50.00th=[ 1336], 60.00th=[ 1582],
| 70.00th=[ 2008], 80.00th=[ 2671], 90.00th=[ 3720], 95.00th=[ 4686],
| 99.00th=[ 6783], 99.50th=[ 7767], 99.90th=[12125], 99.95th=[15401],
| 99.99th=[33817]
bw ( KiB/s): min= 100, max=44498, per=100.00%, avg=29609.35, stdev=4988.31, samples=96
iops : min= 25, max=11124, avg=7401.96, stdev=1247.04, samples=96
lat (usec) : 20=0.01%, 50=0.01%, 100=0.01%, 250=1.56%, 500=5.94%
lat (usec) : 750=8.75%, 1000=13.06%
lat (msec) : 2=40.57%, 4=21.97%, 10=7.96%, 20=0.16%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%, 1000=0.01%, 2000=0.01%, >=2000=0.01%
cpu : usr=0.00%, sys=46.67%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=1.5%, 4=8.9%, 8=58.0%, 16=31.5%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.5%, 8=2.2%, 16=2.3%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,360164,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=23.4MiB/s (24.6MB/s), 23.4MiB/s-23.4MiB/s (24.6MB/s-24.6MB/s), io=1407MiB (1475MB), run=60002-60002msec
# 4KB顺序读
D:\>c:\fio\fio\fio --name=random-read --ioengine=windowsaio --rw=read --bs=4k --size=4g --numjobs=1 --iodepth=16 --runtime=60 --time_based --direct=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
random-read: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=windowsaio, iodepth=16
fio-3.27
Starting 1 thread
random-read: (groupid=0, jobs=1): err= 0: pid=1312: Sat Oct 30 02:42:26 2021
read: IOPS=11.8k, BW=46.1MiB/s (48.3MB/s)(2766MiB/60001msec)
slat (usec): min=9, max=3359, avg=25.73, stdev=18.48
clat (usec): min=9, max=37137, avg=758.01, stdev=583.93
lat (usec): min=109, max=37162, avg=783.75, stdev=584.80
clat percentiles (usec):
| 1.00th=[ 147], 5.00th=[ 208], 10.00th=[ 262], 20.00th=[ 375],
| 30.00th=[ 490], 40.00th=[ 603], 50.00th=[ 717], 60.00th=[ 824],
| 70.00th=[ 938], 80.00th=[ 1057], 90.00th=[ 1237], 95.00th=[ 1401],
| 99.00th=[ 1663], 99.50th=[ 2343], 99.90th=[ 6783], 99.95th=[10421],
| 99.99th=[16909]
bw ( KiB/s): min=32996, max=54700, per=100.00%, avg=47296.58, stdev=3546.02, samples=118
iops : min= 8249, max=13675, avg=11823.78, stdev=886.47, samples=118
lat (usec) : 10=0.01%, 20=0.01%, 50=0.01%, 100=0.05%, 250=8.94%
lat (usec) : 500=21.86%, 750=22.33%, 1000=22.27%
lat (msec) : 2=23.95%, 4=0.33%, 10=0.21%, 20=0.05%, 50=0.01%
cpu : usr=5.00%, sys=90.00%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=7.1%, 4=28.2%, 8=57.3%, 16=7.3%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=93.3%, 8=0.1%, 16=6.6%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=708142,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=46.1MiB/s (48.3MB/s), 46.1MiB/s-46.1MiB/s (48.3MB/s-48.3MB/s), io=2766MiB (2901MB), run=60001-60001msec
# 4KB顺序写
D:\>c:\fio\fio\fio --name=random-write --ioengine=windowsaio --rw=write --bs=4k --size=4g --numjobs=1 --iodepth=16 --runtime=60 --time_based --direct=1 --end_fsync=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
random-write: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=windowsaio, iodepth=16
fio-3.27
Starting 1 thread
random-write: Laying out IO file (1 file / 4096MiB)
random-write: (groupid=0, jobs=1): err= 0: pid=2540: Sat Oct 30 02:43:27 2021
write: IOPS=4289, BW=16.8MiB/s (17.6MB/s)(1006MiB/60010msec); 0 zone resets
slat (usec): min=134, max=22745, avg=210.38, stdev=95.53
clat (usec): min=6, max=34614, avg=1859.32, stdev=1074.17
lat (usec): min=152, max=39000, avg=2069.70, stdev=1083.80
clat percentiles (usec):
| 1.00th=[ 198], 5.00th=[ 247], 10.00th=[ 449], 20.00th=[ 799],
| 30.00th=[ 1139], 40.00th=[ 1483], 50.00th=[ 1827], 60.00th=[ 2180],
| 70.00th=[ 2540], 80.00th=[ 2868], 90.00th=[ 3228], 95.00th=[ 3490],
| 99.00th=[ 4113], 99.50th=[ 4490], 99.90th=[ 6194], 99.95th=[ 6915],
| 99.99th=[15401]
bw ( KiB/s): min=13314, max=18292, per=100.00%, avg=17174.08, stdev=784.88, samples=119
iops : min= 3328, max= 4573, avg=4293.07, stdev=196.15, samples=119
lat (usec) : 10=0.35%, 20=0.15%, 50=0.01%, 100=0.01%, 250=4.71%
lat (usec) : 500=6.92%, 750=6.75%, 1000=7.09%
lat (msec) : 2=28.86%, 4=43.93%, 10=1.22%, 20=0.02%, 50=0.01%
cpu : usr=1.67%, sys=18.33%, ctx=0, majf=0, minf=0
IO depths : 1=0.5%, 2=13.3%, 4=26.5%, 8=53.1%, 16=6.6%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=93.8%, 8=0.1%, 16=6.2%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,257440,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=16.8MiB/s (17.6MB/s), 16.8MiB/s-16.8MiB/s (17.6MB/s-17.6MB/s), io=1006MiB (1054MB), run=60010-60010msec
# 1MB顺序读
D:\>c:\fio\fio\fio --name=seq-read --ioengine=windowsaio --rw=read --bs=1m --size=16g --numjobs=1 --iodepth=1 --runtime=60 --time_based --direct=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
seq-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=windowsaio, iodepth=1
fio-3.27
Starting 1 thread
seq-read: Laying out IO file (1 file / 16384MiB)
seq-read: (groupid=0, jobs=1): err= 0: pid=1500: Sat Oct 30 02:45:12 2021
read: IOPS=1009, BW=1009MiB/s (1058MB/s)(59.1GiB/59992msec)
slat (usec): min=38, max=15026, avg=438.79, stdev=230.48
clat (usec): min=19, max=60713, avg=522.65, stdev=1027.99
lat (usec): min=496, max=61028, avg=961.44, stdev=1018.88
clat percentiles (usec):
| 1.00th=[ 165], 5.00th=[ 192], 10.00th=[ 206], 20.00th=[ 227],
| 30.00th=[ 247], 40.00th=[ 297], 50.00th=[ 400], 60.00th=[ 494],
| 70.00th=[ 578], 80.00th=[ 693], 90.00th=[ 898], 95.00th=[ 1012],
| 99.00th=[ 1860], 99.50th=[ 2704], 99.90th=[13173], 99.95th=[24249],
| 99.99th=[41157]
bw ( KiB/s): min=75624, max=1410490, per=100.00%, avg=1034463.46, stdev=127259.41, samples=117
iops : min= 73, max= 1377, avg=1009.67, stdev=124.33, samples=117
lat (usec) : 20=0.01%, 50=0.02%, 100=0.05%, 250=30.69%, 500=29.89%
lat (usec) : 750=21.90%, 1000=12.10%
lat (msec) : 2=4.44%, 4=0.60%, 10=0.18%, 20=0.07%, 50=0.06%
lat (msec) : 100=0.01%
cpu : usr=1.67%, sys=36.67%, ctx=0, majf=0, minf=0
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=60558,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=1009MiB/s (1058MB/s), 1009MiB/s-1009MiB/s (1058MB/s-1058MB/s), io=59.1GiB (63.5GB), run=59992-59992msec
# 1MB顺序写
D:\>c:\fio\fio\fio --name=seq-write --ioengine=windowsaio --rw=write --bs=1m --size=16g --numjobs=1 --iodepth=1 --runtime=60 --time_based --direct=1 --end_fsync=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=windowsaio, iodepth=1
fio-3.27
Starting 1 thread
seq-write: Laying out IO file (1 file / 16384MiB)
seq-write: (groupid=0, jobs=1): err= 0: pid=2548: Sat Oct 30 02:46:13 2021
write: IOPS=685, BW=686MiB/s (719MB/s)(40.2GiB/60002msec); 0 zone resets
slat (usec): min=45, max=17742, avg=1136.73, stdev=1006.24
clat (usec): min=6, max=33881, avg=298.00, stdev=352.03
lat (usec): min=586, max=40335, avg=1434.73, stdev=871.99
clat percentiles (usec):
| 1.00th=[ 12], 5.00th=[ 34], 10.00th=[ 38], 20.00th=[ 45],
| 30.00th=[ 54], 40.00th=[ 194], 50.00th=[ 281], 60.00th=[ 318],
| 70.00th=[ 355], 80.00th=[ 412], 90.00th=[ 750], 95.00th=[ 873],
| 99.00th=[ 1188], 99.50th=[ 1401], 99.90th=[ 2999], 99.95th=[ 3458],
| 99.99th=[ 6521]
bw ( KiB/s): min=336684, max=1247128, per=99.67%, avg=700054.70, stdev=343978.89, samples=116
iops : min= 328, max= 1217, avg=683.12, stdev=335.96, samples=116
lat (usec) : 10=0.59%, 20=1.12%, 50=25.39%, 100=12.28%, 250=4.27%
lat (usec) : 500=41.26%, 750=5.22%, 1000=7.52%
lat (msec) : 2=2.16%, 4=0.16%, 10=0.03%, 20=0.01%, 50=0.01%
cpu : usr=0.00%, sys=21.67%, ctx=0, majf=0, minf=0
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,41155,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=686MiB/s (719MB/s), 686MiB/s-686MiB/s (719MB/s-719MB/s), io=40.2GiB (43.2GB), run=60002-60002msec
- Windows虚拟机(cache=none,queues=4)
# 4KB随机读
D:\>c:\fio\fio\fio --name=random-read --ioengine=windowsaio --rw=randread --bs=4k --size=4g --numjobs=1 --iodepth=16 --runtime=60 --time_based --direct=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
random-read: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=windowsaio, iodepth=16
fio-3.27
Starting 1 thread
random-read: (groupid=0, jobs=1): err= 0: pid=540: Sat Oct 30 03:56:43 2021
read: IOPS=2794, BW=10.9MiB/s (11.4MB/s)(655MiB/60013msec)
slat (usec): min=10, max=19968, avg=22.98, stdev=55.80
clat (usec): min=17, max=74355, avg=5669.51, stdev=5362.47
lat (usec): min=65, max=74370, avg=5692.49, stdev=5362.73
clat percentiles (usec):
| 1.00th=[ 93], 5.00th=[ 119], 10.00th=[ 155], 20.00th=[ 375],
| 30.00th=[ 2376], 40.00th=[ 3654], 50.00th=[ 4883], 60.00th=[ 6194],
| 70.00th=[ 7439], 80.00th=[ 8586], 90.00th=[11469], 95.00th=[15926],
| 99.00th=[25035], 99.50th=[29754], 99.90th=[41157], 99.95th=[46400],
| 99.99th=[56361]
bw ( KiB/s): min= 9105, max=12389, per=100.00%, avg=11191.02, stdev=483.54, samples=118
iops : min= 2276, max= 3097, avg=2797.34, stdev=120.82, samples=118
lat (usec) : 20=0.01%, 50=0.01%, 100=1.96%, 250=16.53%, 500=1.96%
lat (usec) : 750=0.36%, 1000=0.42%
lat (msec) : 2=6.03%, 4=15.47%, 10=44.79%, 20=10.23%, 50=2.22%
lat (msec) : 100=0.03%
cpu : usr=0.00%, sys=15.00%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=13.0%, 16=86.9%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=99.9%, 8=0.1%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=167713,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=10.9MiB/s (11.4MB/s), 10.9MiB/s-10.9MiB/s (11.4MB/s-11.4MB/s), io=655MiB (687MB), run=60013-60013msec
# 4KB随机写
D:\>c:\fio\fio\fio --name=random-write --ioengine=windowsaio --rw=randwrite --bs=4k --size=4g --numjobs=1 --iodepth=16 --runtime=60 --time_based --direct=1 --end_fsync=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
random-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=windowsaio, iodepth=16
fio-3.27
Starting 1 thread
random-write: (groupid=0, jobs=1): err= 0: pid=560: Sat Oct 30 03:57:43 2021
write: IOPS=7448, BW=29.1MiB/s (30.5MB/s)(1746MiB/60004msec); 0 zone resets
slat (usec): min=10, max=10490, avg=20.82, stdev=39.04
clat (usec): min=8, max=25743, avg=2033.13, stdev=1681.68
lat (usec): min=90, max=25772, avg=2053.95, stdev=1682.24
clat percentiles (usec):
| 1.00th=[ 237], 5.00th=[ 420], 10.00th=[ 553], 20.00th=[ 775],
| 30.00th=[ 971], 40.00th=[ 1205], 50.00th=[ 1500], 60.00th=[ 1893],
| 70.00th=[ 2409], 80.00th=[ 3130], 90.00th=[ 4228], 95.00th=[ 5211],
| 99.00th=[ 7635], 99.50th=[ 8979], 99.90th=[14222], 99.95th=[16581],
| 99.99th=[21890]
bw ( KiB/s): min=21431, max=101105, per=100.00%, avg=29820.11, stdev=6928.42, samples=117
iops : min= 5357, max=25276, avg=7454.65, stdev=1732.10, samples=117
lat (usec) : 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01%, 250=1.16%
lat (usec) : 500=6.64%, 750=10.90%, 1000=12.50%
lat (msec) : 2=31.15%, 4=26.13%, 10=11.17%, 20=0.32%, 50=0.02%
cpu : usr=0.00%, sys=35.00%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=0.3%, 4=2.1%, 8=49.2%, 16=48.4%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=97.6%, 8=1.7%, 16=0.7%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,446927,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=29.1MiB/s (30.5MB/s), 29.1MiB/s-29.1MiB/s (30.5MB/s-30.5MB/s), io=1746MiB (1831MB), run=60004-60004msec
# 4KB顺序读
D:\>c:\fio\fio\fio --name=random-read --ioengine=windowsaio --rw=read --bs=4k --size=4g --numjobs=1 --iodepth=16 --runtime=60 --time_based --direct=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
random-read: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=windowsaio, iodepth=16
fio-3.27
Starting 1 thread
random-read: (groupid=0, jobs=1): err= 0: pid=1128: Sat Oct 30 03:58:44 2021
read: IOPS=21.1k, BW=82.6MiB/s (86.6MB/s)(4956MiB/60002msec)
slat (usec): min=9, max=15198, avg=19.43, stdev=22.72
clat (usec): min=9, max=34197, avg=433.84, stdev=613.35
lat (usec): min=57, max=34209, avg=453.27, stdev=614.34
clat percentiles (usec):
| 1.00th=[ 82], 5.00th=[ 121], 10.00th=[ 155], 20.00th=[ 210],
| 30.00th=[ 269], 40.00th=[ 326], 50.00th=[ 383], 60.00th=[ 441],
| 70.00th=[ 498], 80.00th=[ 570], 90.00th=[ 685], 95.00th=[ 791],
| 99.00th=[ 1045], 99.50th=[ 2073], 99.90th=[ 9110], 99.95th=[13698],
| 99.99th=[22938]
bw ( KiB/s): min=43595, max=103126, per=100.00%, avg=84693.01, stdev=9673.83, samples=118
iops : min=10898, max=25781, avg=21171.96, stdev=2418.17, samples=118
lat (usec) : 10=0.01%, 20=0.01%, 50=0.01%, 100=2.68%, 250=24.04%
lat (usec) : 500=43.43%, 750=23.21%, 1000=5.48%
lat (msec) : 2=0.65%, 4=0.20%, 10=0.22%, 20=0.07%, 50=0.01%
cpu : usr=3.33%, sys=83.33%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=4.7%, 4=27.7%, 8=60.0%, 16=7.6%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=93.0%, 8=0.1%, 16=6.9%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=1268693,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=82.6MiB/s (86.6MB/s), 82.6MiB/s-82.6MiB/s (86.6MB/s-86.6MB/s), io=4956MiB (5197MB), run=60002-60002msec
# 4KB顺序写
D:\>c:\fio\fio\fio --name=random-write --ioengine=windowsaio --rw=write --bs=4k --size=4g --numjobs=1 --iodepth=16 --runtime=60 --time_based --direct=1 --end_fsync=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
random-write: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=windowsaio, iodepth=16
fio-3.27
Starting 1 thread
random-write: (groupid=0, jobs=1): err= 0: pid=1536: Sat Oct 30 03:59:45 2021
write: IOPS=25.5k, BW=99.7MiB/s (105MB/s)(5983MiB/60001msec); 0 zone resets
slat (usec): min=10, max=14796, avg=16.13, stdev=22.73
clat (usec): min=9, max=30672, avg=367.49, stdev=231.32
lat (usec): min=59, max=30770, avg=383.62, stdev=233.71
clat percentiles (usec):
| 1.00th=[ 100], 5.00th=[ 131], 10.00th=[ 159], 20.00th=[ 208],
| 30.00th=[ 258], 40.00th=[ 306], 50.00th=[ 355], 60.00th=[ 400],
| 70.00th=[ 449], 80.00th=[ 494], 90.00th=[ 570], 95.00th=[ 668],
| 99.00th=[ 840], 99.50th=[ 930], 99.90th=[ 1614], 99.95th=[ 2474],
| 99.99th=[ 6194]
bw ( KiB/s): min=79337, max=116184, per=100.00%, avg=102242.19, stdev=6398.98, samples=118
iops : min=19834, max=29046, avg=25560.14, stdev=1599.72, samples=118
lat (usec) : 10=0.01%, 20=0.01%, 50=0.01%, 100=1.07%, 250=27.07%
lat (usec) : 500=53.16%, 750=16.33%, 1000=2.00%
lat (msec) : 2=0.30%, 4=0.04%, 10=0.02%, 20=0.01%, 50=0.01%
cpu : usr=0.00%, sys=95.00%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=1.7%, 4=25.8%, 8=64.1%, 16=8.4%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=92.5%, 8=0.5%, 16=7.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,1531734,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=99.7MiB/s (105MB/s), 99.7MiB/s-99.7MiB/s (105MB/s-105MB/s), io=5983MiB (6274MB), run=60001-60001msec
# 1MB顺序读
D:\>c:\fio\fio\fio --name=seq-read --ioengine=windowsaio --rw=read --bs=1m --size=16g --numjobs=1 --iodepth=1 --runtime=60 --time_based --direct=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
seq-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=windowsaio, iodepth=1
fio-3.27
Starting 1 thread
seq-read: (groupid=0, jobs=1): err= 0: pid=1432: Sat Oct 30 04:00:45 2021
read: IOPS=930, BW=930MiB/s (975MB/s)(54.5GiB/60001msec)
slat (usec): min=41, max=3057, avg=62.87, stdev=29.97
clat (usec): min=38, max=62841, avg=987.57, stdev=1620.63
lat (usec): min=501, max=62898, avg=1050.44, stdev=1623.78
clat percentiles (usec):
| 1.00th=[ 523], 5.00th=[ 570], 10.00th=[ 611], 20.00th=[ 668],
| 30.00th=[ 742], 40.00th=[ 807], 50.00th=[ 848], 60.00th=[ 889],
| 70.00th=[ 930], 80.00th=[ 971], 90.00th=[ 1156], 95.00th=[ 1336],
| 99.00th=[ 3130], 99.50th=[ 9765], 99.90th=[26870], 99.95th=[38011],
| 99.99th=[48497]
bw ( KiB/s): min=442272, max=1111657, per=100.00%, avg=953627.81, stdev=152555.97, samples=118
iops : min= 431, max= 1085, avg=930.76, stdev=149.05, samples=118
lat (usec) : 50=0.01%, 250=0.01%, 500=0.31%, 750=30.91%, 1000=52.41%
lat (msec) : 2=14.64%, 4=0.83%, 10=0.41%, 20=0.29%, 50=0.18%
lat (msec) : 100=0.01%
cpu : usr=0.00%, sys=8.33%, ctx=0, majf=0, minf=0
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=55819,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=930MiB/s (975MB/s), 930MiB/s-930MiB/s (975MB/s-975MB/s), io=54.5GiB (58.5GB), run=60001-60001msec
# 1MB顺序写
D:\>c:\fio\fio\fio --name=seq-write --ioengine=windowsaio --rw=write --bs=1m --size=16g --numjobs=1 --iodepth=1 --runtime=60 --time_based --direct=1 --end_fsync=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=windowsaio, iodepth=1
fio-3.27
Starting 1 thread
seq-write: (groupid=0, jobs=1): err= 0: pid=1328: Sat Oct 30 04:01:46 2021
write: IOPS=1013, BW=1014MiB/s (1063MB/s)(59.4GiB/60001msec); 0 zone resets
slat (usec): min=44, max=3181, avg=80.72, stdev=41.64
clat (usec): min=22, max=51315, avg=881.05, stdev=487.72
lat (usec): min=505, max=51424, avg=961.77, stdev=492.14
clat percentiles (usec):
| 1.00th=[ 562], 5.00th=[ 611], 10.00th=[ 644], 20.00th=[ 693],
| 30.00th=[ 742], 40.00th=[ 775], 50.00th=[ 807], 60.00th=[ 848],
| 70.00th=[ 906], 80.00th=[ 979], 90.00th=[ 1106], 95.00th=[ 1270],
| 99.00th=[ 2245], 99.50th=[ 2966], 99.90th=[ 5932], 99.95th=[ 8225],
| 99.99th=[15533]
bw ( KiB/s): min=364575, max=1290773, per=100.00%, avg=1039189.05, stdev=128733.87, samples=118
iops : min= 356, max= 1260, avg=1014.40, stdev=125.76, samples=118
lat (usec) : 50=0.01%, 250=0.01%, 500=0.08%, 750=33.40%, 1000=48.93%
lat (msec) : 2=16.30%, 4=1.03%, 10=0.23%, 20=0.02%, 50=0.01%
lat (msec) : 100=0.01%
cpu : usr=1.67%, sys=10.00%, ctx=0, majf=0, minf=0
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,60822,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=1014MiB/s (1063MB/s), 1014MiB/s-1014MiB/s (1063MB/s-1063MB/s), io=59.4GiB (63.8GB), run=60001-60001msec
- Windows虚拟机(cache=writeback)
# 4KB随机读
D:\>c:\fio\fio\fio --name=random-read --ioengine=windowsaio --rw=randread --bs=4k --size=4g --numjobs=1 --iodepth=16 --runtime=60 --time_based --direct=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
random-read: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=windowsaio, iodepth=16
fio-3.27
Starting 1 thread
random-read: (groupid=0, jobs=1): err= 0: pid=1860: Sat Oct 30 03:14:35 2021
read: IOPS=2483, BW=9935KiB/s (10.2MB/s)(582MiB/60021msec)
slat (usec): min=10, max=5108, avg=25.73, stdev=28.44
clat (usec): min=11, max=82816, avg=6379.55, stdev=5707.01
lat (usec): min=66, max=82842, avg=6405.28, stdev=5707.04
clat percentiles (usec):
| 1.00th=[ 82], 5.00th=[ 121], 10.00th=[ 215], 20.00th=[ 1647],
| 30.00th=[ 3097], 40.00th=[ 4359], 50.00th=[ 5604], 60.00th=[ 6783],
| 70.00th=[ 7963], 80.00th=[ 9241], 90.00th=[13173], 95.00th=[16909],
| 99.00th=[26608], 99.50th=[32113], 99.90th=[42730], 99.95th=[48497],
| 99.99th=[58983]
bw ( KiB/s): min= 7929, max=11486, per=100.00%, avg=9944.61, stdev=582.24, samples=119
iops : min= 1982, max= 2871, avg=2485.75, stdev=145.63, samples=119
lat (usec) : 20=0.01%, 50=0.02%, 100=2.82%, 250=9.50%, 500=3.95%
lat (usec) : 750=0.32%, 1000=0.27%
lat (msec) : 2=5.28%, 4=14.95%, 10=47.00%, 20=12.91%, 50=2.94%
lat (msec) : 100=0.04%
cpu : usr=0.00%, sys=15.00%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=12.3%, 16=87.6%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=149075,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=9935KiB/s (10.2MB/s), 9935KiB/s-9935KiB/s (10.2MB/s-10.2MB/s), io=582MiB (611MB), run=60021-60021msec
# 4KB随机写
D:\>c:\fio\fio\fio --name=random-write --ioengine=windowsaio --rw=randwrite --bs=4k --size=4g --numjobs=1 --iodepth=16 --runtime=60 --time_based --direct=1 --end_fsync=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
random-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=windowsaio, iodepth=16
fio-3.27
Starting 1 thread
random-write: Laying out IO file (1 file / 4096MiB)
random-write: (groupid=0, jobs=1): err= 0: pid=1644: Sat Oct 30 03:15:40 2021
write: IOPS=7430, BW=29.0MiB/s (30.4MB/s)(1876MiB/64650msec); 0 zone resets
slat (usec): min=9, max=1857.7k, avg=40.31, stdev=4122.89
clat (usec): min=8, max=10449k, avg=1079.48, stdev=55621.16
lat (usec): min=73, max=10449k, avg=1146.62, stdev=58826.07
clat percentiles (usec):
| 1.00th=[ 95], 5.00th=[ 161], 10.00th=[ 231], 20.00th=[ 367],
| 30.00th=[ 478], 40.00th=[ 586], 50.00th=[ 693], 60.00th=[ 799],
| 70.00th=[ 906], 80.00th=[ 1045], 90.00th=[ 1237], 95.00th=[ 1385],
| 99.00th=[ 2180], 99.50th=[ 3556], 99.90th=[ 9896], 99.95th=[13566],
| 99.99th=[52691]
bw ( KiB/s): min= 200, max=63326, per=100.00%, avg=45994.33, stdev=8566.10, samples=82
iops : min= 50, max=15831, avg=11498.26, stdev=2141.49, samples=82
lat (usec) : 10=0.01%, 20=0.01%, 50=0.01%, 100=1.17%, 250=10.12%
lat (usec) : 500=20.87%, 750=23.48%, 1000=21.39%
lat (msec) : 2=21.75%, 4=0.80%, 10=0.33%, 20=0.08%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 2000=0.01%
lat (msec) : >=2000=0.01%
cpu : usr=0.00%, sys=54.14%, ctx=0, majf=0, minf=0
IO depths : 1=0.7%, 2=10.3%, 4=26.0%, 8=55.6%, 16=7.4%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=93.5%, 8=0.3%, 16=6.2%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,480369,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=29.0MiB/s (30.4MB/s), 29.0MiB/s-29.0MiB/s (30.4MB/s-30.4MB/s), io=1876MiB (1968MB), run=64650-64650msec
# 4KB顺序读
D:\>c:\fio\fio\fio --name=random-read --ioengine=windowsaio --rw=read --bs=4k --size=4g --numjobs=1 --iodepth=16 --runtime=60 --time_based --direct=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
random-read: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=windowsaio, iodepth=16
fio-3.27
Starting 1 thread
random-read: (groupid=0, jobs=1): err= 0: pid=1708: Sat Oct 30 03:16:40 2021
read: IOPS=10.9k, BW=42.8MiB/s (44.9MB/s)(2567MiB/60001msec)
slat (usec): min=9, max=16037, avg=30.37, stdev=80.30
clat (usec): min=8, max=53075, avg=822.29, stdev=1068.55
lat (usec): min=70, max=53099, avg=852.66, stdev=1073.29
clat percentiles (usec):
| 1.00th=[ 87], 5.00th=[ 147], 10.00th=[ 223], 20.00th=[ 367],
| 30.00th=[ 486], 40.00th=[ 603], 50.00th=[ 717], 60.00th=[ 832],
| 70.00th=[ 955], 80.00th=[ 1090], 90.00th=[ 1303], 95.00th=[ 1483],
| 99.00th=[ 3326], 99.50th=[ 5800], 99.90th=[16188], 99.95th=[20579],
| 99.99th=[35914]
bw ( KiB/s): min=16779, max=59060, per=100.00%, avg=43863.93, stdev=7149.84, samples=117
iops : min= 4194, max=14765, avg=10965.62, stdev=1787.44, samples=117
lat (usec) : 10=0.01%, 20=0.02%, 50=0.01%, 100=1.53%, 250=10.27%
lat (usec) : 500=19.37%, 750=21.75%, 1000=20.61%
lat (msec) : 2=24.39%, 4=1.25%, 10=0.56%, 20=0.19%, 50=0.06%
lat (msec) : 100=0.01%
cpu : usr=3.33%, sys=73.33%, ctx=0, majf=0, minf=0
IO depths : 1=0.4%, 2=10.8%, 4=25.7%, 8=55.5%, 16=7.5%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=93.5%, 8=0.3%, 16=6.2%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=657055,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=42.8MiB/s (44.9MB/s), 42.8MiB/s-42.8MiB/s (44.9MB/s-44.9MB/s), io=2567MiB (2691MB), run=60001-60001msec
# 4KB顺序写
D:\>c:\fio\fio\fio --name=random-write --ioengine=windowsaio --rw=write --bs=4k --size=4g --numjobs=1 --iodepth=16 --runtime=60 --time_based --direct=1 --end_fsync=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
random-write: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=windowsaio, iodepth=16
fio-3.27
Starting 1 thread
random-write: (groupid=0, jobs=1): err= 0: pid=1844: Sat Oct 30 03:17:42 2021
write: IOPS=13.6k, BW=53.0MiB/s (55.6MB/s)(3225MiB/60869msec); 0 zone resets
slat (usec): min=10, max=31877, avg=24.34, stdev=46.77
clat (usec): min=12, max=36621, avg=613.12, stdev=460.77
lat (usec): min=63, max=36729, avg=637.46, stdev=464.13
clat percentiles (usec):
| 1.00th=[ 76], 5.00th=[ 123], 10.00th=[ 182], 20.00th=[ 281],
| 30.00th=[ 383], 40.00th=[ 486], 50.00th=[ 586], 60.00th=[ 685],
| 70.00th=[ 783], 80.00th=[ 898], 90.00th=[ 1057], 95.00th=[ 1188],
| 99.00th=[ 1401], 99.50th=[ 1549], 99.90th=[ 3523], 99.95th=[ 5211],
| 99.99th=[15533]
bw ( KiB/s): min=42653, max=63911, per=100.00%, avg=55109.35, stdev=5005.73, samples=119
iops : min=10663, max=15977, avg=13776.94, stdev=1251.41, samples=119
lat (usec) : 20=0.01%, 50=0.01%, 100=2.67%, 250=14.12%, 500=24.62%
lat (usec) : 750=25.02%, 1000=20.59%
lat (msec) : 2=12.77%, 4=0.13%, 10=0.07%, 20=0.01%, 50=0.01%
cpu : usr=3.29%, sys=90.36%, ctx=0, majf=0, minf=0
IO depths : 1=0.2%, 2=11.5%, 4=26.9%, 8=54.5%, 16=6.9%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=93.6%, 8=0.1%, 16=6.3%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,825695,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=53.0MiB/s (55.6MB/s), 53.0MiB/s-53.0MiB/s (55.6MB/s-55.6MB/s), io=3225MiB (3382MB), run=60869-60869msec
# 1MB顺序读
D:\>c:\fio\fio\fio --name=seq-read --ioengine=windowsaio --rw=read --bs=1m --size=16g --numjobs=1 --iodepth=1 --runtime=60 --time_based --direct=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
seq-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=windowsaio, iodepth=1
fio-3.27
Starting 1 thread
seq-read: (groupid=0, jobs=1): err= 0: pid=888: Sat Oct 30 03:18:42 2021
read: IOPS=1993, BW=1994MiB/s (2091MB/s)(117GiB/60001msec)
slat (usec): min=36, max=6062, avg=138.32, stdev=57.54
clat (usec): min=8, max=58046, avg=340.18, stdev=1029.30
lat (usec): min=234, max=58150, avg=478.50, stdev=1032.07
clat percentiles (usec):
| 1.00th=[ 127], 5.00th=[ 145], 10.00th=[ 153], 20.00th=[ 165],
| 30.00th=[ 176], 40.00th=[ 184], 50.00th=[ 196], 60.00th=[ 210],
| 70.00th=[ 243], 80.00th=[ 322], 90.00th=[ 750], 95.00th=[ 857],
| 99.00th=[ 1106], 99.50th=[ 1631], 99.90th=[20317], 99.95th=[25035],
| 99.99th=[35914]
bw ( MiB/s): min= 354, max= 2937, per=99.77%, avg=1989.21, stdev=953.24, samples=118
iops : min= 354, max= 2937, avg=1988.64, stdev=953.24, samples=118
lat (usec) : 10=0.01%, 20=0.01%, 50=0.01%, 100=0.53%, 250=70.42%
lat (usec) : 500=14.57%, 750=4.65%, 1000=8.18%
lat (msec) : 2=1.19%, 4=0.12%, 10=0.14%, 20=0.08%, 50=0.10%
lat (msec) : 100=0.01%
cpu : usr=1.67%, sys=23.33%, ctx=0, majf=0, minf=0
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=119635,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=1994MiB/s (2091MB/s), 1994MiB/s-1994MiB/s (2091MB/s-2091MB/s), io=117GiB (125GB), run=60001-60001msec
# 1MB顺序写
D:\>c:\fio\fio\fio --name=seq-write --ioengine=windowsaio --rw=write --bs=1m --size=16g --numjobs=1 --iodepth=1 --runtime=60 --time_based --direct=1 --end_fsync=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=windowsaio, iodepth=1
fio-3.27
Starting 1 thread
seq-write: (groupid=0, jobs=1): err= 0: pid=752: Sat Oct 30 03:19:59 2021
write: IOPS=920, BW=920MiB/s (965MB/s)(68.0GiB/75656msec); 0 zone resets
slat (usec): min=47, max=15591, avg=188.06, stdev=148.77
clat (usec): min=16, max=99062, avg=648.07, stdev=746.30
lat (usec): min=520, max=99575, avg=836.12, stdev=754.78
clat percentiles (usec):
| 1.00th=[ 225], 5.00th=[ 420], 10.00th=[ 445], 20.00th=[ 474],
| 30.00th=[ 502], 40.00th=[ 529], 50.00th=[ 570], 60.00th=[ 611],
| 70.00th=[ 660], 80.00th=[ 734], 90.00th=[ 848], 95.00th=[ 996],
| 99.00th=[ 1844], 99.50th=[ 2802], 99.90th=[ 8094], 99.95th=[12780],
| 99.99th=[22414]
bw ( MiB/s): min= 814, max= 1414, per=100.00%, avg=1163.16, stdev=144.28, samples=117
iops : min= 814, max= 1414, avg=1162.67, stdev=144.21, samples=117
lat (usec) : 20=0.01%, 50=0.07%, 100=0.07%, 250=1.13%, 500=28.58%
lat (usec) : 750=52.20%, 1000=13.03%
lat (msec) : 2=4.04%, 4=0.56%, 10=0.24%, 20=0.06%, 50=0.01%
lat (msec) : 100=0.01%
cpu : usr=1.32%, sys=19.83%, ctx=0, majf=0, minf=0
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,69621,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=920MiB/s (965MB/s), 920MiB/s-920MiB/s (965MB/s-965MB/s), io=68.0GiB (73.0GB), run=75656-75656msec
- Windows虚拟机(cache=writeback,queues=4)
# 4KB随机读
D:\>c:\fio\fio\fio --name=random-read --ioengine=windowsaio --rw=randread --bs=4k --size=4g --numjobs=1 --iodepth=16 --runtime=60 --time_based --direct=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
random-read: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=windowsaio, iodepth=16
fio-3.27
Starting 1 thread
random-read: (groupid=0, jobs=1): err= 0: pid=744: Sat Oct 30 03:26:54 2021
read: IOPS=4520, BW=17.7MiB/s (18.5MB/s)(1060MiB/60018msec)
slat (usec): min=10, max=11291, avg=21.51, stdev=28.16
clat (usec): min=7, max=104583, avg=3483.03, stdev=5554.09
lat (usec): min=40, max=104616, avg=3504.54, stdev=5554.55
clat percentiles (usec):
| 1.00th=[ 46], 5.00th=[ 56], 10.00th=[ 65], 20.00th=[ 78],
| 30.00th=[ 91], 40.00th=[ 113], 50.00th=[ 169], 60.00th=[ 1745],
| 70.00th=[ 4621], 80.00th=[ 7177], 90.00th=[ 9896], 95.00th=[14484],
| 99.00th=[24773], 99.50th=[29754], 99.90th=[41681], 99.95th=[46400],
| 99.99th=[58459]
bw ( KiB/s): min= 9074, max=34435, per=99.64%, avg=18017.24, stdev=8984.02, samples=119
iops : min= 2268, max= 8608, avg=4503.92, stdev=2246.05, samples=119
lat (usec) : 10=0.01%, 20=0.01%, 50=3.30%, 100=31.72%, 250=20.52%
lat (usec) : 500=2.30%, 750=0.30%, 1000=0.19%
lat (msec) : 2=2.37%, 4=6.91%, 10=22.64%, 20=7.81%, 50=1.90%
lat (msec) : 100=0.03%, 250=0.01%
cpu : usr=1.67%, sys=23.33%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=17.5%, 16=82.5%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=99.9%, 8=0.1%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=271323,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=17.7MiB/s (18.5MB/s), 17.7MiB/s-17.7MiB/s (18.5MB/s-18.5MB/s), io=1060MiB (1111MB), run=60018-60018msec
# 4KB随机写
D:\>c:\fio\fio\fio --name=random-write --ioengine=windowsaio --rw=randwrite --bs=4k --size=4g --numjobs=1 --iodepth=16 --runtime=60 --time_based --direct=1 --end_fsync=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
random-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=windowsaio, iodepth=16
fio-3.27
Starting 1 thread
random-write: (groupid=0, jobs=1): err= 0: pid=1560: Sat Oct 30 03:28:09 2021
write: IOPS=18.8k, BW=73.6MiB/s (77.1MB/s)(5516MiB/74982msec); 0 zone resets
slat (usec): min=10, max=13548, avg=17.65, stdev=20.57
clat (usec): min=7, max=52330, avg=353.93, stdev=290.90
lat (usec): min=41, max=52372, avg=371.58, stdev=292.80
clat percentiles (usec):
| 1.00th=[ 42], 5.00th=[ 74], 10.00th=[ 108], 20.00th=[ 163],
| 30.00th=[ 219], 40.00th=[ 277], 50.00th=[ 338], 60.00th=[ 400],
| 70.00th=[ 457], 80.00th=[ 519], 90.00th=[ 603], 95.00th=[ 693],
| 99.00th=[ 832], 99.50th=[ 898], 99.90th=[ 1532], 99.95th=[ 3261],
| 99.99th=[ 9765]
bw ( KiB/s): min=69809, max=108038, per=100.00%, avg=94186.97, stdev=5958.06, samples=117
iops : min=17452, max=27009, avg=23546.34, stdev=1489.56, samples=117
lat (usec) : 10=0.01%, 20=0.01%, 50=2.00%, 100=6.50%, 250=26.64%
lat (usec) : 500=42.11%, 750=19.75%, 1000=2.73%
lat (msec) : 2=0.18%, 4=0.04%, 10=0.02%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%
cpu : usr=1.33%, sys=74.69%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=10.7%, 4=27.4%, 8=55.0%, 16=6.9%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=93.6%, 8=0.1%, 16=6.4%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,1411978,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=73.6MiB/s (77.1MB/s), 73.6MiB/s-73.6MiB/s (77.1MB/s-77.1MB/s), io=5516MiB (5783MB), run=74982-74982msec
# 4KB顺序读
D:\>c:\fio\fio\fio --name=random-read --ioengine=windowsaio --rw=read --bs=4k --size=4g --numjobs=1 --iodepth=16 --runtime=60 --time_based --direct=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
random-read: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=windowsaio, iodepth=16
fio-3.27
Starting 1 thread
random-read: (groupid=0, jobs=1): err= 0: pid=1176: Sat Oct 30 03:29:10 2021
read: IOPS=23.3k, BW=90.8MiB/s (95.2MB/s)(5450MiB/60001msec)
slat (usec): min=10, max=7312, avg=17.20, stdev=11.52
clat (usec): min=8, max=130244, avg=373.35, stdev=982.26
lat (usec): min=34, max=130256, avg=390.55, stdev=982.72
clat percentiles (usec):
| 1.00th=[ 40], 5.00th=[ 73], 10.00th=[ 104], 20.00th=[ 155],
| 30.00th=[ 210], 40.00th=[ 269], 50.00th=[ 330], 60.00th=[ 388],
| 70.00th=[ 445], 80.00th=[ 506], 90.00th=[ 594], 95.00th=[ 685],
| 99.00th=[ 848], 99.50th=[ 1004], 99.90th=[ 8848], 99.95th=[17171],
| 99.99th=[45351]
bw ( KiB/s): min=12959, max=109779, per=100.00%, avg=93136.24, stdev=12016.94, samples=119
iops : min= 3239, max=27444, avg=23283.70, stdev=3004.27, samples=119
lat (usec) : 10=0.01%, 20=0.01%, 50=2.48%, 100=6.84%, 250=27.50%
lat (usec) : 500=42.20%, 750=18.18%, 1000=2.30%
lat (msec) : 2=0.19%, 4=0.10%, 10=0.13%, 20=0.04%, 50=0.03%
lat (msec) : 100=0.01%, 250=0.01%
cpu : usr=5.00%, sys=88.33%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=10.7%, 4=26.7%, 8=55.2%, 16=7.3%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=93.6%, 8=0.1%, 16=6.3%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=1395265,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=90.8MiB/s (95.2MB/s), 90.8MiB/s-90.8MiB/s (95.2MB/s-95.2MB/s), io=5450MiB (5715MB), run=60001-60001msec
# 4KB顺序写
D:\>c:\fio\fio\fio --name=random-write --ioengine=windowsaio --rw=write --bs=4k --size=4g --numjobs=1 --iodepth=16 --runtime=60 --time_based --direct=1 --end_fsync=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
random-write: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=windowsaio, iodepth=16
fio-3.27
Starting 1 thread
random-write: (groupid=0, jobs=1): err= 0: pid=1904: Sat Oct 30 03:30:12 2021
write: IOPS=21.8k, BW=85.1MiB/s (89.3MB/s)(5307MiB/62348msec); 0 zone resets
slat (usec): min=10, max=5404, avg=18.20, stdev=14.48
clat (usec): min=8, max=42255, avg=363.23, stdev=253.82
lat (usec): min=30, max=42322, avg=381.42, stdev=255.57
clat percentiles (usec):
| 1.00th=[ 42], 5.00th=[ 73], 10.00th=[ 105], 20.00th=[ 161],
| 30.00th=[ 223], 40.00th=[ 285], 50.00th=[ 347], 60.00th=[ 408],
| 70.00th=[ 469], 80.00th=[ 537], 90.00th=[ 627], 95.00th=[ 725],
| 99.00th=[ 898], 99.50th=[ 938], 99.90th=[ 1450], 99.95th=[ 2507],
| 99.99th=[ 5604]
bw ( KiB/s): min=67968, max=105995, per=100.00%, avg=90679.54, stdev=7415.13, samples=118
iops : min=16992, max=26498, avg=22669.50, stdev=1853.74, samples=118
lat (usec) : 10=0.01%, 20=0.01%, 50=2.97%, 100=6.69%, 250=25.02%
lat (usec) : 500=39.93%, 750=21.18%, 1000=3.88%
lat (msec) : 2=0.26%, 4=0.04%, 10=0.03%, 20=0.01%, 50=0.01%
cpu : usr=1.60%, sys=88.22%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=11.9%, 4=27.0%, 8=54.2%, 16=6.8%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=93.7%, 8=0.1%, 16=6.3%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,1358610,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=85.1MiB/s (89.3MB/s), 85.1MiB/s-85.1MiB/s (89.3MB/s-89.3MB/s), io=5307MiB (5565MB), run=62348-62348msec
# 1MB顺序读
D:\>c:\fio\fio\fio --name=seq-read --ioengine=windowsaio --rw=read --bs=1m --size=16g --numjobs=1 --iodepth=1 --runtime=60 --time_based --direct=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
seq-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=windowsaio, iodepth=1
fio-3.27
Starting 1 thread
seq-read: (groupid=0, jobs=1): err= 0: pid=1112: Sat Oct 30 03:31:13 2021
read: IOPS=2628, BW=2628MiB/s (2756MB/s)(154GiB/60001msec)
slat (usec): min=41, max=5273, avg=62.79, stdev=26.58
clat (usec): min=9, max=35828, avg=292.48, stdev=202.04
lat (usec): min=221, max=35886, avg=355.27, stdev=205.63
clat percentiles (usec):
| 1.00th=[ 208], 5.00th=[ 225], 10.00th=[ 235], 20.00th=[ 249],
| 30.00th=[ 260], 40.00th=[ 269], 50.00th=[ 277], 60.00th=[ 289],
| 70.00th=[ 302], 80.00th=[ 318], 90.00th=[ 343], 95.00th=[ 371],
| 99.00th=[ 529], 99.50th=[ 693], 99.90th=[ 1516], 99.95th=[ 2638],
| 99.99th=[ 8029]
bw ( MiB/s): min= 1827, max= 2839, per=100.00%, avg=2629.96, stdev=129.75, samples=119
iops : min= 1827, max= 2839, avg=2629.53, stdev=129.76, samples=119
lat (usec) : 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01%, 250=21.27%
lat (usec) : 500=77.55%, 750=0.76%, 1000=0.20%
lat (msec) : 2=0.16%, 4=0.02%, 10=0.04%, 20=0.01%, 50=0.01%
cpu : usr=0.00%, sys=25.00%, ctx=0, majf=0, minf=0
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=157697,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=2628MiB/s (2756MB/s), 2628MiB/s-2628MiB/s (2756MB/s-2756MB/s), io=154GiB (165GB), run=60001-60001msec
# 1MB顺序写
D:\>c:\fio\fio\fio --name=seq-write --ioengine=windowsaio --rw=write --bs=1m --size=16g --numjobs=1 --iodepth=1 --runtime=60 --time_based --direct=1 --end_fsync=1
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=windowsaio, iodepth=1
fio-3.27
Starting 1 thread
seq-write: (groupid=0, jobs=1): err= 0: pid=1768: Sat Oct 30 03:32:25 2021
write: IOPS=1191, BW=1192MiB/s (1250MB/s)(83.1GiB/71391msec); 0 zone resets
slat (usec): min=47, max=15215, avg=79.43, stdev=97.94
clat (usec): min=18, max=279462, avg=600.78, stdev=1584.12
lat (usec): min=434, max=279525, avg=680.21, stdev=1587.71
clat percentiles (usec):
| 1.00th=[ 424], 5.00th=[ 449], 10.00th=[ 461], 20.00th=[ 482],
| 30.00th=[ 494], 40.00th=[ 510], 50.00th=[ 523], 60.00th=[ 537],
| 70.00th=[ 562], 80.00th=[ 586], 90.00th=[ 627], 95.00th=[ 701],
| 99.00th=[ 2180], 99.50th=[ 3228], 99.90th=[12387], 99.95th=[16450],
| 99.99th=[46400]
bw ( MiB/s): min= 384, max= 1747, per=100.00%, avg=1418.29, stdev=247.74, samples=118
iops : min= 384, max= 1747, avg=1417.84, stdev=247.79, samples=118
lat (usec) : 20=0.01%, 50=0.01%, 100=0.01%, 250=0.01%, 500=33.56%
lat (usec) : 750=62.63%, 1000=1.74%
lat (msec) : 2=0.96%, 4=0.73%, 10=0.22%, 20=0.11%, 50=0.02%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%
cpu : usr=1.40%, sys=11.21%, ctx=0, majf=0, minf=0
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,85092,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=1192MiB/s (1250MB/s), 1192MiB/s-1192MiB/s (1250MB/s-1250MB/s), io=83.1GiB (89.2GB), run=71391-71391msec
共享存储（ceph）IO性能测试
测试数据
- Linux虚拟机 (cache=none)
# 4KB随机读
random-read: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
random-read: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [r(1)][100.0%][r=4436KiB/s,w=0KiB/s][r=1109,w=0 IOPS][eta 00m:00s]
random-read: (groupid=0, jobs=1): err= 0: pid=1531: Mon Nov 1 17:01:59 2021
read: IOPS=1054, BW=4217KiB/s (4318kB/s)(247MiB/60014msec)
slat (nsec): min=776, max=1030.1k, avg=4559.89, stdev=5863.28
clat (usec): min=9767, max=50045, avg=15115.37, stdev=1982.74
lat (usec): min=9772, max=50050, avg=15119.93, stdev=1982.80
clat percentiles (usec):
| 1.00th=[12387], 5.00th=[13042], 10.00th=[13435], 20.00th=[13829],
| 30.00th=[14222], 40.00th=[14484], 50.00th=[14877], 60.00th=[15270],
| 70.00th=[15533], 80.00th=[16057], 90.00th=[16909], 95.00th=[17695],
| 99.00th=[22676], 99.50th=[26608], 99.90th=[35914], 99.95th=[45876],
| 99.99th=[49546]
bw ( KiB/s): min= 3704, max= 4992, per=99.99%, avg=4215.57, stdev=267.66, samples=120
iops : min= 926, max= 1248, avg=1053.83, stdev=66.90, samples=120
lat (msec) : 10=0.01%, 20=98.72%, 50=1.27%, 100=0.01%
cpu : usr=3.51%, sys=2.62%, ctx=31649, majf=0, minf=51
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.0%, 16=50.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.8%, 8=0.1%, 16=4.2%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=63264,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=4217KiB/s (4318kB/s), 4217KiB/s-4217KiB/s (4318kB/s-4318kB/s), io=247MiB (259MB), run=60014-60014msec
# 4KB随机写
Disk stats (read/write):
sdc: ios=63169/7, merge=0/4, ticks=56012/16, in_queue=55977, util=93.37%
random-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
random-write: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=1600KiB/s][r=0,w=400 IOPS][eta 00m:00s]
random-write: (groupid=0, jobs=1): err= 0: pid=1549: Mon Nov 1 17:03:00 2021
write: IOPS=405, BW=1622KiB/s (1661kB/s)(95.2MiB/60080msec)
slat (nsec): min=911, max=792365, avg=5502.29, stdev=6526.82
clat (usec): min=27204, max=70981, avg=39357.70, stdev=2721.36
lat (usec): min=27213, max=70986, avg=39363.20, stdev=2721.26
clat percentiles (usec):
| 1.00th=[33817], 5.00th=[35390], 10.00th=[36439], 20.00th=[37487],
| 30.00th=[38011], 40.00th=[38536], 50.00th=[39584], 60.00th=[40109],
| 70.00th=[40633], 80.00th=[41157], 90.00th=[42206], 95.00th=[42730],
| 99.00th=[48497], 99.50th=[51643], 99.90th=[61080], 99.95th=[67634],
| 99.99th=[70779]
bw ( KiB/s): min= 1464, max= 1792, per=100.00%, avg=1623.00, stdev=65.87, samples=120
iops : min= 366, max= 448, avg=405.75, stdev=16.47, samples=120
lat (msec) : 50=99.13%, 100=0.87%
cpu : usr=2.46%, sys=1.58%, ctx=12188, majf=0, minf=48
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.0%, 16=50.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.8%, 8=0.0%, 16=4.2%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,24364,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=1622KiB/s (1661kB/s), 1622KiB/s-1622KiB/s (1661kB/s-1661kB/s), io=95.2MiB (99.8MB), run=60080-60080msec
# 4KB顺序读
Disk stats (read/write):
sdc: ios=0/24416, merge=0/0, ticks=0/56250, in_queue=56214, util=92.79%
random-read: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [R(1)][100.0%][r=7668KiB/s,w=0KiB/s][r=1917,w=0 IOPS][eta 00m:00s]
random-read: (groupid=0, jobs=1): err= 0: pid=1554: Mon Nov 1 17:04:00 2021
read: IOPS=1864, BW=7456KiB/s (7635kB/s)(437MiB/60005msec)
slat (nsec): min=726, max=363502, avg=4053.32, stdev=2934.08
clat (usec): min=4751, max=44503, avg=8530.20, stdev=1396.47
lat (usec): min=4755, max=44507, avg=8534.26, stdev=1396.48
clat percentiles (usec):
| 1.00th=[ 5997], 5.00th=[ 6783], 10.00th=[ 7111], 20.00th=[ 7570],
| 30.00th=[ 7898], 40.00th=[ 8160], 50.00th=[ 8455], 60.00th=[ 8717],
| 70.00th=[ 8979], 80.00th=[ 9372], 90.00th=[10028], 95.00th=[10421],
| 99.00th=[11469], 99.50th=[11994], 99.90th=[22938], 99.95th=[25822],
| 99.99th=[43779]
bw ( KiB/s): min= 6312, max= 9232, per=99.99%, avg=7455.02, stdev=636.53, samples=120
iops : min= 1578, max= 2308, avg=1863.73, stdev=159.15, samples=120
lat (msec) : 10=90.16%, 20=9.68%, 50=0.16%
cpu : usr=5.20%, sys=4.17%, ctx=55930, majf=0, minf=53
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.0%, 16=50.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.8%, 8=0.1%, 16=4.2%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=111856,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=7456KiB/s (7635kB/s), 7456KiB/s-7456KiB/s (7635kB/s-7635kB/s), io=437MiB (458MB), run=60005-60005msec
# 4KB顺序写
Disk stats (read/write):
sdc: ios=111661/9, merge=0/236, ticks=54941/76, in_queue=54958, util=91.58%
random-write: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [W(1)][100.0%][r=0KiB/s,w=2128KiB/s][r=0,w=532 IOPS][eta 00m:00s]
random-write: (groupid=0, jobs=1): err= 0: pid=1558: Mon Nov 1 17:05:01 2021
write: IOPS=552, BW=2209KiB/s (2262kB/s)(130MiB/60023msec)
slat (nsec): min=929, max=151745, avg=5243.52, stdev=3891.39
clat (usec): min=13208, max=55880, avg=28901.03, stdev=4556.88
lat (usec): min=13209, max=55887, avg=28906.27, stdev=4556.96
clat percentiles (usec):
| 1.00th=[17695], 5.00th=[20841], 10.00th=[22938], 20.00th=[25560],
| 30.00th=[27132], 40.00th=[28181], 50.00th=[29230], 60.00th=[30016],
| 70.00th=[31065], 80.00th=[32375], 90.00th=[34341], 95.00th=[35914],
| 99.00th=[40109], 99.50th=[41681], 99.90th=[52691], 99.95th=[53740],
| 99.99th=[55837]
bw ( KiB/s): min= 1720, max= 2944, per=99.98%, avg=2208.57, stdev=224.65, samples=120
iops : min= 430, max= 736, avg=552.08, stdev=56.16, samples=120
lat (msec) : 20=3.88%, 50=95.93%, 100=0.19%
cpu : usr=3.11%, sys=2.10%, ctx=16583, majf=0, minf=50
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.0%, 16=50.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.8%, 8=0.1%, 16=4.2%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,33152,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=2209KiB/s (2262kB/s), 2209KiB/s-2209KiB/s (2262kB/s-2262kB/s), io=130MiB (136MB), run=60023-60023msec
# 1MB顺序读
Disk stats (read/write):
sdc: ios=0/33156, merge=0/0, ticks=0/55067, in_queue=55003, util=91.55%
seq-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=posixaio, iodepth=1
fio-3.7
Starting 1 process
seq-read: Laying out IO file (1 file / 16384MiB)
Jobs: 1 (f=1): [R(1)][100.0%][r=160MiB/s,w=0KiB/s][r=160,w=0 IOPS][eta 00m:00s]
seq-read: (groupid=0, jobs=1): err= 0: pid=1693: Mon Nov 1 17:06:18 2021
read: IOPS=159, BW=159MiB/s (167MB/s)(9570MiB/60001msec)
slat (usec): min=5, max=397, avg=40.43, stdev=11.29
clat (usec): min=4992, max=20537, avg=6209.77, stdev=624.64
lat (usec): min=5028, max=20575, avg=6250.19, stdev=625.42
clat percentiles (usec):
| 1.00th=[ 5473], 5.00th=[ 5669], 10.00th=[ 5735], 20.00th=[ 5866],
| 30.00th=[ 5997], 40.00th=[ 6063], 50.00th=[ 6128], 60.00th=[ 6259],
| 70.00th=[ 6325], 80.00th=[ 6456], 90.00th=[ 6652], 95.00th=[ 6783],
| 99.00th=[ 7439], 99.50th=[ 8586], 99.90th=[15533], 99.95th=[17171],
| 99.99th=[20579]
bw ( KiB/s): min=151249, max=169984, per=99.97%, avg=163276.76, stdev=2878.69, samples=119
iops : min= 147, max= 166, avg=159.39, stdev= 2.88, samples=119
lat (msec) : 10=99.66%, 20=0.33%, 50=0.01%
cpu : usr=1.39%, sys=1.97%, ctx=9572, majf=0, minf=50
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=9570,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=159MiB/s (167MB/s), 159MiB/s-159MiB/s (167MB/s-167MB/s), io=9570MiB (10.0GB), run=60001-60001msec
# 1MB顺序写
Disk stats (read/write):
sdc: ios=28650/4, merge=0/1, ticks=150516/10, in_queue=150027, util=93.26%
seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=posixaio, iodepth=1
fio-3.7
Starting 1 process
seq-write: Laying out IO file (1 file / 16384MiB)
Jobs: 1 (f=1): [W(1)][100.0%][r=0KiB/s,w=162MiB/s][r=0,w=162 IOPS][eta 00m:00s]
seq-write: (groupid=0, jobs=1): err= 0: pid=1698: Mon Nov 1 17:07:18 2021
write: IOPS=163, BW=164MiB/s (172MB/s)(9831MiB/60006msec)
slat (usec): min=25, max=357, avg=105.09, stdev=20.95
clat (usec): min=4693, max=19505, avg=5978.97, stdev=532.88
lat (usec): min=4764, max=19612, avg=6084.07, stdev=534.17
clat percentiles (usec):
| 1.00th=[ 5276], 5.00th=[ 5473], 10.00th=[ 5604], 20.00th=[ 5735],
| 30.00th=[ 5800], 40.00th=[ 5866], 50.00th=[ 5932], 60.00th=[ 5997],
| 70.00th=[ 6063], 80.00th=[ 6194], 90.00th=[ 6325], 95.00th=[ 6456],
| 99.00th=[ 6718], 99.50th=[ 6849], 99.90th=[14484], 99.95th=[17171],
| 99.99th=[19530]
bw ( KiB/s): min=159744, max=176128, per=99.99%, avg=167751.33, stdev=2470.41, samples=120
iops : min= 156, max= 172, avg=163.79, stdev= 2.42, samples=120
lat (msec) : 10=99.75%, 20=0.25%
cpu : usr=2.90%, sys=1.67%, ctx=9836, majf=0, minf=47
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,9831,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=164MiB/s (172MB/s), 164MiB/s-164MiB/s (172MB/s-172MB/s), io=9831MiB (10.3GB), run=60006-60006msec
Disk stats (read/write):
sdc: ios=0/29502, merge=0/1, ticks=0/162775, in_queue=162226, util=91.91%
- Linux虚拟机(cache=none,queues=4)
# 4KB随机读
random-read: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
random-read: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [r(1)][100.0%][r=4715KiB/s,w=0KiB/s][r=1178,w=0 IOPS][eta 00m:00s]
random-read: (groupid=0, jobs=1): err= 0: pid=1407: Mon Nov 1 15:58:54 2021
read: IOPS=1152, BW=4608KiB/s (4719kB/s)(270MiB/60010msec)
slat (nsec): min=794, max=439011, avg=4344.27, stdev=3510.29
clat (usec): min=10063, max=27656, avg=13827.04, stdev=1322.77
lat (usec): min=10067, max=27660, avg=13831.39, stdev=1322.78
clat percentiles (usec):
| 1.00th=[11731], 5.00th=[12256], 10.00th=[12518], 20.00th=[12911],
| 30.00th=[13173], 40.00th=[13435], 50.00th=[13698], 60.00th=[13960],
| 70.00th=[14222], 80.00th=[14484], 90.00th=[15139], 95.00th=[15795],
| 99.00th=[17695], 99.50th=[21103], 99.90th=[26870], 99.95th=[27132],
| 99.99th=[27395]
bw ( KiB/s): min= 4224, max= 5248, per=99.96%, avg=4606.38, stdev=190.03, samples=120
iops : min= 1056, max= 1312, avg=1151.53, stdev=47.56, samples=120
lat (msec) : 20=99.47%, 50=0.53%
cpu : usr=3.70%, sys=2.58%, ctx=34579, majf=0, minf=51
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.0%, 16=50.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.8%, 8=0.1%, 16=4.2%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=69136,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=4608KiB/s (4719kB/s), 4608KiB/s-4608KiB/s (4719kB/s-4719kB/s), io=270MiB (283MB), run=60010-60010msec
# 4KB随机写
Disk stats (read/write):
sdc: ios=69004/7, merge=0/4, ticks=56048/16, in_queue=56030, util=93.46%
random-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
random-write: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=1856KiB/s][r=0,w=464 IOPS][eta 00m:00s]
random-write: (groupid=0, jobs=1): err= 0: pid=1413: Mon Nov 1 15:59:54 2021
write: IOPS=441, BW=1767KiB/s (1810kB/s)(104MiB/60076msec)
slat (nsec): min=835, max=304849, avg=5458.22, stdev=4263.50
clat (usec): min=24800, max=73340, avg=36111.87, stdev=2760.31
lat (usec): min=24804, max=73346, avg=36117.33, stdev=2760.32
clat percentiles (usec):
| 1.00th=[30540], 5.00th=[32375], 10.00th=[33162], 20.00th=[33817],
| 30.00th=[34866], 40.00th=[35390], 50.00th=[35914], 60.00th=[36439],
| 70.00th=[36963], 80.00th=[38011], 90.00th=[39060], 95.00th=[40109],
| 99.00th=[43254], 99.50th=[47973], 99.90th=[55837], 99.95th=[72877],
| 99.99th=[72877]
bw ( KiB/s): min= 1536, max= 1920, per=100.00%, avg=1768.10, stdev=73.45, samples=120
iops : min= 384, max= 480, avg=442.03, stdev=18.36, samples=120
lat (msec) : 50=99.59%, 100=0.41%
cpu : usr=2.52%, sys=1.59%, ctx=13273, majf=0, minf=48
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.0%, 16=50.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.8%, 8=0.1%, 16=4.2%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,26544,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=1767KiB/s (1810kB/s), 1767KiB/s-1767KiB/s (1810kB/s-1810kB/s), io=104MiB (109MB), run=60076-60076msec
# 4KB顺序读
Disk stats (read/write):
sdc: ios=0/26609, merge=0/0, ticks=0/56576, in_queue=56525, util=92.75%
random-read: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [R(1)][100.0%][r=8392KiB/s,w=0KiB/s][r=2098,w=0 IOPS][eta 00m:00s]
random-read: (groupid=0, jobs=1): err= 0: pid=1418: Mon Nov 1 16:00:55 2021
read: IOPS=2066, BW=8265KiB/s (8463kB/s)(484MiB/60006msec)
slat (nsec): min=744, max=457916, avg=4120.39, stdev=3331.87
clat (usec): min=3786, max=44373, avg=7688.63, stdev=1249.73
lat (usec): min=3787, max=44377, avg=7692.75, stdev=1249.79
clat percentiles (usec):
| 1.00th=[ 5276], 5.00th=[ 5997], 10.00th=[ 6390], 20.00th=[ 6783],
| 30.00th=[ 7111], 40.00th=[ 7439], 50.00th=[ 7635], 60.00th=[ 7898],
| 70.00th=[ 8160], 80.00th=[ 8455], 90.00th=[ 8979], 95.00th=[ 9372],
| 99.00th=[10552], 99.50th=[11731], 99.90th=[19268], 99.95th=[21103],
| 99.99th=[43779]
bw ( KiB/s): min= 7040, max=10728, per=99.99%, avg=8263.15, stdev=690.54, samples=120
iops : min= 1760, max= 2682, avg=2065.74, stdev=172.65, samples=120
lat (msec) : 4=0.03%, 10=98.19%, 20=1.71%, 50=0.08%
cpu : usr=5.94%, sys=4.37%, ctx=61939, majf=0, minf=53
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.0%, 16=50.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.8%, 8=0.1%, 16=4.2%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=123984,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=8265KiB/s (8463kB/s), 8265KiB/s-8265KiB/s (8463kB/s-8463kB/s), io=484MiB (508MB), run=60006-60006msec
# 4KB顺序写
Disk stats (read/write):
sdc: ios=123792/9, merge=0/253, ticks=54685/32, in_queue=54656, util=91.15%
random-write: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [W(1)][2.9%][r=0KiB/s,w=0KiB/s][r=0,w=0 IOPS][eta 45m:41s]
random-write: (groupid=0, jobs=1): err= 0: pid=1423: Mon Nov 1 16:02:18 2021
write: IOPS=368, BW=1472KiB/s (1508kB/s)(119MiB/82766msec)
slat (nsec): min=1021, max=330640, avg=5245.94, stdev=4051.13
clat (msec): min=14, max=26580, avg=43.40, stdev=613.63
lat (msec): min=14, max=26580, avg=43.41, stdev=613.63
clat percentiles (msec):
| 1.00th=[ 16], 5.00th=[ 18], 10.00th=[ 19], 20.00th=[ 22],
| 30.00th=[ 24], 40.00th=[ 26], 50.00th=[ 27], 60.00th=[ 28],
| 70.00th=[ 30], 80.00th=[ 31], 90.00th=[ 33], 95.00th=[ 35],
| 99.00th=[ 40], 99.50th=[ 49], 99.90th=[ 3373], 99.95th=[17113],
| 99.99th=[17113]
bw ( KiB/s): min= 128, max= 3320, per=100.00%, avg=2276.17, stdev=591.39, samples=107
iops : min= 32, max= 830, avg=569.01, stdev=147.84, samples=107
lat (msec) : 20=14.82%, 50=84.71%, 100=0.05%, 500=0.21%, 750=0.05%
lat (msec) : 1000=0.05%
cpu : usr=1.76%, sys=1.30%, ctx=15236, majf=0, minf=51
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.0%, 16=50.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.8%, 8=0.1%, 16=4.2%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,30464,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=1472KiB/s (1508kB/s), 1472KiB/s-1472KiB/s (1508kB/s-1508kB/s), io=119MiB (125MB), run=82766-82766msec
# 1MB顺序读
Disk stats (read/write):
sdc: ios=0/30469, merge=0/0, ticks=0/93251, in_queue=93214, util=95.22%
seq-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=posixaio, iodepth=1
fio-3.7
Starting 1 process
seq-read: Laying out IO file (1 file / 16384MiB)
Jobs: 1 (f=1): [R(1)][100.0%][r=171MiB/s,w=0KiB/s][r=171,w=0 IOPS][eta 00m:00s]
seq-read: (groupid=0, jobs=1): err= 0: pid=1698: Mon Nov 1 16:03:41 2021
read: IOPS=168, BW=169MiB/s (177MB/s)(9.90GiB/60005msec)
slat (usec): min=4, max=248, avg=37.64, stdev=11.31
clat (usec): min=2070, max=24238, avg=5864.65, stdev=750.98
lat (usec): min=2151, max=24274, avg=5902.29, stdev=751.62
clat percentiles (usec):
| 1.00th=[ 3064], 5.00th=[ 5145], 10.00th=[ 5342], 20.00th=[ 5538],
| 30.00th=[ 5669], 40.00th=[ 5800], 50.00th=[ 5866], 60.00th=[ 5997],
| 70.00th=[ 6063], 80.00th=[ 6194], 90.00th=[ 6390], 95.00th=[ 6587],
| 99.00th=[ 7308], 99.50th=[ 8029], 99.90th=[14353], 99.95th=[16581],
| 99.99th=[18744]
bw ( KiB/s): min=161792, max=198656, per=99.98%, avg=172916.57, stdev=5682.61, samples=120
iops : min= 158, max= 194, avg=168.79, stdev= 5.58, samples=120
lat (msec) : 4=1.91%, 10=97.82%, 20=0.26%, 50=0.01%
cpu : usr=1.28%, sys=1.96%, ctx=10138, majf=0, minf=51
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=10135,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=169MiB/s (177MB/s), 169MiB/s-169MiB/s (177MB/s-177MB/s), io=9.90GiB (10.6GB), run=60005-60005msec
# 1MB顺序写
Disk stats (read/write):
sdc: ios=30312/4, merge=0/1, ticks=140893/12, in_queue=140411, util=93.45%
seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=posixaio, iodepth=1
fio-3.7
Starting 1 process
seq-write: Laying out IO file (1 file / 16384MiB)
Jobs: 1 (f=1): [W(1)][100.0%][r=0KiB/s,w=166MiB/s][r=0,w=166 IOPS][eta 00m:00s]
seq-write: (groupid=0, jobs=1): err= 0: pid=1702: Mon Nov 1 16:04:41 2021
write: IOPS=168, BW=168MiB/s (176MB/s)(9.85GiB/60004msec)
slat (usec): min=30, max=433, avg=101.35, stdev=22.23
clat (usec): min=4575, max=18939, avg=5828.79, stdev=571.24
lat (usec): min=4661, max=19022, avg=5930.14, stdev=572.50
clat percentiles (usec):
| 1.00th=[ 5014], 5.00th=[ 5276], 10.00th=[ 5407], 20.00th=[ 5538],
| 30.00th=[ 5669], 40.00th=[ 5735], 50.00th=[ 5800], 60.00th=[ 5866],
| 70.00th=[ 5932], 80.00th=[ 6063], 90.00th=[ 6194], 95.00th=[ 6325],
| 99.00th=[ 6980], 99.50th=[ 8029], 99.90th=[14222], 99.95th=[16909],
| 99.99th=[18744]
bw ( KiB/s): min=165556, max=178176, per=100.00%, avg=172120.71, stdev=2727.46, samples=119
iops : min= 161, max= 174, avg=168.03, stdev= 2.69, samples=119
lat (msec) : 10=99.73%, 20=0.27%
cpu : usr=2.78%, sys=1.62%, ctx=10088, majf=0, minf=47
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,10085,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=168MiB/s (176MB/s), 168MiB/s-168MiB/s (176MB/s-176MB/s), io=9.85GiB (10.6GB), run=60004-60004msec
Disk stats (read/write):
sdc: ios=0/30264, merge=0/1, ticks=0/162607, in_queue=162158, util=91.96%
- Linux虚拟机(cache=writeback)
# 4KB随机读
random-read: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
random-read: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [r(1)][100.0%][r=4044KiB/s,w=0KiB/s][r=1011,w=0 IOPS][eta 00m:00s]
random-read: (groupid=0, jobs=1): err= 0: pid=1412: Mon Nov 1 12:57:14 2021
read: IOPS=978, BW=3913KiB/s (4006kB/s)(229MiB/60016msec)
slat (nsec): min=752, max=581841, avg=4320.80, stdev=3787.00
clat (usec): min=10929, max=53119, avg=16296.60, stdev=1831.63
lat (usec): min=10933, max=53121, avg=16300.92, stdev=1831.63
clat percentiles (usec):
| 1.00th=[13435], 5.00th=[14353], 10.00th=[14746], 20.00th=[15139],
| 30.00th=[15533], 40.00th=[15795], 50.00th=[16057], 60.00th=[16319],
| 70.00th=[16712], 80.00th=[17171], 90.00th=[17957], 95.00th=[18744],
| 99.00th=[22938], 99.50th=[27132], 99.90th=[33424], 99.95th=[33817],
| 99.99th=[53216]
bw ( KiB/s): min= 3432, max= 4423, per=99.97%, avg=3910.99, stdev=146.62, samples=120
iops : min= 858, max= 1105, avg=977.69, stdev=36.58, samples=120
lat (msec) : 20=97.87%, 50=2.11%, 100=0.03%
cpu : usr=3.29%, sys=2.44%, ctx=29349, majf=0, minf=51
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.0%, 16=50.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.8%, 8=0.1%, 16=4.2%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=58704,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=3913KiB/s (4006kB/s), 3913KiB/s-3913KiB/s (4006kB/s-4006kB/s), io=229MiB (240MB), run=60016-60016msec
# 4KB随机写
Disk stats (read/write):
sdc: ios=58691/7, merge=0/4, ticks=56420/3, in_queue=56400, util=93.92%
random-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
random-write: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=39.3MiB/s][r=0,w=10.1k IOPS][eta 00m:00s]
random-write: (groupid=0, jobs=1): err= 0: pid=1416: Mon Nov 1 12:58:15 2021
write: IOPS=9425, BW=36.8MiB/s (38.6MB/s)(2219MiB/60257msec)
slat (nsec): min=747, max=613065, avg=1764.96, stdev=1296.73
clat (usec): min=115, max=60312, avg=1671.98, stdev=1573.22
lat (usec): min=133, max=60313, avg=1673.74, stdev=1573.24
clat percentiles (usec):
| 1.00th=[ 1123], 5.00th=[ 1172], 10.00th=[ 1205], 20.00th=[ 1254],
| 30.00th=[ 1303], 40.00th=[ 1450], 50.00th=[ 1549], 60.00th=[ 1614],
| 70.00th=[ 1696], 80.00th=[ 1811], 90.00th=[ 2040], 95.00th=[ 2343],
| 99.00th=[ 3392], 99.50th=[ 5014], 99.90th=[30802], 99.95th=[38011],
| 99.99th=[56361]
bw ( KiB/s): min=30032, max=44848, per=100.00%, avg=37854.69, stdev=2638.83, samples=120
iops : min= 7508, max=11212, avg=9463.64, stdev=659.71, samples=120
lat (usec) : 250=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=88.69%, 4=10.60%, 10=0.40%, 20=0.12%, 50=0.17%
lat (msec) : 100=0.01%
cpu : usr=9.61%, sys=6.77%, ctx=279981, majf=0, minf=49
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.0%, 16=50.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=96.2%, 8=1.7%, 16=2.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,567952,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=36.8MiB/s (38.6MB/s), 36.8MiB/s-36.8MiB/s (38.6MB/s-38.6MB/s), io=2219MiB (2326MB), run=60257-60257msec
# 4KB顺序读
Disk stats (read/write):
sdc: ios=0/586493, merge=0/21, ticks=0/219865, in_queue=219751, util=85.11%
random-read: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [R(1)][100.0%][r=6410KiB/s,w=0KiB/s][r=1602,w=0 IOPS][eta 00m:00s]
random-read: (groupid=0, jobs=1): err= 0: pid=1422: Mon Nov 1 12:59:16 2021
read: IOPS=1623, BW=6494KiB/s (6649kB/s)(381MiB/60009msec)
slat (nsec): min=772, max=424289, avg=4095.25, stdev=3245.01
clat (usec): min=5375, max=43394, avg=9801.13, stdev=1741.75
lat (usec): min=5377, max=43399, avg=9805.23, stdev=1741.80
clat percentiles (usec):
| 1.00th=[ 6915], 5.00th=[ 7635], 10.00th=[ 8029], 20.00th=[ 8455],
| 30.00th=[ 8979], 40.00th=[ 9241], 50.00th=[ 9634], 60.00th=[10028],
| 70.00th=[10421], 80.00th=[10945], 90.00th=[11600], 95.00th=[12256],
| 99.00th=[14746], 99.50th=[16909], 99.90th=[23200], 99.95th=[28705],
| 99.99th=[42730]
bw ( KiB/s): min= 5160, max= 7736, per=99.99%, avg=6492.65, stdev=507.78, samples=120
iops : min= 1290, max= 1934, avg=1623.12, stdev=126.96, samples=120
lat (msec) : 10=59.76%, 20=39.96%, 50=0.28%
cpu : usr=4.71%, sys=3.69%, ctx=48720, majf=0, minf=53
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.0%, 16=50.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.8%, 8=0.0%, 16=4.2%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=97418,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=6494KiB/s (6649kB/s), 6494KiB/s-6494KiB/s (6649kB/s-6649kB/s), io=381MiB (399MB), run=60009-60009msec
# 4KB顺序写
Disk stats (read/write):
sdc: ios=97204/21, merge=0/2090, ticks=55284/88, in_queue=55314, util=92.16%
random-write: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [W(1)][100.0%][r=0KiB/s,w=39.2MiB/s][r=0,w=10.0k IOPS][eta 00m:00s]
random-write: (groupid=0, jobs=1): err= 0: pid=1426: Mon Nov 1 13:00:16 2021
write: IOPS=10.7k, BW=41.7MiB/s (43.8MB/s)(2507MiB/60060msec)
slat (nsec): min=744, max=1235.1k, avg=1527.84, stdev=1680.34
clat (usec): min=674, max=43657, avg=1478.62, stdev=442.29
lat (usec): min=676, max=43659, avg=1480.14, stdev=442.32
clat percentiles (usec):
| 1.00th=[ 1074], 5.00th=[ 1139], 10.00th=[ 1172], 20.00th=[ 1221],
| 30.00th=[ 1303], 40.00th=[ 1385], 50.00th=[ 1450], 60.00th=[ 1500],
| 70.00th=[ 1565], 80.00th=[ 1647], 90.00th=[ 1762], 95.00th=[ 1909],
| 99.00th=[ 2606], 99.50th=[ 2933], 99.90th=[ 4113], 99.95th=[ 5211],
| 99.99th=[11469]
bw ( KiB/s): min=38016, max=47032, per=100.00%, avg=42802.82, stdev=1577.02, samples=119
iops : min= 9504, max=11758, avg=10700.68, stdev=394.24, samples=119
lat (usec) : 750=0.01%, 1000=0.09%
lat (msec) : 2=96.06%, 4=3.75%, 10=0.09%, 20=0.01%, 50=0.01%
cpu : usr=8.99%, sys=6.64%, ctx=319186, majf=0, minf=52
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.0%, 16=50.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.7%, 8=2.1%, 16=2.2%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,641777,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=41.7MiB/s (43.8MB/s), 41.7MiB/s-41.7MiB/s (43.8MB/s-43.8MB/s), io=2507MiB (2629MB), run=60060-60060msec
# 1MB顺序读
Disk stats (read/write):
sdc: ios=0/641789, merge=0/0, ticks=0/50335, in_queue=50248, util=83.10%
seq-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=posixaio, iodepth=1
fio-3.7
Starting 1 process
seq-read: Laying out IO file (1 file / 16384MiB)
Jobs: 1 (f=1): [R(1)][100.0%][r=172MiB/s,w=0KiB/s][r=172,w=0 IOPS][eta 00m:00s]
seq-read: (groupid=0, jobs=1): err= 0: pid=1431: Mon Nov 1 13:01:34 2021
read: IOPS=149, BW=149MiB/s (156MB/s)(8952MiB/60003msec)
slat (usec): min=4, max=480, avg=36.74, stdev=12.35
clat (msec): min=3, max=2006, avg= 6.65, stdev=42.25
lat (msec): min=3, max=2006, avg= 6.68, stdev=42.25
clat percentiles (msec):
| 1.00th=[ 5], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 5],
| 30.00th=[ 6], 40.00th=[ 6], 50.00th=[ 6], 60.00th=[ 6],
| 70.00th=[ 7], 80.00th=[ 7], 90.00th=[ 7], 95.00th=[ 7],
| 99.00th=[ 9], 99.50th=[ 12], 99.90th=[ 16], 99.95th=[ 19],
| 99.99th=[ 2005]
bw ( KiB/s): min= 2048, max=186368, per=100.00%, avg=169651.27, stdev=29789.32, samples=108
iops : min= 2, max= 182, avg=165.51, stdev=29.08, samples=108
lat (msec) : 4=0.80%, 10=98.57%, 20=0.58%
cpu : usr=1.07%, sys=1.72%, ctx=8956, majf=0, minf=51
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=8952,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=149MiB/s (156MB/s), 149MiB/s-149MiB/s (156MB/s-156MB/s), io=8952MiB (9387MB), run=60003-60003msec
# 1MB顺序写
Disk stats (read/write):
sdc: ios=26854/6, merge=0/2, ticks=138547/6, in_queue=138084, util=94.31%
seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=posixaio, iodepth=1
fio-3.7
Starting 1 process
seq-write: Laying out IO file (1 file / 16384MiB)
Jobs: 1 (f=1): [W(1)][100.0%][r=0KiB/s,w=478MiB/s][r=0,w=478 IOPS][eta 00m:00s]
seq-write: (groupid=0, jobs=1): err= 0: pid=1449: Mon Nov 1 13:02:35 2021
write: IOPS=474, BW=474MiB/s (497MB/s)(27.8GiB/60038msec)
slat (usec): min=18, max=472, avg=77.45, stdev=20.60
clat (usec): min=482, max=25324, avg=2016.70, stdev=3079.65
lat (usec): min=517, max=25398, avg=2094.15, stdev=3077.88
clat percentiles (usec):
| 1.00th=[ 635], 5.00th=[ 693], 10.00th=[ 725], 20.00th=[ 766],
| 30.00th=[ 791], 40.00th=[ 816], 50.00th=[ 840], 60.00th=[ 873],
| 70.00th=[ 914], 80.00th=[ 1057], 90.00th=[ 5866], 95.00th=[10028],
| 99.00th=[14484], 99.50th=[16188], 99.90th=[20055], 99.95th=[21103],
| 99.99th=[23200]
bw ( KiB/s): min=438272, max=507904, per=100.00%, avg=485662.79, stdev=12253.35, samples=120
iops : min= 428, max= 496, avg=474.27, stdev=11.97, samples=120
lat (usec) : 500=0.01%, 750=15.96%, 1000=62.64%
lat (msec) : 2=4.01%, 4=3.84%, 10=8.56%, 20=4.87%, 50=0.10%
cpu : usr=5.04%, sys=2.55%, ctx=28502, majf=0, minf=48
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,28464,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=474MiB/s (497MB/s), 474MiB/s-474MiB/s (497MB/s-497MB/s), io=27.8GiB (29.8GB), run=60038-60038msec
Disk stats (read/write):
sdc: ios=0/85401, merge=0/1, ticks=0/151025, in_queue=150169, util=85.69%
#4KB随机读
random-read: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
random-read: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [r(1)][100.0%][r=4288KiB/s,w=0KiB/s][r=1072,w=0 IOPS][eta 00m:00s]
random-read: (groupid=0, jobs=1): err= 0: pid=1411: Mon Nov 1 13:12:42 2021
read: IOPS=1071, BW=4284KiB/s (4387kB/s)(251MiB/60009msec)
slat (nsec): min=826, max=318013, avg=4466.50, stdev=3194.04
clat (usec): min=9835, max=32483, avg=14876.69, stdev=1454.66
lat (usec): min=9838, max=32487, avg=14881.15, stdev=1454.67
clat percentiles (usec):
| 1.00th=[12518], 5.00th=[13173], 10.00th=[13566], 20.00th=[13960],
| 30.00th=[14222], 40.00th=[14484], 50.00th=[14746], 60.00th=[15008],
| 70.00th=[15270], 80.00th=[15664], 90.00th=[16188], 95.00th=[16909],
| 99.00th=[19006], 99.50th=[24511], 99.90th=[28443], 99.95th=[29754],
| 99.99th=[31851]
bw ( KiB/s): min= 3968, max= 4864, per=99.97%, avg=4282.57, stdev=169.14, samples=120
iops : min= 992, max= 1216, avg=1070.58, stdev=42.28, samples=120
lat (msec) : 10=0.01%, 20=99.30%, 50=0.69%
cpu : usr=3.46%, sys=2.56%, ctx=32128, majf=0, minf=51
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.0%, 16=50.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.8%, 8=0.1%, 16=4.2%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=64272,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=4284KiB/s (4387kB/s), 4284KiB/s-4284KiB/s (4387kB/s-4387kB/s), io=251MiB (263MB), run=60009-60009msec
# 4KB随机写
Disk stats (read/write):
sdc: ios=64228/7, merge=0/5, ticks=56562/11, in_queue=56535, util=94.21%
random-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
random-write: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=47.5MiB/s][r=0,w=12.2k IOPS][eta 00m:00s]
random-write: (groupid=0, jobs=1): err= 0: pid=1415: Mon Nov 1 13:13:42 2021
write: IOPS=12.6k, BW=49.3MiB/s (51.7MB/s)(2969MiB/60163msec)
slat (nsec): min=718, max=1177.9k, avg=1558.39, stdev=1588.82
clat (usec): min=286, max=56193, avg=1246.86, stdev=1371.70
lat (usec): min=288, max=56194, avg=1248.42, stdev=1371.72
clat percentiles (usec):
| 1.00th=[ 734], 5.00th=[ 783], 10.00th=[ 807], 20.00th=[ 848],
| 30.00th=[ 906], 40.00th=[ 1004], 50.00th=[ 1090], 60.00th=[ 1156],
| 70.00th=[ 1237], 80.00th=[ 1418], 90.00th=[ 1729], 95.00th=[ 2114],
| 99.00th=[ 2868], 99.50th=[ 4015], 99.90th=[24249], 99.95th=[33817],
| 99.99th=[46400]
bw ( KiB/s): min=43000, max=58944, per=100.00%, avg=50652.78, stdev=3182.35, samples=120
iops : min=10750, max=14736, avg=12663.15, stdev=795.57, samples=120
lat (usec) : 500=0.01%, 750=1.81%, 1000=38.10%
lat (msec) : 2=53.77%, 4=5.81%, 10=0.26%, 20=0.10%, 50=0.14%
lat (msec) : 100=0.01%
cpu : usr=10.89%, sys=7.73%, ctx=361847, majf=0, minf=49
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=49.9%, 16=50.1%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=96.2%, 8=2.2%, 16=1.6%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,760012,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=49.3MiB/s (51.7MB/s), 49.3MiB/s-49.3MiB/s (51.7MB/s-51.7MB/s), io=2969MiB (3113MB), run=60163-60163msec
# 4KB顺序读
Disk stats (read/write):
sdc: ios=0/789401, merge=0/19, ticks=0/298587, in_queue=298419, util=84.25%
random-read: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [R(1)][100.0%][r=7051KiB/s,w=0KiB/s][r=1762,w=0 IOPS][eta 00m:00s]
random-read: (groupid=0, jobs=1): err= 0: pid=1422: Mon Nov 1 13:14:43 2021
read: IOPS=1711, BW=6848KiB/s (7012kB/s)(401MiB/60010msec)
slat (nsec): min=743, max=319538, avg=4090.56, stdev=2958.29
clat (usec): min=5427, max=41575, avg=9291.36, stdev=1699.89
lat (usec): min=5431, max=41581, avg=9295.45, stdev=1699.93
clat percentiles (usec):
| 1.00th=[ 6652], 5.00th=[ 7439], 10.00th=[ 7767], 20.00th=[ 8160],
| 30.00th=[ 8455], 40.00th=[ 8848], 50.00th=[ 9110], 60.00th=[ 9372],
| 70.00th=[ 9765], 80.00th=[10159], 90.00th=[10814], 95.00th=[11469],
| 99.00th=[14222], 99.50th=[18220], 99.90th=[26084], 99.95th=[28705],
| 99.99th=[41681]
bw ( KiB/s): min= 5760, max= 7824, per=100.00%, avg=6847.47, stdev=466.18, samples=120
iops : min= 1440, max= 1956, avg=1711.83, stdev=116.53, samples=120
lat (msec) : 10=76.19%, 20=23.45%, 50=0.36%
cpu : usr=5.05%, sys=3.72%, ctx=51374, majf=0, minf=53
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.0%, 16=50.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.8%, 8=0.0%, 16=4.2%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=102734,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=6848KiB/s (7012kB/s), 6848KiB/s-6848KiB/s (7012kB/s-7012kB/s), io=401MiB (421MB), run=60010-60010msec
# 4KB顺序写
Disk stats (read/write):
sdc: ios=102525/127, merge=0/1855, ticks=55303/651, in_queue=55902, util=92.20%
random-write: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=16
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [W(1)][100.0%][r=0KiB/s,w=59.4MiB/s][r=0,w=15.2k IOPS][eta 00m:00s]
random-write: (groupid=0, jobs=1): err= 0: pid=1427: Mon Nov 1 13:15:44 2021
write: IOPS=15.1k, BW=59.1MiB/s (62.0MB/s)(3552MiB/60044msec)
slat (nsec): min=718, max=575107, avg=1440.13, stdev=949.88
clat (usec): min=187, max=41588, avg=1040.16, stdev=506.15
lat (usec): min=191, max=41589, avg=1041.60, stdev=506.16
clat percentiles (usec):
| 1.00th=[ 644], 5.00th=[ 709], 10.00th=[ 742], 20.00th=[ 783],
| 30.00th=[ 816], 40.00th=[ 873], 50.00th=[ 1012], 60.00th=[ 1090],
| 70.00th=[ 1172], 80.00th=[ 1254], 90.00th=[ 1369], 95.00th=[ 1450],
| 99.00th=[ 1942], 99.50th=[ 2114], 99.90th=[ 3687], 99.95th=[ 6390],
| 99.99th=[28443]
bw ( KiB/s): min=52064, max=71257, per=100.00%, avg=60605.82, stdev=3661.06, samples=120
iops : min=13016, max=17814, avg=15151.44, stdev=915.26, samples=120
lat (usec) : 250=0.01%, 500=0.01%, 750=12.18%, 1000=36.57%
lat (msec) : 2=50.44%, 4=0.70%, 10=0.07%, 20=0.01%, 50=0.01%
cpu : usr=11.10%, sys=8.37%, ctx=432283, majf=0, minf=52
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.0%, 16=50.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=95.8%, 8=1.6%, 16=2.6%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,909196,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=59.1MiB/s (62.0MB/s), 59.1MiB/s-59.1MiB/s (62.0MB/s-62.0MB/s), io=3552MiB (3724MB), run=60044-60044msec
# 1MB顺序读
Disk stats (read/write):
sdc: ios=0/909211, merge=0/0, ticks=0/47331, in_queue=47190, util=78.00%
seq-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=posixaio, iodepth=1
fio-3.7
Starting 1 process
seq-read: Laying out IO file (1 file / 16384MiB)
Jobs: 1 (f=1): [R(1)][100.0%][r=172MiB/s,w=0KiB/s][r=172,w=0 IOPS][eta 00m:00s]
seq-read: (groupid=0, jobs=1): err= 0: pid=1431: Mon Nov 1 13:17:02 2021
read: IOPS=161, BW=161MiB/s (169MB/s)(9668MiB/60006msec)
slat (usec): min=3, max=476, avg=36.76, stdev=13.91
clat (msec): min=3, max=2004, avg= 6.15, stdev=28.76
lat (msec): min=3, max=2004, avg= 6.19, stdev=28.76
clat percentiles (msec):
| 1.00th=[ 5], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6],
| 30.00th=[ 6], 40.00th=[ 6], 50.00th=[ 6], 60.00th=[ 6],
| 70.00th=[ 7], 80.00th=[ 7], 90.00th=[ 7], 95.00th=[ 7],
| 99.00th=[ 8], 99.50th=[ 10], 99.90th=[ 16], 99.95th=[ 18],
| 99.99th=[ 2005]
bw ( KiB/s): min=20480, max=186368, per=100.00%, avg=173629.19, stdev=19275.58, samples=114
iops : min= 20, max= 182, avg=169.49, stdev=18.81, samples=114
lat (msec) : 4=0.57%, 10=98.94%, 20=0.47%
cpu : usr=1.21%, sys=1.80%, ctx=9675, majf=0, minf=51
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=9668,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=161MiB/s (169MB/s), 161MiB/s-161MiB/s (169MB/s-169MB/s), io=9668MiB (10.1GB), run=60006-60006msec
# 1MB顺序写
Disk stats (read/write):
sdc: ios=28972/5, merge=0/3, ticks=144884/7, in_queue=144375, util=93.66%
seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=posixaio, iodepth=1
fio-3.7
Starting 1 process
seq-write: Laying out IO file (1 file / 16384MiB)
Jobs: 1 (f=1): [W(1)][100.0%][r=0KiB/s,w=475MiB/s][r=0,w=475 IOPS][eta 00m:00s]
seq-write: (groupid=0, jobs=1): err= 0: pid=1435: Mon Nov 1 13:18:02 2021
write: IOPS=475, BW=475MiB/s (498MB/s)(27.9GiB/60035msec)
slat (usec): min=21, max=466, avg=86.43, stdev=21.91
clat (usec): min=504, max=33838, avg=2002.40, stdev=2933.35
lat (usec): min=549, max=33915, avg=2088.83, stdev=2931.69
clat percentiles (usec):
| 1.00th=[ 693], 5.00th=[ 758], 10.00th=[ 791], 20.00th=[ 832],
| 30.00th=[ 857], 40.00th=[ 881], 50.00th=[ 906], 60.00th=[ 930],
| 70.00th=[ 971], 80.00th=[ 1057], 90.00th=[ 5538], 95.00th=[ 9503],
| 99.00th=[13698], 99.50th=[15926], 99.90th=[19792], 99.95th=[21103],
| 99.99th=[23725]
bw ( KiB/s): min=454656, max=518144, per=100.00%, avg=486784.61, stdev=12450.43, samples=119
iops : min= 444, max= 506, avg=475.37, stdev=12.16, samples=119
lat (usec) : 750=4.25%, 1000=70.86%
lat (msec) : 2=7.98%, 4=3.72%, 10=8.85%, 20=4.27%, 50=0.09%
cpu : usr=5.70%, sys=2.63%, ctx=28533, majf=0, minf=48
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,28520,0,1 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=475MiB/s (498MB/s), 475MiB/s-475MiB/s (498MB/s-498MB/s), io=27.9GiB (29.9GB), run=60035-60035msec
Disk stats (read/write):
sdc: ios=0/85569, merge=0/1, ticks=0/150421, in_queue=149476, util=85.20%
网络测试
经典网络性能测试
测试数据
- 虚拟机到虚拟机
[cloudroot@testperf ~]$ iperf3 -c 10.127.100.234 -t 30
Connecting to host 10.127.100.234, port 5201
[ 4] local 10.127.90.238 port 56856 connected to 10.127.100.234 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 818 MBytes 6.86 Gbits/sec 32 1.09 MBytes
[ 4] 1.00-2.00 sec 1014 MBytes 8.50 Gbits/sec 0 1.64 MBytes
[ 4] 2.00-3.00 sec 1014 MBytes 8.50 Gbits/sec 0 2.02 MBytes
[ 4] 3.00-4.00 sec 975 MBytes 8.18 Gbits/sec 900 1.71 MBytes
[ 4] 4.00-5.00 sec 1022 MBytes 8.58 Gbits/sec 770 1.65 MBytes
[ 4] 5.00-6.00 sec 1.00 GBytes 8.62 Gbits/sec 0 2.03 MBytes
[ 4] 6.00-7.00 sec 1020 MBytes 8.56 Gbits/sec 0 2.30 MBytes
[ 4] 7.00-8.00 sec 1.03 GBytes 8.82 Gbits/sec 0 2.53 MBytes
[ 4] 8.00-9.00 sec 1004 MBytes 8.42 Gbits/sec 0 2.71 MBytes
[ 4] 9.00-10.00 sec 1018 MBytes 8.54 Gbits/sec 509 2.17 MBytes
[ 4] 10.00-11.00 sec 1.03 GBytes 8.82 Gbits/sec 0 2.42 MBytes
[ 4] 11.00-12.00 sec 1018 MBytes 8.54 Gbits/sec 245 2.61 MBytes
[ 4] 12.00-13.00 sec 994 MBytes 8.33 Gbits/sec 2623 1.08 MBytes
[ 4] 13.00-14.00 sec 1002 MBytes 8.41 Gbits/sec 0 1.63 MBytes
[ 4] 14.00-15.00 sec 1.01 GBytes 8.71 Gbits/sec 76 2.01 MBytes
[ 4] 15.00-16.00 sec 1014 MBytes 8.50 Gbits/sec 647 2.30 MBytes
[ 4] 16.00-17.00 sec 1.01 GBytes 8.66 Gbits/sec 87 1.98 MBytes
[ 4] 17.00-18.00 sec 1.02 GBytes 8.72 Gbits/sec 990 2.28 MBytes
[ 4] 18.00-19.00 sec 1.03 GBytes 8.85 Gbits/sec 0 2.52 MBytes
[ 4] 19.00-20.00 sec 1.01 GBytes 8.65 Gbits/sec 0 2.71 MBytes
[ 4] 20.00-21.00 sec 798 MBytes 6.69 Gbits/sec 1193 2.82 MBytes
[ 4] 21.00-22.00 sec 839 MBytes 7.04 Gbits/sec 971 2.05 MBytes
[ 4] 22.00-23.00 sec 819 MBytes 6.87 Gbits/sec 0 2.24 MBytes
[ 4] 23.00-24.00 sec 951 MBytes 7.98 Gbits/sec 1149 1.53 MBytes
[ 4] 24.00-25.00 sec 1020 MBytes 8.56 Gbits/sec 0 1.94 MBytes
[ 4] 25.00-26.00 sec 1006 MBytes 8.44 Gbits/sec 940 1.69 MBytes
[ 4] 26.00-27.00 sec 1.02 GBytes 8.76 Gbits/sec 35 1.46 MBytes
[ 4] 27.00-28.00 sec 1.04 GBytes 8.90 Gbits/sec 0 1.91 MBytes
[ 4] 28.00-29.00 sec 999 MBytes 8.38 Gbits/sec 540 2.22 MBytes
[ 4] 29.00-30.00 sec 1000 MBytes 8.39 Gbits/sec 1889 1.77 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-30.00 sec 29.1 GBytes 8.33 Gbits/sec 13596 sender
[ 4] 0.00-30.00 sec 29.1 GBytes 8.33 Gbits/sec receiver
- 虚拟机到物理机
[cloudroot@testperf ~]$ iperf3 -c 10.127.100.4 -t 30
Connecting to host 10.127.100.4, port 5201
[ 4] local 10.127.90.238 port 51382 connected to 10.127.100.4 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 927 MBytes 7.77 Gbits/sec 31 1.03 MBytes
[ 4] 1.00-2.00 sec 1018 MBytes 8.54 Gbits/sec 0 1.60 MBytes
[ 4] 2.00-3.00 sec 1.02 GBytes 8.77 Gbits/sec 0 2.02 MBytes
[ 4] 3.00-4.00 sec 844 MBytes 7.08 Gbits/sec 486 1.38 MBytes
[ 4] 4.00-5.00 sec 844 MBytes 7.08 Gbits/sec 0 1.70 MBytes
[ 4] 5.00-6.00 sec 945 MBytes 7.93 Gbits/sec 1829 512 KBytes
[ 4] 6.00-7.00 sec 836 MBytes 7.01 Gbits/sec 1525 1.58 MBytes
[ 4] 7.00-8.00 sec 1.04 GBytes 8.95 Gbits/sec 0 2.00 MBytes
[ 4] 8.00-9.00 sec 1.01 GBytes 8.66 Gbits/sec 1704 1.46 MBytes
[ 4] 9.00-10.00 sec 1.05 GBytes 9.01 Gbits/sec 160 1.61 MBytes
[ 4] 10.00-11.00 sec 938 MBytes 7.86 Gbits/sec 863 1.48 MBytes
[ 4] 11.00-12.00 sec 1.02 GBytes 8.79 Gbits/sec 814 1.18 MBytes
[ 4] 12.00-13.00 sec 1.04 GBytes 8.89 Gbits/sec 0 1.72 MBytes
[ 4] 13.00-14.00 sec 1.02 GBytes 8.76 Gbits/sec 559 1.71 MBytes
[ 4] 14.00-15.00 sec 1.01 GBytes 8.69 Gbits/sec 0 2.09 MBytes
[ 4] 15.00-16.00 sec 1.01 GBytes 8.71 Gbits/sec 818 1.36 MBytes
[ 4] 16.00-17.00 sec 981 MBytes 8.23 Gbits/sec 979 1.22 MBytes
[ 4] 17.00-18.00 sec 1010 MBytes 8.47 Gbits/sec 625 1003 KBytes
[ 4] 18.00-19.00 sec 1.01 GBytes 8.64 Gbits/sec 0 1.57 MBytes
[ 4] 19.00-20.00 sec 1019 MBytes 8.55 Gbits/sec 968 1.24 MBytes
[ 4] 20.00-21.00 sec 1.05 GBytes 9.01 Gbits/sec 0 1.77 MBytes
[ 4] 21.00-22.00 sec 1.01 GBytes 8.64 Gbits/sec 0 2.15 MBytes
[ 4] 22.00-23.00 sec 992 MBytes 8.33 Gbits/sec 492 1.45 MBytes
[ 4] 23.00-24.00 sec 1.01 GBytes 8.67 Gbits/sec 16 1.89 MBytes
[ 4] 24.00-25.00 sec 1006 MBytes 8.44 Gbits/sec 0 2.23 MBytes
[ 4] 25.00-26.00 sec 1008 MBytes 8.45 Gbits/sec 0 2.47 MBytes
[ 4] 26.00-27.00 sec 984 MBytes 8.25 Gbits/sec 1567 1.87 MBytes
[ 4] 27.00-28.00 sec 1.02 GBytes 8.77 Gbits/sec 361 1.60 MBytes
[ 4] 28.00-29.00 sec 1012 MBytes 8.49 Gbits/sec 0 1.99 MBytes
[ 4] 29.00-30.00 sec 1.04 GBytes 8.97 Gbits/sec 374 1.84 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-30.00 sec 29.4 GBytes 8.41 Gbits/sec 14171 sender
[ 4] 0.00-30.00 sec 29.4 GBytes 8.41 Gbits/sec receiver
- 物理机到虚拟机
[cloudroot@test-66-onecloud02 ~]$ iperf3 -c 10.127.100.234 -t 30
Connecting to host 10.127.100.234, port 5201
[ 4] local 10.127.100.3 port 54230 connected to 10.127.100.234 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 864 MBytes 7.24 Gbits/sec 7 1004 KBytes
[ 4] 1.00-2.00 sec 956 MBytes 8.03 Gbits/sec 0 1.31 MBytes
[ 4] 2.00-3.00 sec 985 MBytes 8.25 Gbits/sec 0 1.53 MBytes
[ 4] 3.00-4.00 sec 1019 MBytes 8.55 Gbits/sec 0 1.66 MBytes
[ 4] 4.00-5.00 sec 999 MBytes 8.38 Gbits/sec 0 1.74 MBytes
[ 4] 5.00-6.00 sec 1.01 GBytes 8.70 Gbits/sec 0 1.83 MBytes
[ 4] 6.00-7.00 sec 1018 MBytes 8.54 Gbits/sec 0 1.90 MBytes
[ 4] 7.00-8.00 sec 1019 MBytes 8.54 Gbits/sec 0 1.97 MBytes
[ 4] 8.00-9.00 sec 1024 MBytes 8.59 Gbits/sec 0 2.00 MBytes
[ 4] 9.00-10.00 sec 1020 MBytes 8.56 Gbits/sec 0 2.05 MBytes
[ 4] 10.00-11.00 sec 991 MBytes 8.31 Gbits/sec 0 2.06 MBytes
[ 4] 11.00-12.00 sec 1.01 GBytes 8.65 Gbits/sec 0 2.07 MBytes
[ 4] 12.00-13.00 sec 1018 MBytes 8.53 Gbits/sec 0 2.10 MBytes
[ 4] 13.00-14.00 sec 1.00 GBytes 8.63 Gbits/sec 0 2.12 MBytes
[ 4] 14.00-15.00 sec 996 MBytes 8.35 Gbits/sec 0 2.14 MBytes
[ 4] 15.00-16.00 sec 804 MBytes 6.75 Gbits/sec 492 1.95 MBytes
[ 4] 16.00-17.00 sec 982 MBytes 8.24 Gbits/sec 0 2.15 MBytes
[ 4] 17.00-18.00 sec 1021 MBytes 8.57 Gbits/sec 0 2.43 MBytes
[ 4] 18.00-19.00 sec 1020 MBytes 8.55 Gbits/sec 0 2.47 MBytes
[ 4] 19.00-20.00 sec 1.00 GBytes 8.61 Gbits/sec 110 1.90 MBytes
[ 4] 20.00-21.00 sec 1006 MBytes 8.44 Gbits/sec 0 2.05 MBytes
[ 4] 21.00-22.00 sec 1.01 GBytes 8.65 Gbits/sec 0 2.18 MBytes
[ 4] 22.00-23.00 sec 1006 MBytes 8.45 Gbits/sec 0 2.27 MBytes
[ 4] 23.00-24.00 sec 1.01 GBytes 8.71 Gbits/sec 0 2.34 MBytes
[ 4] 24.00-25.00 sec 1.01 GBytes 8.66 Gbits/sec 0 2.39 MBytes
[ 4] 25.00-26.00 sec 1024 MBytes 8.60 Gbits/sec 0 2.44 MBytes
[ 4] 26.00-27.00 sec 985 MBytes 8.26 Gbits/sec 0 2.46 MBytes
[ 4] 27.00-28.00 sec 1.01 GBytes 8.65 Gbits/sec 209 1.82 MBytes
[ 4] 28.00-29.00 sec 1011 MBytes 8.48 Gbits/sec 0 1.92 MBytes
[ 4] 29.00-30.00 sec 1016 MBytes 8.52 Gbits/sec 0 1.98 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-30.00 sec 29.3 GBytes 8.40 Gbits/sec 818 sender
[ 4] 0.00-30.00 sec 29.3 GBytes 8.40 Gbits/sec receiver
- 物理机到物理机
[cloudroot@test-66-onecloud02 ~]$ iperf3 -c 10.127.100.4 -t 30
Connecting to host 10.127.100.4, port 5201
[ 4] local 10.127.100.3 port 34186 connected to 10.127.100.4 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 966 MBytes 8.10 Gbits/sec 17 691 KBytes
[ 4] 1.00-2.00 sec 995 MBytes 8.35 Gbits/sec 48 991 KBytes
[ 4] 2.00-3.00 sec 1.01 GBytes 8.64 Gbits/sec 429 878 KBytes
[ 4] 3.00-4.00 sec 979 MBytes 8.21 Gbits/sec 99 857 KBytes
[ 4] 4.00-5.00 sec 972 MBytes 8.16 Gbits/sec 0 1.05 MBytes
[ 4] 5.00-6.00 sec 994 MBytes 8.34 Gbits/sec 0 1.24 MBytes
[ 4] 6.00-7.00 sec 1011 MBytes 8.48 Gbits/sec 131 868 KBytes
[ 4] 7.00-8.00 sec 1016 MBytes 8.53 Gbits/sec 242 1.08 MBytes
[ 4] 8.00-9.00 sec 945 MBytes 7.92 Gbits/sec 280 921 KBytes
[ 4] 9.00-10.00 sec 962 MBytes 8.07 Gbits/sec 79 1.14 MBytes
[ 4] 10.00-11.00 sec 985 MBytes 8.26 Gbits/sec 568 824 KBytes
[ 4] 11.00-12.00 sec 922 MBytes 7.73 Gbits/sec 239 700 KBytes
[ 4] 12.00-13.00 sec 990 MBytes 8.32 Gbits/sec 217 799 KBytes
[ 4] 13.00-14.00 sec 998 MBytes 8.37 Gbits/sec 157 949 KBytes
[ 4] 14.00-15.00 sec 948 MBytes 7.94 Gbits/sec 126 901 KBytes
[ 4] 15.00-16.00 sec 1015 MBytes 8.52 Gbits/sec 0 1.18 MBytes
[ 4] 16.00-17.00 sec 1019 MBytes 8.53 Gbits/sec 146 1.34 MBytes
[ 4] 17.00-18.00 sec 1.00 GBytes 8.63 Gbits/sec 19 1.45 MBytes
[ 4] 18.00-19.00 sec 979 MBytes 8.21 Gbits/sec 270 915 KBytes
[ 4] 19.00-20.00 sec 988 MBytes 8.28 Gbits/sec 179 823 KBytes
[ 4] 20.00-21.00 sec 1021 MBytes 8.56 Gbits/sec 184 689 KBytes
[ 4] 21.00-22.00 sec 1000 MBytes 8.39 Gbits/sec 0 1.10 MBytes
[ 4] 22.00-23.00 sec 1.02 GBytes 8.77 Gbits/sec 410 946 KBytes
[ 4] 23.00-24.00 sec 979 MBytes 8.21 Gbits/sec 120 906 KBytes
[ 4] 24.00-25.00 sec 1022 MBytes 8.57 Gbits/sec 431 892 KBytes
[ 4] 25.00-26.00 sec 988 MBytes 8.29 Gbits/sec 103 930 KBytes
[ 4] 26.00-27.00 sec 1.01 GBytes 8.68 Gbits/sec 22 870 KBytes
[ 4] 27.00-28.00 sec 1.02 GBytes 8.74 Gbits/sec 360 700 KBytes
[ 4] 28.00-29.00 sec 988 MBytes 8.28 Gbits/sec 0 1.09 MBytes
[ 4] 29.00-30.00 sec 990 MBytes 8.31 Gbits/sec 49 1.27 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-30.00 sec 29.2 GBytes 8.35 Gbits/sec 4925 sender
[ 4] 0.00-30.00 sec 29.1 GBytes 8.35 Gbits/sec receiver
VPC网络性能测试
测试数据
- 物理机到物理机:
[root@x86-node06 ~]# iperf3 -c 10.90.0.47 -t 30
Connecting to host 10.90.0.47, port 5201
[ 4] local 10.90.0.46 port 40526 connected to 10.90.0.47 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 1.08 GBytes 9.27 Gbits/sec 476 503 KBytes
[ 4] 1.00-2.00 sec 1.09 GBytes 9.35 Gbits/sec 0 720 KBytes
[ 4] 2.00-3.00 sec 1.09 GBytes 9.33 Gbits/sec 205 711 KBytes
[ 4] 3.00-4.00 sec 1.09 GBytes 9.33 Gbits/sec 90 758 KBytes
[ 4] 4.00-5.00 sec 1.09 GBytes 9.34 Gbits/sec 145 706 KBytes
[ 4] 5.00-6.00 sec 1.08 GBytes 9.26 Gbits/sec 3 535 KBytes
[ 4] 6.00-7.00 sec 1.08 GBytes 9.23 Gbits/sec 45 730 KBytes
[ 4] 7.00-8.00 sec 1.08 GBytes 9.28 Gbits/sec 308 752 KBytes
[ 4] 8.00-9.00 sec 1.07 GBytes 9.21 Gbits/sec 0 904 KBytes
[ 4] 9.00-10.00 sec 1.06 GBytes 9.12 Gbits/sec 413 839 KBytes
[ 4] 10.00-11.00 sec 1.09 GBytes 9.31 Gbits/sec 0 981 KBytes
[ 4] 11.00-12.00 sec 1.07 GBytes 9.22 Gbits/sec 0 1020 KBytes
[ 4] 12.00-13.00 sec 1.07 GBytes 9.21 Gbits/sec 448 840 KBytes
[ 4] 13.00-14.00 sec 1.09 GBytes 9.33 Gbits/sec 0 938 KBytes
[ 4] 14.00-15.00 sec 862 MBytes 7.23 Gbits/sec 182 1.36 MBytes
[ 4] 15.00-16.00 sec 1.07 GBytes 9.16 Gbits/sec 0 1.79 MBytes
[ 4] 16.00-17.00 sec 1.09 GBytes 9.40 Gbits/sec 0 1.79 MBytes
[ 4] 17.00-18.00 sec 1.07 GBytes 9.16 Gbits/sec 441 1.32 MBytes
[ 4] 18.00-19.00 sec 1.05 GBytes 9.06 Gbits/sec 53 1.03 MBytes
[ 4] 19.00-20.00 sec 1.09 GBytes 9.35 Gbits/sec 0 1.11 MBytes
[ 4] 20.00-21.00 sec 1.07 GBytes 9.22 Gbits/sec 244 798 KBytes
[ 4] 21.00-22.00 sec 1.08 GBytes 9.29 Gbits/sec 0 916 KBytes
[ 4] 22.00-23.00 sec 1.08 GBytes 9.29 Gbits/sec 337 481 KBytes
[ 4] 23.00-24.00 sec 1.07 GBytes 9.21 Gbits/sec 0 868 KBytes
[ 4] 24.00-25.00 sec 1.09 GBytes 9.35 Gbits/sec 0 952 KBytes
[ 4] 25.00-26.00 sec 1.08 GBytes 9.25 Gbits/sec 0 991 KBytes
[ 4] 26.00-27.00 sec 1.04 GBytes 8.93 Gbits/sec 0 1015 KBytes
[ 4] 27.00-28.00 sec 1.09 GBytes 9.41 Gbits/sec 0 1.02 MBytes
[ 4] 28.00-29.00 sec 1.06 GBytes 9.09 Gbits/sec 670 773 KBytes
[ 4] 29.00-30.00 sec 1.09 GBytes 9.39 Gbits/sec 0 926 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-30.00 sec 32.1 GBytes 9.19 Gbits/sec 4060 sender
[ 4] 0.00-30.00 sec 32.1 GBytes 9.19 Gbits/sec receiver
- 虚拟机到虚拟机:
[cloudroot@test-eip-46 ~]$ iperf3 -c 10.0.123.241 -t 30
Connecting to host 10.0.123.241, port 5201
[ 4] local 10.0.123.242 port 57238 connected to 10.0.123.241 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 1.01 GBytes 8.66 Gbits/sec 30 1.67 MBytes
[ 4] 1.00-2.00 sec 1.03 GBytes 8.87 Gbits/sec 208 1.61 MBytes
[ 4] 2.00-3.00 sec 1.05 GBytes 9.02 Gbits/sec 7 1.57 MBytes
[ 4] 3.00-4.00 sec 1.05 GBytes 9.04 Gbits/sec 1 1.52 MBytes
[ 4] 4.00-5.00 sec 1.03 GBytes 8.86 Gbits/sec 189 1.40 MBytes
[ 4] 5.00-6.00 sec 1.04 GBytes 8.95 Gbits/sec 24 1.35 MBytes
[ 4] 6.00-7.00 sec 1.01 GBytes 8.65 Gbits/sec 0 1.81 MBytes
[ 4] 7.00-8.00 sec 1.05 GBytes 9.03 Gbits/sec 4 1.76 MBytes
[ 4] 8.00-9.00 sec 1.04 GBytes 8.98 Gbits/sec 3 1.69 MBytes
[ 4] 9.00-10.00 sec 1.05 GBytes 9.03 Gbits/sec 6 1.62 MBytes
[ 4] 10.00-11.00 sec 1.04 GBytes 8.97 Gbits/sec 3 1.53 MBytes
[ 4] 11.00-12.00 sec 1.05 GBytes 8.99 Gbits/sec 15 1.46 MBytes
[ 4] 12.00-13.00 sec 1.05 GBytes 9.03 Gbits/sec 2 1.36 MBytes
[ 4] 13.00-14.00 sec 1.04 GBytes 8.97 Gbits/sec 708 1.37 MBytes
[ 4] 14.00-15.00 sec 1.04 GBytes 8.95 Gbits/sec 771 1.39 MBytes
[ 4] 15.00-16.00 sec 1.05 GBytes 9.01 Gbits/sec 0 1.86 MBytes
[ 4] 16.00-17.00 sec 1.05 GBytes 9.03 Gbits/sec 19 1.79 MBytes
[ 4] 17.00-18.00 sec 1.05 GBytes 9.04 Gbits/sec 2 1.72 MBytes
[ 4] 18.00-19.00 sec 1.05 GBytes 9.02 Gbits/sec 2 1.66 MBytes
[ 4] 19.00-20.00 sec 1.05 GBytes 9.03 Gbits/sec 22 1.57 MBytes
[ 4] 20.00-21.00 sec 1.05 GBytes 9.01 Gbits/sec 8 1.48 MBytes
[ 4] 21.00-22.00 sec 1.04 GBytes 8.95 Gbits/sec 1 1.39 MBytes
[ 4] 22.00-23.00 sec 1.05 GBytes 9.01 Gbits/sec 2 1.32 MBytes
[ 4] 23.00-24.00 sec 1.04 GBytes 8.97 Gbits/sec 0 1.80 MBytes
[ 4] 24.00-25.00 sec 1.05 GBytes 9.02 Gbits/sec 3 1.76 MBytes
[ 4] 25.00-26.00 sec 1.02 GBytes 8.74 Gbits/sec 3 1.67 MBytes
[ 4] 26.00-27.00 sec 1.04 GBytes 8.91 Gbits/sec 16 1.58 MBytes
[ 4] 27.00-28.00 sec 1.05 GBytes 9.01 Gbits/sec 2 1.48 MBytes
[ 4] 28.00-29.00 sec 1.04 GBytes 8.98 Gbits/sec 0 1.93 MBytes
[ 4] 29.00-30.00 sec 1.05 GBytes 9.03 Gbits/sec 1 1.80 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-30.00 sec 31.3 GBytes 8.96 Gbits/sec 2052 sender
[ 4] 0.00-30.00 sec 31.3 GBytes 8.96 Gbits/sec receiver
虚拟机EIP到虚拟机EIP：
[cloudroot@test-eip-46 ~]$ iperf3 -c 100.89.2.2 -t 30
Connecting to host 100.89.2.2, port 5201
[ 4] local 10.0.123.242 port 55730 connected to 100.89.2.2 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 663 MBytes 5.56 Gbits/sec 212 756 KBytes
[ 4] 1.00-2.00 sec 694 MBytes 5.82 Gbits/sec 55 706 KBytes
[ 4] 2.00-3.00 sec 688 MBytes 5.77 Gbits/sec 111 795 KBytes
[ 4] 3.00-4.00 sec 694 MBytes 5.82 Gbits/sec 26 778 KBytes
[ 4] 4.00-5.00 sec 696 MBytes 5.84 Gbits/sec 32 744 KBytes
[ 4] 5.00-6.00 sec 696 MBytes 5.84 Gbits/sec 23 965 KBytes
[ 4] 6.00-7.00 sec 692 MBytes 5.81 Gbits/sec 47 930 KBytes
[ 4] 7.00-8.00 sec 695 MBytes 5.83 Gbits/sec 30 884 KBytes
[ 4] 8.00-9.00 sec 672 MBytes 5.64 Gbits/sec 56 863 KBytes
[ 4] 9.00-10.00 sec 681 MBytes 5.71 Gbits/sec 22 827 KBytes
[ 4] 10.00-11.00 sec 685 MBytes 5.75 Gbits/sec 64 775 KBytes
[ 4] 11.00-12.00 sec 674 MBytes 5.65 Gbits/sec 92 790 KBytes
[ 4] 12.00-13.00 sec 688 MBytes 5.77 Gbits/sec 28 766 KBytes
[ 4] 13.00-14.00 sec 696 MBytes 5.84 Gbits/sec 39 741 KBytes
[ 4] 14.00-15.00 sec 681 MBytes 5.71 Gbits/sec 13 1003 KBytes
[ 4] 15.00-16.00 sec 685 MBytes 5.75 Gbits/sec 60 714 KBytes
[ 4] 16.00-17.00 sec 635 MBytes 5.33 Gbits/sec 12 1000 KBytes
[ 4] 17.00-18.00 sec 704 MBytes 5.90 Gbits/sec 206 721 KBytes
[ 4] 18.00-19.00 sec 688 MBytes 5.77 Gbits/sec 52 741 KBytes
[ 4] 19.00-20.00 sec 694 MBytes 5.82 Gbits/sec 12 753 KBytes
[ 4] 20.00-21.00 sec 728 MBytes 6.10 Gbits/sec 35 747 KBytes
[ 4] 21.00-22.00 sec 714 MBytes 5.99 Gbits/sec 117 759 KBytes
[ 4] 22.00-23.00 sec 726 MBytes 6.09 Gbits/sec 86 797 KBytes
[ 4] 23.00-24.00 sec 731 MBytes 6.13 Gbits/sec 43 772 KBytes
[ 4] 24.00-25.00 sec 708 MBytes 5.94 Gbits/sec 24 719 KBytes
[ 4] 25.00-26.00 sec 696 MBytes 5.84 Gbits/sec 61 1019 KBytes
[ 4] 26.00-27.00 sec 725 MBytes 6.08 Gbits/sec 41 760 KBytes
[ 4] 27.00-28.00 sec 711 MBytes 5.97 Gbits/sec 17 744 KBytes
[ 4] 28.00-29.00 sec 695 MBytes 5.83 Gbits/sec 18 791 KBytes
[ 4] 29.00-30.00 sec 710 MBytes 5.96 Gbits/sec 11 806 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-30.00 sec 20.4 GBytes 5.83 Gbits/sec 1645 sender
[ 4] 0.00-30.00 sec 20.4 GBytes 5.83 Gbits/sec receiver