Очень часто наши клиенты интересуются, отчего такая разница в цене между VPS на облачной платформе и VPS/KVM с выделенными накопителями. Почему виртуальный сервер с выделенными накопителями значительно дороже и есть ли смысл переплачивать? Ведь стоимость виртуального сервера на облачной платформе лежит в пределах $3.99-$32.99 против $39 у VPS/KVM с выделенными накопителями, где ресурсов CPU / RAM оказывается значительно меньше.
На волне данных вопросов было решено провести краткое исследование, сравнение и оценку возможностей данных позиций. Конечно, нельзя полноценно сравнить данные услуги из-за различия в количестве выделенной памяти, типе процессоров, дисковой квоты и лимита трафика. Но одну вещь мы все же смогли полноценно исследовать — производительность дисковой подсистемы.
Для более
Тест проводился на следующих услугах — VPS, VPS-KVM, Dedicated server.
VPS: виртуализация VMware, ssd storage.
VPS-KVM: виртуализация KVM, для каждой VPS выделен свой ssd накопитель.
Dedicated server: сервер с ssd накопителями.
Программ и вариантов для измерения скорости дисков существует довольно много:
iozone <www.iozone.org>
iometr <www.iometer.org>
fio <freecode.com/projects/fio>
bonnie++ <www.coker.com.au/bonnie++>
вплоть до банального dd, и cp с замером времени выполнения.
Для начала стоит решить, что именно для Вас является показателем производительности диска, я считаю, что главным критерием является количество IOPS и значение latency, в связи с чем для тестов и была выбрана утилита fio.
Так как целью данного теста не являлось показать большую и красивую цифру максимальных IOPS, а производительность в обстановке приближенной к боевой, то был выбран самый жесткий вариант нагрузки — параллельный тест на случайную запись и случайное чтение (randread, randwrite).
При использовании fio следует не только понимать, как расшифровывать результаты, но и грамотно подобрать параметры для теста. На эту тему есть хорошая статья от amarao.
Конфиг-файл теста:
[readtest]
blocksize=4k
filename=/dev/sdb
rw=randread
ioengine=libaio
buffered=0
direct=1
iodepth=32
[writetest]
blocksize=4k
filename=/dev/sdb
rw=randwrite
ioengine=libaio
buffered=0
direct=1
iodepth=32
Краткое описание содержания задающего тест файла
blocksize — размер блока которым ведется чтение\запись.
filename — имя файла на котором проводится тестирование, можно задать весь диск.
rw — метод тестирования (использовалась случайная запись, и случайное чтение).
ioengine — метод работы с дисковой подсистемой (задан libaio, как самый производительный).
Изменение iodepth задаст допустимую глубину очереди запроса на IO, при увеличении данного параметра повысится количество выдаваемых IOPS, но возрастет latency.
buffered=0, direct=1 — отключаем буферизацию IO со стороны ОС.
Т.к. randrw дает несколько некорректный результат (сначала идет тест на запись, потом тест на чтение), то в конфиг-файле задаем два параллельных теста — один тест на случайное чтение (randread), второй на случайную запись (randwrite).
При тесте выделенного сервера областью теста брался весь диск (в данном случае sdb), в тестах на VPS создавался файл максимально допустимого размера. Это было сделано для наибольшей утилизации диска при проведении теста (нас же интересует реальная производительность, а не красивый результат).
Для чистоты эксперимента, перед тестом диск затирался командой:
#dd if=/dev/zero of=/dev/sdb bs=2M oflag=direct
Для быстрого анализа достаточно посмотреть на данные по IOPS (wiops/riops) и latency:
slat — время потребовавшееся на отправку запроса.
clat — время до завершение операций IO (completion latency).
Краткая сводка полученных результатов теста каждой конфигурации при iodepht=64(32+32):
?Cloud VPS: read iops=808, clat avg=19739.76 | write iops=894, clat avg=17806.34
VPS/KVM: read iops=11940, clat avg=2655.36 | write iops=11245, clat avg=2824.64
Dedicated server: read iops=16201, clat avg=1968.08 | write iops=16083, clat avg=1994.89
Значение clat предоставляется в usec, 19739usec = 19.73msec.
Время тестов варьировалось от часа до шести. Также стоит учесть, что ssd накопители показывают меньшую производительность при такой нагрузке длящейся более 12-ти часов из-за работы housekeeping'a.
Стоит отметить, что цифра в 770 IOPS и latensy в 10msec при iodepht16 — неплохой результат для VPS в сравнении с количеством IOPS, которые может предоставить SATA-диск.
Как и ожидалось, VPS намного отстал от своих «старших братьев», но VPS\KVM показал результаты немногим меньшие от результатов выделенного сервера (!), правда, с задействованным только одним диском.?
Как и все синтетические тесты, данный тест обладает своими недостатками, но он может дать представление о количестве доступных IOPS и latency, в зависимости от количества потоков.
Как уже упоминалось ранее, для данных тестов был выбран наихудший вариант нагрузки — параллельный запуск случайного чтения и случайной записи блоками по 4 kb, но на практике результат может отличаться, чаще в лучшую сторону. Тесты можно и нужно использовать для предварительной оценки площадки под проект, но полностью картину можно оценить только после боевого запуска.
Однозначно сказать какой вариант лучше нельзя, т.к. каждый из них обладает своими достоинствами и недостатками, и имеет право на жизнь в определенных ситуациях.
Согласитесь, ведь не все проекты могут себе позволить сразу арендовать сервер с SSD накопителями и намного большим количеством памяти, чем требует проект на данный момент, а собирать сервер custom built иногда выходит дороже, чем взять некоторые акционные предложения. И совсем нецелесообразно арендовать сервер c 16 ГБ памяти для перспективного, но еще развивающегося проекта, который на данном этапе вполне впишется в один из тарифов VPS.
Нельзя сразу говорить, что «дешево означает плохо», в данном случае это утверждение не работает.
Ниже предоставлен полный вывод результатов теста данных услуг с различным значением iodepht.
Cloud VPS iodepth32 (total-64)
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
fio-2.0.13
Starting 2 processes
readtest: (groupid=0, jobs=1): err= 0: pid=19975: Mon Oct 19 15:16:56 2015
read: io=35230MB, bw=3232.8KB/s, iops=808, runt=11159440msec
slat (usec): min=4, max=117491, avg=47.43, stdev=235.20
clat (usec): min=276, max=604321, avg=19739.76, stdev=15515.99
lat (usec): min=305, max=604346, avg=19788.62, stdev=15522.33
clat percentiles (msec):
| 1.00th=[ 3], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 10],
| 30.00th=[ 11], 40.00th=[ 14], 50.00th=[ 18], 60.00th=[ 20],
| 70.00th=[ 23], 80.00th=[ 29], 90.00th=[ 37], 95.00th=[ 45],
| 99.00th=[ 68], 99.50th=[ 84], 99.90th=[ 145], 99.95th=[ 184],
| 99.99th=[ 523]
bw (KB/s): min= 5, max=18056, per=99.99%, avg=3231.58, stdev=1601.26
lat (usec): 500=0.01%, 750=0.01%, 1000=0.03%
lat (msec): 2=0.41%, 4=2.27%, 10=21.99%, 20=35.93%, 50=36.25%
lat (msec): 100=2.82%, 250=0.27%, 500=0.01%, 750=0.01%
cpu: usr=1.41%, sys=5.64%, ctx=5865491, majf=0, minf=39
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=9018850/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=19976: Mon Oct 19 15:16:56 2015
write: io=34162MB, bw=3579.1KB/s, iops=894, runt=9771782msec
slat (usec): min=6, max=117450, avg=61.26, stdev=259.97
clat (usec): min=495, max=554638, avg=17806.34, stdev=12969.12
lat (usec): min=667, max=554662, avg=17869.08, stdev=12974.63
clat percentiles (msec):
| 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 10],
| 30.00th=[ 11], 40.00th=[ 12], 50.00th=[ 14], 60.00th=[ 18],
| 70.00th=[ 21], 80.00th=[ 24], 90.00th=[ 32], 95.00th=[ 39],
| 99.00th=[ 63], 99.50th=[ 81], 99.90th=[ 143], 99.95th=[ 174],
| 99.99th=[ 243]
bw (KB/s): min= 331, max= 6459, per=100.00%, avg=3584.60, stdev=843.67
lat (usec): 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2=0.01%, 4=0.05%, 10=23.37%, 20=46.53%, 50=27.96%
lat (msec): 100=1.79%, 250=0.27%, 500=0.01%, 750=0.01%
cpu: usr=1.80%, sys=7.66%, ctx=7102973, majf=0, minf=24
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=8745453/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=35230MB, aggrb=3232KB/s, minb=3232KB/s, maxb=3232KB/s, mint=11159440msec, maxt=11159440msec
WRITE: io=34162MB, aggrb=3579KB/s, minb=3579KB/s, maxb=3579KB/s, mint=9771782msec, maxt=9771782msec
Disk stats (read/write):
vda: ios=9018855/8752140, merge=0/2669, ticks=177404092/155390140, in_queue=332779245, util=100.00%
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
fio-2.0.13
Starting 2 processes
readtest: (groupid=0, jobs=1): err= 0: pid=19975: Mon Oct 19 15:16:56 2015
read: io=35230MB, bw=3232.8KB/s, iops=808, runt=11159440msec
slat (usec): min=4, max=117491, avg=47.43, stdev=235.20
clat (usec): min=276, max=604321, avg=19739.76, stdev=15515.99
lat (usec): min=305, max=604346, avg=19788.62, stdev=15522.33
clat percentiles (msec):
| 1.00th=[ 3], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 10],
| 30.00th=[ 11], 40.00th=[ 14], 50.00th=[ 18], 60.00th=[ 20],
| 70.00th=[ 23], 80.00th=[ 29], 90.00th=[ 37], 95.00th=[ 45],
| 99.00th=[ 68], 99.50th=[ 84], 99.90th=[ 145], 99.95th=[ 184],
| 99.99th=[ 523]
bw (KB/s): min= 5, max=18056, per=99.99%, avg=3231.58, stdev=1601.26
lat (usec): 500=0.01%, 750=0.01%, 1000=0.03%
lat (msec): 2=0.41%, 4=2.27%, 10=21.99%, 20=35.93%, 50=36.25%
lat (msec): 100=2.82%, 250=0.27%, 500=0.01%, 750=0.01%
cpu: usr=1.41%, sys=5.64%, ctx=5865491, majf=0, minf=39
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=9018850/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=19976: Mon Oct 19 15:16:56 2015
write: io=34162MB, bw=3579.1KB/s, iops=894, runt=9771782msec
slat (usec): min=6, max=117450, avg=61.26, stdev=259.97
clat (usec): min=495, max=554638, avg=17806.34, stdev=12969.12
lat (usec): min=667, max=554662, avg=17869.08, stdev=12974.63
clat percentiles (msec):
| 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 10],
| 30.00th=[ 11], 40.00th=[ 12], 50.00th=[ 14], 60.00th=[ 18],
| 70.00th=[ 21], 80.00th=[ 24], 90.00th=[ 32], 95.00th=[ 39],
| 99.00th=[ 63], 99.50th=[ 81], 99.90th=[ 143], 99.95th=[ 174],
| 99.99th=[ 243]
bw (KB/s): min= 331, max= 6459, per=100.00%, avg=3584.60, stdev=843.67
lat (usec): 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2=0.01%, 4=0.05%, 10=23.37%, 20=46.53%, 50=27.96%
lat (msec): 100=1.79%, 250=0.27%, 500=0.01%, 750=0.01%
cpu: usr=1.80%, sys=7.66%, ctx=7102973, majf=0, minf=24
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=8745453/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=35230MB, aggrb=3232KB/s, minb=3232KB/s, maxb=3232KB/s, mint=11159440msec, maxt=11159440msec
WRITE: io=34162MB, aggrb=3579KB/s, minb=3579KB/s, maxb=3579KB/s, mint=9771782msec, maxt=9771782msec
Disk stats (read/write):
vda: ios=9018855/8752140, merge=0/2669, ticks=177404092/155390140, in_queue=332779245, util=100.00%
VPS/KVM iodepth32 (total-64)
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1.3
Starting 2 processes
readtest: (groupid=0, jobs=1): err= 0: pid=2533: Tue Oct 20 10:38:14 2015
read: io=91806MB, bw=47763KB/s, iops=11940, runt=1968229msec
slat (usec): min=0, max=64513, avg=21.14, stdev=178.81
clat (usec): min=0, max=2261.3K, avg=2655.36, stdev=17141.37
lat (usec): min=15, max=2261.3K, avg=2676.95, stdev=17142.70
clat percentiles (usec):
| 1.00th=[ 30], 5.00th=[ 44], 10.00th=[ 67], 20.00th=[ 326],
| 30.00th=[ 620], 40.00th=[ 860], 50.00th=[ 1112], 60.00th=[ 1416],
| 70.00th=[ 1832], 80.00th=[ 2512], 90.00th=[ 3824], 95.00th=[ 5664],
| 99.00th=[18816], 99.50th=[36096], 99.90th=[280576], 99.95th=[301056],
| 99.99th=[667648]
bw (KB /s): min= 818, max=128048, per=100.00%, avg=49429.61, stdev=20535.78
lat (usec): 2=0.01%, 4=0.01%, 10=0.01%, 20=0.03%, 50=6.39%
lat (usec): 100=7.58%, 250=4.10%, 500=7.22%, 750=9.98%, 1000=10.55%
lat (msec): 2=27.13%, 4=17.75%, 10=7.07%, 20=1.26%, 50=0.50%
lat (msec): 100=0.13%, 250=0.14%, 500=0.13%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.01%, >=2000=0.01%
cpu: usr=5.54%, sys=16.09%, ctx=8334339, majf=0, minf=59
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=23502336/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=2534: Tue Oct 20 10:38:14 2015
write: io=91806MB, bw=44983KB/s, iops=11245, runt=2089898msec
slat (usec): min=0, max=72993, avg=18.01, stdev=152.10
clat (usec): min=49, max=1414.8K, avg=2824.64, stdev=18128.78
lat (usec): min=112, max=1414.8K, avg=2843.15, stdev=18130.31
clat percentiles (usec):
| 1.00th=[ 306], 5.00th=[ 394], 10.00th=[ 458], 20.00th=[ 548],
| 30.00th=[ 612], 40.00th=[ 676], 50.00th=[ 724], 60.00th=[ 780],
| 70.00th=[ 852], 80.00th=[ 1128], 90.00th=[ 3696], 95.00th=[ 6304],
| 99.00th=[22144], 99.50th=[102912], 99.90th=[288768], 99.95th=[292864],
| 99.99th=[305152]
bw (KB /s): min= 665, max=163536, per=100.00%, avg=46729.97, stdev=43649.73
lat (usec): 50=0.01%, 100=0.01%, 250=0.27%, 500=14.03%, 750=40.36%
lat (usec): 1000=22.53%
lat (msec): 2=7.36%, 4=6.19%, 10=6.77%, 20=1.39%, 50=0.48%
lat (msec): 100=0.12%, 250=0.23%, 500=0.27%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.01%
cpu: usr=4.17%, sys=11.85%, ctx=3279482, majf=0, minf=29
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=23502336/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=91806MB, aggrb=47763KB/s, minb=47763KB/s, maxb=47763KB/s, mint=1968229msec, maxt=1968229msec
WRITE: io=91806MB, aggrb=44982KB/s, minb=44982KB/s, maxb=44982KB/s, mint=2089898msec, maxt=2089898msec
Disk stats (read/write):
vda: ios=23502336/23493070, merge=0/596, ticks=58747256/62913768, in_queue=121664488, util=100.00%
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1.3
Starting 2 processes
readtest: (groupid=0, jobs=1): err= 0: pid=2533: Tue Oct 20 10:38:14 2015
read: io=91806MB, bw=47763KB/s, iops=11940, runt=1968229msec
slat (usec): min=0, max=64513, avg=21.14, stdev=178.81
clat (usec): min=0, max=2261.3K, avg=2655.36, stdev=17141.37
lat (usec): min=15, max=2261.3K, avg=2676.95, stdev=17142.70
clat percentiles (usec):
| 1.00th=[ 30], 5.00th=[ 44], 10.00th=[ 67], 20.00th=[ 326],
| 30.00th=[ 620], 40.00th=[ 860], 50.00th=[ 1112], 60.00th=[ 1416],
| 70.00th=[ 1832], 80.00th=[ 2512], 90.00th=[ 3824], 95.00th=[ 5664],
| 99.00th=[18816], 99.50th=[36096], 99.90th=[280576], 99.95th=[301056],
| 99.99th=[667648]
bw (KB /s): min= 818, max=128048, per=100.00%, avg=49429.61, stdev=20535.78
lat (usec): 2=0.01%, 4=0.01%, 10=0.01%, 20=0.03%, 50=6.39%
lat (usec): 100=7.58%, 250=4.10%, 500=7.22%, 750=9.98%, 1000=10.55%
lat (msec): 2=27.13%, 4=17.75%, 10=7.07%, 20=1.26%, 50=0.50%
lat (msec): 100=0.13%, 250=0.14%, 500=0.13%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.01%, >=2000=0.01%
cpu: usr=5.54%, sys=16.09%, ctx=8334339, majf=0, minf=59
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=23502336/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=2534: Tue Oct 20 10:38:14 2015
write: io=91806MB, bw=44983KB/s, iops=11245, runt=2089898msec
slat (usec): min=0, max=72993, avg=18.01, stdev=152.10
clat (usec): min=49, max=1414.8K, avg=2824.64, stdev=18128.78
lat (usec): min=112, max=1414.8K, avg=2843.15, stdev=18130.31
clat percentiles (usec):
| 1.00th=[ 306], 5.00th=[ 394], 10.00th=[ 458], 20.00th=[ 548],
| 30.00th=[ 612], 40.00th=[ 676], 50.00th=[ 724], 60.00th=[ 780],
| 70.00th=[ 852], 80.00th=[ 1128], 90.00th=[ 3696], 95.00th=[ 6304],
| 99.00th=[22144], 99.50th=[102912], 99.90th=[288768], 99.95th=[292864],
| 99.99th=[305152]
bw (KB /s): min= 665, max=163536, per=100.00%, avg=46729.97, stdev=43649.73
lat (usec): 50=0.01%, 100=0.01%, 250=0.27%, 500=14.03%, 750=40.36%
lat (usec): 1000=22.53%
lat (msec): 2=7.36%, 4=6.19%, 10=6.77%, 20=1.39%, 50=0.48%
lat (msec): 100=0.12%, 250=0.23%, 500=0.27%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.01%
cpu: usr=4.17%, sys=11.85%, ctx=3279482, majf=0, minf=29
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=23502336/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=91806MB, aggrb=47763KB/s, minb=47763KB/s, maxb=47763KB/s, mint=1968229msec, maxt=1968229msec
WRITE: io=91806MB, aggrb=44982KB/s, minb=44982KB/s, maxb=44982KB/s, mint=2089898msec, maxt=2089898msec
Disk stats (read/write):
vda: ios=23502336/23493070, merge=0/596, ticks=58747256/62913768, in_queue=121664488, util=100.00%
Dedicated server iodepth32 (total-64)
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.0.13
Starting 2 processes
readtest: (groupid=0, jobs=1): err= 0: pid=20741: Tue Oct 20 10:59:38 2015
read: io=114473MB, bw=64571KB/s, iops=16142, runt=1815385msec
slat (usec): min=1, max=2302, avg= 5.61, stdev= 3.60
clat (usec): min=52, max=125404, avg=1975.28, stdev=9778.95
lat (usec): min=106, max=125409, avg=1981.13, stdev=9779.04
clat percentiles (usec):
| 1.00th=[ 231], 5.00th=[ 274], 10.00th=[ 290], 20.00th=[ 310],
| 30.00th=[ 326], 40.00th=[ 342], 50.00th=[ 362], 60.00th=[ 406],
| 70.00th=[ 660], 80.00th=[ 1592], 90.00th=[ 3472], 95.00th=[ 4768],
| 99.00th=[ 9920], 99.50th=[103936], 99.90th=[109056], 99.95th=[110080],
| 99.99th=[112128]
bw (KB/s): min=13667, max=371800, per=100.00%, avg=65031.79, stdev=110063.72
lat (usec): 100=0.01%, 250=1.82%, 500=64.95%, 750=4.70%, 1000=3.74%
lat (msec): 2=7.27%, 4=9.78%, 10=6.76%, 20=0.11%, 50=0.01%
lat (msec): 100=0.02%, 250=0.86%
cpu: usr=3.75%, sys=14.29%, ctx=11609454, majf=0, minf=55
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=29305206/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=20742: Tue Oct 20 10:59:38 2015
write: io=108250MB, bw=64135KB/s, iops=16083, runt=1603347msec
slat (usec): min=1, max=2401, avg= 5.97, stdev= 4.04
clat (usec): min=54, max=128975, avg=1984.89, stdev=9781.75
lat (usec): min=111, max=128980, avg=1850.16, stdev=9782.83
clat percentiles (usec):
| 1.00th=[ 84], 5.00th=[ 127], 10.00th=[ 175], 20.00th=[ 258],
| 30.00th=[ 338], 40.00th=[ 418], 50.00th=[ 502], 60.00th=[ 588],
| 70.00th=[ 732], 80.00th=[ 1096], 90.00th=[ 2480], 95.00th=[ 3888],
| 99.00th=[10304], 99.50th=[101888], 99.90th=[104960], 99.95th=[105984],
| 99.99th=[109056]
bw (KB/s): min=15105, max=185808, per=99.12%, avg=68527.72, stdev=37578.10
lat (usec): 20=0.01%, 50=0.05%, 100=2.08%, 250=16.78%, 500=30.99%
lat (usec): 750=21.01%, 1000=7.26%
lat (msec): 2=9.00%, 4=8.08%, 10=3.73%, 20=0.09%, 50=0.01%
lat (msec): 100=0.03%, 250=0.90%
cpu: usr=2.98%, sys=14.94%, ctx=11106546, majf=0, minf=24
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=27712022/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=114473MB, aggrb=64570KB/s, minb=64570KB/s, maxb=64570KB/s, mint=1815385msec, maxt=1815385msec
WRITE: io=121250MB, aggrb=63135KB/s, minb=63135KB/s, maxb=63135KB/s, mint=1823347msec, maxt=1823347msec
Disk stats (read/write):
sdb: ios=29261256/27442451, merge=41072/269571, ticks=57451178/50361986, in_queue=107793663, util=100.00%
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.0.13
Starting 2 processes
readtest: (groupid=0, jobs=1): err= 0: pid=20741: Tue Oct 20 10:59:38 2015
read: io=114473MB, bw=64571KB/s, iops=16142, runt=1815385msec
slat (usec): min=1, max=2302, avg= 5.61, stdev= 3.60
clat (usec): min=52, max=125404, avg=1975.28, stdev=9778.95
lat (usec): min=106, max=125409, avg=1981.13, stdev=9779.04
clat percentiles (usec):
| 1.00th=[ 231], 5.00th=[ 274], 10.00th=[ 290], 20.00th=[ 310],
| 30.00th=[ 326], 40.00th=[ 342], 50.00th=[ 362], 60.00th=[ 406],
| 70.00th=[ 660], 80.00th=[ 1592], 90.00th=[ 3472], 95.00th=[ 4768],
| 99.00th=[ 9920], 99.50th=[103936], 99.90th=[109056], 99.95th=[110080],
| 99.99th=[112128]
bw (KB/s): min=13667, max=371800, per=100.00%, avg=65031.79, stdev=110063.72
lat (usec): 100=0.01%, 250=1.82%, 500=64.95%, 750=4.70%, 1000=3.74%
lat (msec): 2=7.27%, 4=9.78%, 10=6.76%, 20=0.11%, 50=0.01%
lat (msec): 100=0.02%, 250=0.86%
cpu: usr=3.75%, sys=14.29%, ctx=11609454, majf=0, minf=55
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=29305206/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=20742: Tue Oct 20 10:59:38 2015
write: io=108250MB, bw=64135KB/s, iops=16083, runt=1603347msec
slat (usec): min=1, max=2401, avg= 5.97, stdev= 4.04
clat (usec): min=54, max=128975, avg=1984.89, stdev=9781.75
lat (usec): min=111, max=128980, avg=1850.16, stdev=9782.83
clat percentiles (usec):
| 1.00th=[ 84], 5.00th=[ 127], 10.00th=[ 175], 20.00th=[ 258],
| 30.00th=[ 338], 40.00th=[ 418], 50.00th=[ 502], 60.00th=[ 588],
| 70.00th=[ 732], 80.00th=[ 1096], 90.00th=[ 2480], 95.00th=[ 3888],
| 99.00th=[10304], 99.50th=[101888], 99.90th=[104960], 99.95th=[105984],
| 99.99th=[109056]
bw (KB/s): min=15105, max=185808, per=99.12%, avg=68527.72, stdev=37578.10
lat (usec): 20=0.01%, 50=0.05%, 100=2.08%, 250=16.78%, 500=30.99%
lat (usec): 750=21.01%, 1000=7.26%
lat (msec): 2=9.00%, 4=8.08%, 10=3.73%, 20=0.09%, 50=0.01%
lat (msec): 100=0.03%, 250=0.90%
cpu: usr=2.98%, sys=14.94%, ctx=11106546, majf=0, minf=24
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=27712022/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=114473MB, aggrb=64570KB/s, minb=64570KB/s, maxb=64570KB/s, mint=1815385msec, maxt=1815385msec
WRITE: io=121250MB, aggrb=63135KB/s, minb=63135KB/s, maxb=63135KB/s, mint=1823347msec, maxt=1823347msec
Disk stats (read/write):
sdb: ios=29261256/27442451, merge=41072/269571, ticks=57451178/50361986, in_queue=107793663, util=100.00%
Cloud VPS iodepth24 (total-48)
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=24
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=24
fio-2.0.13
Starting 2 processes
readtest: (groupid=0, jobs=1): err= 0: pid=22687: Tue Oct 20 12:28:31 2015
read: io=35332MB, bw=3324.1KB/s, iops=831, runt=10881292msec
slat (usec): min=4, max=178328, avg=45.46, stdev=304.03
clat (usec): min=308, max=652088, avg=28818.31, stdev=20780.24
lat (usec): min=321, max=652111, avg=28865.10, stdev=20787.02
clat percentiles (msec):
| 1.00th=[ 5], 5.00th=[ 8], 10.00th=[ 11], 20.00th=[ 14],
| 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 25], 60.00th=[ 30],
| 70.00th=[ 34], 80.00th=[ 42], 90.00th=[ 53], 95.00th=[ 65],
| 99.00th=[ 93], 99.50th=[ 109], 99.90th=[ 174], 99.95th=[ 223],
| 99.99th=[ 545]
bw (KB/s): min= 406, max=16088, per=99.95%, avg=3322.21, stdev=1580.93
lat (usec): 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2=0.15%, 4=0.77%, 10=8.90%, 20=30.75%, 50=47.80%
lat (msec): 100=10.92%, 250=0.68%, 500=0.02%, 750=0.02%
cpu: usr=1.39%, sys=5.44%, ctx=5757944, majf=0, minf=47
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=9044992/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=22688: Tue Oct 20 12:28:31 2015
write: io=33974MB, bw=3700.4KB/s, iops=925, runt=9401627msec
slat (usec): min=6, max=198169, avg=57.22, stdev=335.09
clat (usec): min=542, max=561342, avg=25877.12, stdev=17322.27
lat (usec): min=625, max=561368, avg=25935.79, stdev=17328.52
clat percentiles (msec):
| 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 14],
| 30.00th=[ 16], 40.00th=[ 18], 50.00th=[ 21], 60.00th=[ 26],
| 70.00th=[ 30], 80.00th=[ 36], 90.00th=[ 46], 95.00th=[ 58],
| 99.00th=[ 85], 99.50th=[ 101], 99.90th=[ 165], 99.95th=[ 204],
| 99.99th=[ 297]
bw (KB/s): min= 344, max= 6432, per=100.00%, avg=3704.64, stdev=828.54
lat (usec): 750=0.01%, 1000=0.01%
lat (msec): 2=0.01%, 4=0.02%, 10=4.49%, 20=45.39%, 50=42.45%
lat (msec): 100=7.14%, 250=0.49%, 500=0.02%, 750=0.01%
cpu: usr=1.79%, sys=7.31%, ctx=6996874, majf=0, minf=24
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=8697222/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=35332MB, aggrb=3324KB/s, minb=3324KB/s, maxb=3324KB/s, mint=10881292msec, maxt=10881292msec
WRITE: io=33974MB, aggrb=3700KB/s, minb=3700KB/s, maxb=3700KB/s, mint=9401627msec, maxt=9401627msec
Disk stats (read/write):
vda: ios=9044992/8703596, merge=0/2506, ticks=259917574/224617047, in_queue=484520851, util=100.00%
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=24
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=24
fio-2.0.13
Starting 2 processes
readtest: (groupid=0, jobs=1): err= 0: pid=22687: Tue Oct 20 12:28:31 2015
read: io=35332MB, bw=3324.1KB/s, iops=831, runt=10881292msec
slat (usec): min=4, max=178328, avg=45.46, stdev=304.03
clat (usec): min=308, max=652088, avg=28818.31, stdev=20780.24
lat (usec): min=321, max=652111, avg=28865.10, stdev=20787.02
clat percentiles (msec):
| 1.00th=[ 5], 5.00th=[ 8], 10.00th=[ 11], 20.00th=[ 14],
| 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 25], 60.00th=[ 30],
| 70.00th=[ 34], 80.00th=[ 42], 90.00th=[ 53], 95.00th=[ 65],
| 99.00th=[ 93], 99.50th=[ 109], 99.90th=[ 174], 99.95th=[ 223],
| 99.99th=[ 545]
bw (KB/s): min= 406, max=16088, per=99.95%, avg=3322.21, stdev=1580.93
lat (usec): 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2=0.15%, 4=0.77%, 10=8.90%, 20=30.75%, 50=47.80%
lat (msec): 100=10.92%, 250=0.68%, 500=0.02%, 750=0.02%
cpu: usr=1.39%, sys=5.44%, ctx=5757944, majf=0, minf=47
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=9044992/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=22688: Tue Oct 20 12:28:31 2015
write: io=33974MB, bw=3700.4KB/s, iops=925, runt=9401627msec
slat (usec): min=6, max=198169, avg=57.22, stdev=335.09
clat (usec): min=542, max=561342, avg=25877.12, stdev=17322.27
lat (usec): min=625, max=561368, avg=25935.79, stdev=17328.52
clat percentiles (msec):
| 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 14],
| 30.00th=[ 16], 40.00th=[ 18], 50.00th=[ 21], 60.00th=[ 26],
| 70.00th=[ 30], 80.00th=[ 36], 90.00th=[ 46], 95.00th=[ 58],
| 99.00th=[ 85], 99.50th=[ 101], 99.90th=[ 165], 99.95th=[ 204],
| 99.99th=[ 297]
bw (KB/s): min= 344, max= 6432, per=100.00%, avg=3704.64, stdev=828.54
lat (usec): 750=0.01%, 1000=0.01%
lat (msec): 2=0.01%, 4=0.02%, 10=4.49%, 20=45.39%, 50=42.45%
lat (msec): 100=7.14%, 250=0.49%, 500=0.02%, 750=0.01%
cpu: usr=1.79%, sys=7.31%, ctx=6996874, majf=0, minf=24
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=8697222/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=35332MB, aggrb=3324KB/s, minb=3324KB/s, maxb=3324KB/s, mint=10881292msec, maxt=10881292msec
WRITE: io=33974MB, aggrb=3700KB/s, minb=3700KB/s, maxb=3700KB/s, mint=9401627msec, maxt=9401627msec
Disk stats (read/write):
vda: ios=9044992/8703596, merge=0/2506, ticks=259917574/224617047, in_queue=484520851, util=100.00%
VPS/KVM iodepth24 (total-48)
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=24
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=24
fio-2.1.3
Starting 2 processes
?
readtest: (groupid=0, jobs=1): err= 0: pid=2677: Tue Oct 20 11:55:31 2015
read: io=91806MB, bw=41765KB/s, iops=10441, runt=2250896msec
slat (usec): min=0, max=69982, avg=20.72, stdev=134.60
clat (usec): min=0, max=1992.6K, avg=2273.70, stdev=2780.06
lat (usec): min=14, max=1992.6K, avg=2294.96, stdev=2782.77
clat percentiles (usec):
| 1.00th=[ 34], 5.00th=[ 53], 10.00th=[ 76], 20.00th=[ 342],
| 30.00th=[ 756], 40.00th=[ 1112], 50.00th=[ 1608], 60.00th=[ 2288],
| 70.00th=[ 2960], 80.00th=[ 3888], 90.00th=[ 5344], 95.00th=[ 6624],
| 99.00th=[ 9536], 99.50th=[10560], 99.90th=[13376], 99.95th=[14784],
| 99.99th=[20096]
bw (KB /s): min=20632, max=103224, per=100.00%, avg=41784.98, stdev=10948.71
lat (usec): 2=0.03%, 4=0.01%, 10=0.01%, 20=0.04%, 50=4.20%
lat (usec): 100=9.32%, 250=4.78%, 500=4.84%, 750=6.71%, 1000=7.10%
lat (msec): 2=18.80%, 4=25.21%, 10=18.22%, 20=0.73%, 50=0.01%
lat (msec): 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.01%
cpu: usr=5.43%, sys=15.49%, ctx=10567850, majf=0, minf=51
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=23502336/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=2678: Tue Oct 20 11:55:31 2015
write: io=91806MB, bw=38064KB/s, iops=9516, runt=2469757msec
slat (usec): min=0, max=46311, avg=15.14, stdev=96.38
clat (usec): min=78, max=1992.5K, avg=2503.84, stdev=2913.20
lat (usec): min=108, max=1992.5K, avg=2519.49, stdev=2917.23
clat percentiles (usec):
| 1.00th=[ 322], 5.00th=[ 506], 10.00th=[ 556], 20.00th=[ 620],
| 30.00th=[ 700], 40.00th=[ 1012], 50.00th=[ 1720], 60.00th=[ 2480],
| 70.00th=[ 3280], 80.00th=[ 4192], 90.00th=[ 5664], 95.00th=[ 7008],
| 99.00th=[ 9920], 99.50th=[11072], 99.90th=[13888], 99.95th=[15424],
| 99.99th=[21120]
bw (KB /s): min=10762, max=157528, per=100.00%, avg=38083.55, stdev=34148.51
lat (usec): 100=0.01%, 250=0.32%, 500=4.47%, 750=28.33%, 1000=6.67%
lat (msec): 2=13.95%, 4=24.17%, 10=21.14%, 20=0.94%, 50=0.01%
lat (msec): 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.01%
cpu: usr=3.95%, sys=10.93%, ctx=4200133, majf=0, minf=29
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=23502336/d=0, short=r=0/w=0/d=0
?
Run status group 0 (all jobs):
READ: io=91806MB, aggrb=41765KB/s, minb=41765KB/s, maxb=41765KB/s, mint=2250896msec, maxt=2250896msec
WRITE: io=91806MB, aggrb=38064KB/s, minb=38064KB/s, maxb=38064KB/s, mint=2469757msec, maxt=2469757msec
?
Disk stats (read/write):
vda: ios=23502336/23499007, merge=0/1213, ticks=51271576/56546800, in_queue=107817232, util=100.00%
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=24
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=24
fio-2.1.3
Starting 2 processes
?
readtest: (groupid=0, jobs=1): err= 0: pid=2677: Tue Oct 20 11:55:31 2015
read: io=91806MB, bw=41765KB/s, iops=10441, runt=2250896msec
slat (usec): min=0, max=69982, avg=20.72, stdev=134.60
clat (usec): min=0, max=1992.6K, avg=2273.70, stdev=2780.06
lat (usec): min=14, max=1992.6K, avg=2294.96, stdev=2782.77
clat percentiles (usec):
| 1.00th=[ 34], 5.00th=[ 53], 10.00th=[ 76], 20.00th=[ 342],
| 30.00th=[ 756], 40.00th=[ 1112], 50.00th=[ 1608], 60.00th=[ 2288],
| 70.00th=[ 2960], 80.00th=[ 3888], 90.00th=[ 5344], 95.00th=[ 6624],
| 99.00th=[ 9536], 99.50th=[10560], 99.90th=[13376], 99.95th=[14784],
| 99.99th=[20096]
bw (KB /s): min=20632, max=103224, per=100.00%, avg=41784.98, stdev=10948.71
lat (usec): 2=0.03%, 4=0.01%, 10=0.01%, 20=0.04%, 50=4.20%
lat (usec): 100=9.32%, 250=4.78%, 500=4.84%, 750=6.71%, 1000=7.10%
lat (msec): 2=18.80%, 4=25.21%, 10=18.22%, 20=0.73%, 50=0.01%
lat (msec): 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.01%
cpu: usr=5.43%, sys=15.49%, ctx=10567850, majf=0, minf=51
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=23502336/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=2678: Tue Oct 20 11:55:31 2015
write: io=91806MB, bw=38064KB/s, iops=9516, runt=2469757msec
slat (usec): min=0, max=46311, avg=15.14, stdev=96.38
clat (usec): min=78, max=1992.5K, avg=2503.84, stdev=2913.20
lat (usec): min=108, max=1992.5K, avg=2519.49, stdev=2917.23
clat percentiles (usec):
| 1.00th=[ 322], 5.00th=[ 506], 10.00th=[ 556], 20.00th=[ 620],
| 30.00th=[ 700], 40.00th=[ 1012], 50.00th=[ 1720], 60.00th=[ 2480],
| 70.00th=[ 3280], 80.00th=[ 4192], 90.00th=[ 5664], 95.00th=[ 7008],
| 99.00th=[ 9920], 99.50th=[11072], 99.90th=[13888], 99.95th=[15424],
| 99.99th=[21120]
bw (KB /s): min=10762, max=157528, per=100.00%, avg=38083.55, stdev=34148.51
lat (usec): 100=0.01%, 250=0.32%, 500=4.47%, 750=28.33%, 1000=6.67%
lat (msec): 2=13.95%, 4=24.17%, 10=21.14%, 20=0.94%, 50=0.01%
lat (msec): 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.01%
cpu: usr=3.95%, sys=10.93%, ctx=4200133, majf=0, minf=29
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=23502336/d=0, short=r=0/w=0/d=0
?
Run status group 0 (all jobs):
READ: io=91806MB, aggrb=41765KB/s, minb=41765KB/s, maxb=41765KB/s, mint=2250896msec, maxt=2250896msec
WRITE: io=91806MB, aggrb=38064KB/s, minb=38064KB/s, maxb=38064KB/s, mint=2469757msec, maxt=2469757msec
?
Disk stats (read/write):
vda: ios=23502336/23499007, merge=0/1213, ticks=51271576/56546800, in_queue=107817232, util=100.00%
Dedicated server iodepth24 (total-48)
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=24
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=24
fio-2.0.13
Starting 2 processes
?
readtest: (groupid=0, jobs=1): err= 0: pid=18205: Tue Oct 20 09:34:37 2015
read: io=114473MB, bw=64824KB/s, iops=16205, runt=1808297msec
slat (usec): min=1, max=682, avg= 5.77, stdev= 3.45
clat (usec): min=21, max=119492, avg=1473.74, stdev=8376.31
lat (usec): min=56, max=119498, avg=1479.76, stdev=8376.39
clat percentiles (usec):
| 1.00th=[ 167], 5.00th=[ 197], 10.00th=[ 209], 20.00th=[ 225],
| 30.00th=[ 239], 40.00th=[ 255], 50.00th=[ 278], 60.00th=[ 314],
| 70.00th=[ 490], 80.00th=[ 1096], 90.00th=[ 2608], 95.00th=[ 3664],
| 99.00th=[ 6624], 99.50th=[101888], 99.90th=[105984], 99.95th=[107008],
| 99.99th=[109056]
bw (KB/s): min=13714, max=371568, per=100.00%, avg=65209.44, stdev=110524.20
lat (usec): 50=0.01%, 100=0.01%, 250=37.15%, 500=33.07%, 750=5.29%
lat (usec): 1000=3.38%
lat (msec): 2=7.74%, 4=9.31%, 10=3.41%, 20=0.01%, 50=0.01%
lat (msec): 100=0.02%, 250=0.64%
cpu: usr=3.65%, sys=14.32%, ctx=11922468, majf=0, minf=47
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=29305206/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=18206: Tue Oct 20 09:34:37 2015
write: io=118250MB, bw=63533KB/s, iops=16018, runt=1894178msec
slat (usec): min=1, max=699, avg= 5.86, stdev= 4.95
clat (usec): min=25, max=119772, avg=1494.13, stdev=8418.87
lat (usec): min=57, max=119774, avg=1499.34, stdev=8418.94
clat percentiles (usec):
| 1.00th=[ 78], 5.00th=[ 112], 10.00th=[ 143], 20.00th=[ 203],
| 30.00th=[ 258], 40.00th=[ 310], 50.00th=[ 362], 60.00th=[ 418],
| 70.00th=[ 516], 80.00th=[ 756], 90.00th=[ 1848], 95.00th=[ 2896],
| 99.00th=[ 6368], 99.50th=[100864], 99.90th=[103936], 99.95th=[104960],
| 99.99th=[107008]
bw (KB/s): min=13556, max=195287, per=99.26%, avg=69021.39, stdev=37403.15
lat (usec): 50=0.06%, 100=2.86%, 250=25.82%, 500=40.25%, 750=10.83%
lat (usec): 1000=4.45%
lat (msec): 2=6.46%, 4=6.55%, 10=2.01%, 20=0.02%, 50=0.01%
lat (msec): 100=0.03%, 250=0.66%
cpu: usr=2.98%, sys=14.91%, ctx=11187158, majf=0, minf=24
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=27712022/d=0, short=r=0/w=0/d=0
?
Run status group 0 (all jobs):
READ: io=114473MB, aggrb=64823KB/s, minb=64823KB/s, maxb=64823KB/s, mint=1808297msec, maxt=1808297msec
WRITE: io=118250MB, aggrb=64533KB/s, minb=64533KB/s, maxb=64533KB/s, mint=1894178msec, maxt=1894178msec
?
Disk stats (read/write):
sdb: ios=29255679/27492666, merge=28536/219355, ticks=42845849/37461396, in_queue=80289337, util=100.00%
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=24
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=24
fio-2.0.13
Starting 2 processes
?
readtest: (groupid=0, jobs=1): err= 0: pid=18205: Tue Oct 20 09:34:37 2015
read: io=114473MB, bw=64824KB/s, iops=16205, runt=1808297msec
slat (usec): min=1, max=682, avg= 5.77, stdev= 3.45
clat (usec): min=21, max=119492, avg=1473.74, stdev=8376.31
lat (usec): min=56, max=119498, avg=1479.76, stdev=8376.39
clat percentiles (usec):
| 1.00th=[ 167], 5.00th=[ 197], 10.00th=[ 209], 20.00th=[ 225],
| 30.00th=[ 239], 40.00th=[ 255], 50.00th=[ 278], 60.00th=[ 314],
| 70.00th=[ 490], 80.00th=[ 1096], 90.00th=[ 2608], 95.00th=[ 3664],
| 99.00th=[ 6624], 99.50th=[101888], 99.90th=[105984], 99.95th=[107008],
| 99.99th=[109056]
bw (KB/s): min=13714, max=371568, per=100.00%, avg=65209.44, stdev=110524.20
lat (usec): 50=0.01%, 100=0.01%, 250=37.15%, 500=33.07%, 750=5.29%
lat (usec): 1000=3.38%
lat (msec): 2=7.74%, 4=9.31%, 10=3.41%, 20=0.01%, 50=0.01%
lat (msec): 100=0.02%, 250=0.64%
cpu: usr=3.65%, sys=14.32%, ctx=11922468, majf=0, minf=47
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=29305206/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=18206: Tue Oct 20 09:34:37 2015
write: io=118250MB, bw=63533KB/s, iops=16018, runt=1894178msec
slat (usec): min=1, max=699, avg= 5.86, stdev= 4.95
clat (usec): min=25, max=119772, avg=1494.13, stdev=8418.87
lat (usec): min=57, max=119774, avg=1499.34, stdev=8418.94
clat percentiles (usec):
| 1.00th=[ 78], 5.00th=[ 112], 10.00th=[ 143], 20.00th=[ 203],
| 30.00th=[ 258], 40.00th=[ 310], 50.00th=[ 362], 60.00th=[ 418],
| 70.00th=[ 516], 80.00th=[ 756], 90.00th=[ 1848], 95.00th=[ 2896],
| 99.00th=[ 6368], 99.50th=[100864], 99.90th=[103936], 99.95th=[104960],
| 99.99th=[107008]
bw (KB/s): min=13556, max=195287, per=99.26%, avg=69021.39, stdev=37403.15
lat (usec): 50=0.06%, 100=2.86%, 250=25.82%, 500=40.25%, 750=10.83%
lat (usec): 1000=4.45%
lat (msec): 2=6.46%, 4=6.55%, 10=2.01%, 20=0.02%, 50=0.01%
lat (msec): 100=0.03%, 250=0.66%
cpu: usr=2.98%, sys=14.91%, ctx=11187158, majf=0, minf=24
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=27712022/d=0, short=r=0/w=0/d=0
?
Run status group 0 (all jobs):
READ: io=114473MB, aggrb=64823KB/s, minb=64823KB/s, maxb=64823KB/s, mint=1808297msec, maxt=1808297msec
WRITE: io=118250MB, aggrb=64533KB/s, minb=64533KB/s, maxb=64533KB/s, mint=1894178msec, maxt=1894178msec
?
Disk stats (read/write):
sdb: ios=29255679/27492666, merge=28536/219355, ticks=42845849/37461396, in_queue=80289337, util=100.00%
Cloud VPS iodepth16 (total-32)
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
fio-2.0.13
Starting 2 processes
?
readtest: (groupid=0, jobs=1): err= 0: pid=19457: Mon Oct 19 11:08:11 2015
read: io=30959MB, bw=2845.3KB/s, iops=711, runt=11142084msec
slat (usec): min=4, max=215016, avg=48.40, stdev=298.68
clat (usec): min=300, max=531477, avg=22435.29, stdev=15939.78
lat (usec): min=319, max=531488, avg=22485.08, stdev=15947.24
clat percentiles (msec):
| 1.00th=[ 5], 5.00th=[ 7], 10.00th=[ 9], 20.00th=[ 11],
| 30.00th=[ 14], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 22],
| 70.00th=[ 26], 80.00th=[ 31], 90.00th=[ 40], 95.00th=[ 49],
| 99.00th=[ 78], 99.50th=[ 99], 99.90th=[ 165], 99.95th=[ 198],
| 99.99th=[ 265]
bw (KB/s): min= 361, max=10496, per=100.00%, avg=2849.40, stdev=1185.33
lat (usec): 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2=0.02%, 4=0.68%, 10=15.26%, 20=36.14%, 50=43.44%
lat (msec): 100=3.99%, 250=0.47%, 500=0.01%, 750=0.01%
cpu: usr=1.30%, sys=5.07%, ctx=5475213, majf=0, minf=40
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=7925553/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=19458: Mon Oct 19 11:08:11 2015
write: io=34220MB, bw=3375.9KB/s, iops=843, runt=10380006msec
slat (usec): min=6, max=244780, avg=59.65, stdev=318.61
clat (usec): min=440, max=511525, avg=18887.14, stdev=14467.62
lat (usec): min=841, max=511544, avg=18948.18, stdev=14474.90
clat percentiles (msec):
| 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 10],
| 30.00th=[ 12], 40.00th=[ 13], 50.00th=[ 15], 60.00th=[ 19],
| 70.00th=[ 21], 80.00th=[ 26], 90.00th=[ 34], 95.00th=[ 42],
| 99.00th=[ 72], 99.50th=[ 95], 99.90th=[ 163], 99.95th=[ 194],
| 99.99th=[ 260]
bw (KB/s): min= 4, max= 6456, per=100.00%, avg=3380.68, stdev=887.56
lat (usec): 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2=0.01%, 4=0.04%, 10=20.74%, 20=45.87%, 50=30.55%
lat (msec): 100=2.35%, 250=0.43%, 500=0.01%, 750=0.01%
cpu: usr=1.72%, sys=7.04%, ctx=7143450, majf=0, minf=24
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=8760316/d=0, short=r=0/w=0/d=0
?
Run status group 0 (all jobs):
READ: io=30959MB, aggrb=2845KB/s, minb=2845KB/s, maxb=2845KB/s, mint=11142084msec, maxt=11142084msec
WRITE: io=34220MB, aggrb=3375KB/s, minb=3375KB/s, maxb=3375KB/s, mint=10380006msec, maxt=10380006msec
?
Disk stats (read/write):
vda: ios=7925553/8767379, merge=0/2844, ticks=177249078/165106615, in_queue=342341875, util=100.00%
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
fio-2.0.13
Starting 2 processes
?
readtest: (groupid=0, jobs=1): err= 0: pid=19457: Mon Oct 19 11:08:11 2015
read: io=30959MB, bw=2845.3KB/s, iops=711, runt=11142084msec
slat (usec): min=4, max=215016, avg=48.40, stdev=298.68
clat (usec): min=300, max=531477, avg=22435.29, stdev=15939.78
lat (usec): min=319, max=531488, avg=22485.08, stdev=15947.24
clat percentiles (msec):
| 1.00th=[ 5], 5.00th=[ 7], 10.00th=[ 9], 20.00th=[ 11],
| 30.00th=[ 14], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 22],
| 70.00th=[ 26], 80.00th=[ 31], 90.00th=[ 40], 95.00th=[ 49],
| 99.00th=[ 78], 99.50th=[ 99], 99.90th=[ 165], 99.95th=[ 198],
| 99.99th=[ 265]
bw (KB/s): min= 361, max=10496, per=100.00%, avg=2849.40, stdev=1185.33
lat (usec): 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2=0.02%, 4=0.68%, 10=15.26%, 20=36.14%, 50=43.44%
lat (msec): 100=3.99%, 250=0.47%, 500=0.01%, 750=0.01%
cpu: usr=1.30%, sys=5.07%, ctx=5475213, majf=0, minf=40
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=7925553/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=19458: Mon Oct 19 11:08:11 2015
write: io=34220MB, bw=3375.9KB/s, iops=843, runt=10380006msec
slat (usec): min=6, max=244780, avg=59.65, stdev=318.61
clat (usec): min=440, max=511525, avg=18887.14, stdev=14467.62
lat (usec): min=841, max=511544, avg=18948.18, stdev=14474.90
clat percentiles (msec):
| 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 10],
| 30.00th=[ 12], 40.00th=[ 13], 50.00th=[ 15], 60.00th=[ 19],
| 70.00th=[ 21], 80.00th=[ 26], 90.00th=[ 34], 95.00th=[ 42],
| 99.00th=[ 72], 99.50th=[ 95], 99.90th=[ 163], 99.95th=[ 194],
| 99.99th=[ 260]
bw (KB/s): min= 4, max= 6456, per=100.00%, avg=3380.68, stdev=887.56
lat (usec): 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2=0.01%, 4=0.04%, 10=20.74%, 20=45.87%, 50=30.55%
lat (msec): 100=2.35%, 250=0.43%, 500=0.01%, 750=0.01%
cpu: usr=1.72%, sys=7.04%, ctx=7143450, majf=0, minf=24
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=8760316/d=0, short=r=0/w=0/d=0
?
Run status group 0 (all jobs):
READ: io=30959MB, aggrb=2845KB/s, minb=2845KB/s, maxb=2845KB/s, mint=11142084msec, maxt=11142084msec
WRITE: io=34220MB, aggrb=3375KB/s, minb=3375KB/s, maxb=3375KB/s, mint=10380006msec, maxt=10380006msec
?
Disk stats (read/write):
vda: ios=7925553/8767379, merge=0/2844, ticks=177249078/165106615, in_queue=342341875, util=100.00%
VPS/KVM iodepth16 (total-32)
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
fio-2.1.3
Starting 2 processes
?
readtest: (groupid=0, jobs=1): err= 0: pid=1197: Mon Oct 19 11:26:56 2015
read: io=91538MB, bw=38069KB/s, iops=9517, runt=2462276msec
slat (usec): min=0, max=39930, avg=14.91, stdev=32.26
clat (usec): min=0, max=2920.5K, avg=1661.54, stdev=2331.81
lat (usec): min=15, max=2920.5K, avg=1677.02, stdev=2332.12
clat percentiles (usec):
| 1.00th=[ 34], 5.00th=[ 52], 10.00th=[ 70], 20.00th=[ 191],
| 30.00th=[ 494], 40.00th=[ 772], 50.00th=[ 1112], 60.00th=[ 1560],
| 70.00th=[ 2128], 80.00th=[ 2832], 90.00th=[ 4048], 95.00th=[ 5152],
| 99.00th=[ 7392], 99.50th=[ 8256], 99.90th=[10560], 99.95th=[11712],
| 99.99th=[16192]
bw (KB /s): min=15912, max=94592, per=100.00%, avg=38084.72, stdev=10569.87
lat (usec): 2=0.03%, 4=0.01%, 10=0.01%, 20=0.05%, 50=4.31%
lat (usec): 100=11.47%, 250=5.36%, 500=8.99%, 750=9.08%, 1000=7.61%
lat (msec): 2=20.97%, 4=21.81%, 10=10.16%, 20=0.14%, 50=0.01%
lat (msec): 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.01%, >=2000=0.01%
cpu: usr=5.15%, sys=17.98%, ctx=11516094, majf=0, minf=44
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=23433841/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=1198: Mon Oct 19 11:26:56 2015
write: io=91538MB, bw=34902KB/s, iops=8725, runt=2685646msec
slat (usec): min=0, max=35603, avg=13.85, stdev=28.29
clat (usec): min=15, max=2118.1K, avg=1816.17, stdev=2194.88
lat (usec): min=107, max=2118.1K, avg=1830.57, stdev=2196.09
clat percentiles (usec):
| 1.00th=[ 233], 5.00th=[ 346], 10.00th=[ 390], 20.00th=[ 426],
| 30.00th=[ 470], 40.00th=[ 588], 50.00th=[ 988], 60.00th=[ 1576],
| 70.00th=[ 2288], 80.00th=[ 3120], 90.00th=[ 4448], 95.00th=[ 5600],
| 99.00th=[ 8096], 99.50th=[ 9152], 99.90th=[11584], 99.95th=[12736],
| 99.99th=[17792]
bw (KB /s): min=10040, max=147392, per=100.00%, avg=34916.51, stdev=31820.67
lat (usec): 20=0.01%, 50=0.01%, 100=0.01%, 250=1.41%, 500=32.44%
lat (usec): 750=11.04%, 1000=5.29%
lat (msec): 2=16.07%, 4=20.97%, 10=12.50%, 20=0.28%, 50=0.01%
lat (msec): 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.01%, >=2000=0.01%
cpu: usr=3.92%, sys=14.03%, ctx=4492852, majf=0, minf=29
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
?
Run status group 0 (all jobs):
READ: io=91538MB, aggrb=38068KB/s, minb=38068KB/s, maxb=38068KB/s, mint=2462276msec, maxt=2462276msec
WRITE: io=91538MB, aggrb=34902KB/s, minb=34902KB/s, maxb=34902KB/s, mint=2685646msec, maxt=2685646msec
?
Disk stats (read/write):
vda: ios=23433856/23430586, merge=0/551, ticks=37944004/40834784, in_queue=78772804, util=99.92%
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
fio-2.1.3
Starting 2 processes
?
readtest: (groupid=0, jobs=1): err= 0: pid=1197: Mon Oct 19 11:26:56 2015
read: io=91538MB, bw=38069KB/s, iops=9517, runt=2462276msec
slat (usec): min=0, max=39930, avg=14.91, stdev=32.26
clat (usec): min=0, max=2920.5K, avg=1661.54, stdev=2331.81
lat (usec): min=15, max=2920.5K, avg=1677.02, stdev=2332.12
clat percentiles (usec):
| 1.00th=[ 34], 5.00th=[ 52], 10.00th=[ 70], 20.00th=[ 191],
| 30.00th=[ 494], 40.00th=[ 772], 50.00th=[ 1112], 60.00th=[ 1560],
| 70.00th=[ 2128], 80.00th=[ 2832], 90.00th=[ 4048], 95.00th=[ 5152],
| 99.00th=[ 7392], 99.50th=[ 8256], 99.90th=[10560], 99.95th=[11712],
| 99.99th=[16192]
bw (KB /s): min=15912, max=94592, per=100.00%, avg=38084.72, stdev=10569.87
lat (usec): 2=0.03%, 4=0.01%, 10=0.01%, 20=0.05%, 50=4.31%
lat (usec): 100=11.47%, 250=5.36%, 500=8.99%, 750=9.08%, 1000=7.61%
lat (msec): 2=20.97%, 4=21.81%, 10=10.16%, 20=0.14%, 50=0.01%
lat (msec): 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.01%, >=2000=0.01%
cpu: usr=5.15%, sys=17.98%, ctx=11516094, majf=0, minf=44
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=23433841/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=1198: Mon Oct 19 11:26:56 2015
write: io=91538MB, bw=34902KB/s, iops=8725, runt=2685646msec
slat (usec): min=0, max=35603, avg=13.85, stdev=28.29
clat (usec): min=15, max=2118.1K, avg=1816.17, stdev=2194.88
lat (usec): min=107, max=2118.1K, avg=1830.57, stdev=2196.09
clat percentiles (usec):
| 1.00th=[ 233], 5.00th=[ 346], 10.00th=[ 390], 20.00th=[ 426],
| 30.00th=[ 470], 40.00th=[ 588], 50.00th=[ 988], 60.00th=[ 1576],
| 70.00th=[ 2288], 80.00th=[ 3120], 90.00th=[ 4448], 95.00th=[ 5600],
| 99.00th=[ 8096], 99.50th=[ 9152], 99.90th=[11584], 99.95th=[12736],
| 99.99th=[17792]
bw (KB /s): min=10040, max=147392, per=100.00%, avg=34916.51, stdev=31820.67
lat (usec): 20=0.01%, 50=0.01%, 100=0.01%, 250=1.41%, 500=32.44%
lat (usec): 750=11.04%, 1000=5.29%
lat (msec): 2=16.07%, 4=20.97%, 10=12.50%, 20=0.28%, 50=0.01%
lat (msec): 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.01%, >=2000=0.01%
cpu: usr=3.92%, sys=14.03%, ctx=4492852, majf=0, minf=29
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
?
Run status group 0 (all jobs):
READ: io=91538MB, aggrb=38068KB/s, minb=38068KB/s, maxb=38068KB/s, mint=2462276msec, maxt=2462276msec
WRITE: io=91538MB, aggrb=34902KB/s, minb=34902KB/s, maxb=34902KB/s, mint=2685646msec, maxt=2685646msec
?
Disk stats (read/write):
vda: ios=23433856/23430586, merge=0/551, ticks=37944004/40834784, in_queue=78772804, util=99.92%
Dedicated server iodepth16 (total-32)
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
fio-2.0.13
Starting 2 processes
?
readtest: (groupid=0, jobs=1): err= 0: pid=48824: Mon Oct 19 10:21:30 2015
read: io=114473MB, bw=58649KB/s, iops=14662, runt=1998682msec
slat (usec): min=1, max=2339, avg= 5.22, stdev= 4.82
clat (usec): min=34, max=110175, avg=1084.00, stdev=2834.01
lat (usec): min=36, max=110181, avg=1089.48, stdev=2834.25
clat percentiles (usec):
| 1.00th=[ 145], 5.00th=[ 187], 10.00th=[ 221], 20.00th=[ 270],
| 30.00th=[ 322], 40.00th=[ 382], 50.00th=[ 486], 60.00th=[ 644],
| 70.00th=[ 876], 80.00th=[ 1448], 90.00th=[ 2416], 95.00th=[ 3152],
| 99.00th=[ 7520], 99.50th=[12608], 99.90th=[32640], 99.95th=[44288],
| 99.99th=[102912]
bw (KB/s): min= 9217, max=186336, per=100.00%, avg=58845.27, stdev=42973.67
lat (usec): 50=0.01%, 100=0.01%, 250=15.86%, 500=35.34%, 750=14.15%
lat (usec): 1000=8.01%
lat (msec): 2=11.71%, 4=12.10%, 10=2.14%, 20=0.45%, 50=0.20%
lat (msec): 100=0.01%, 250=0.03%
cpu: usr=5.93%, sys=14.82%, ctx=14883559, majf=0, minf=39
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=29305206/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=48825: Mon Oct 19 10:21:30 2015
write: io=108250MB, bw=54151KB/s, iops=13537, runt=2047006msec
slat (usec): min=1, max=584, avg= 5.26, stdev= 4.92
clat (usec): min=33, max=108679, avg=1175.07, stdev=8859.49
lat (usec): min=50, max=108682, avg=1180.65, stdev=8859.59
clat percentiles (usec):
| 1.00th=[ 89], 5.00th=[ 104], 10.00th=[ 119], 20.00th=[ 145],
| 30.00th=[ 171], 40.00th=[ 191], 50.00th=[ 215], 60.00th=[ 241],
| 70.00th=[ 282], 80.00th=[ 354], 90.00th=[ 516], 95.00th=[ 820],
| 99.00th=[19584], 99.50th=[99840], 99.90th=[99840], 99.95th=[100864],
| 99.99th=[101888]
bw (KB/s): min= 9348, max=237608, per=100.00%, avg=55214.32, stdev=39468.27
lat (usec): 50=0.02%, 100=3.37%, 250=59.29%, 500=26.77%, 750=5.07%
lat (usec): 1000=1.18%
lat (msec): 2=1.74%, 4=0.99%, 10=0.40%, 20=0.16%, 50=0.22%
lat (msec): 100=0.59%, 250=0.19%
cpu: usr=2.15%, sys=11.56%, ctx=12028211, majf=0, minf=24
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=27712022/d=0, short=r=0/w=0/d=0
?
Run status group 0 (all jobs):
READ: io=114473MB, aggrb=58649KB/s, minb=58649KB/s, maxb=58649KB/s, mint=1998682msec, maxt=1998682msec
WRITE: io=108250MB, aggrb=54151KB/s, minb=54151KB/s, maxb=54151KB/s, mint=2047006msec, maxt=2047006msec
?
Disk stats (read/write):
sdb: ios=29022088/27640063, merge=283184/58260, ticks=30502266/32183243, in_queue=62664453, util=100.00%
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
fio-2.0.13
Starting 2 processes
?
readtest: (groupid=0, jobs=1): err= 0: pid=48824: Mon Oct 19 10:21:30 2015
read: io=114473MB, bw=58649KB/s, iops=14662, runt=1998682msec
slat (usec): min=1, max=2339, avg= 5.22, stdev= 4.82
clat (usec): min=34, max=110175, avg=1084.00, stdev=2834.01
lat (usec): min=36, max=110181, avg=1089.48, stdev=2834.25
clat percentiles (usec):
| 1.00th=[ 145], 5.00th=[ 187], 10.00th=[ 221], 20.00th=[ 270],
| 30.00th=[ 322], 40.00th=[ 382], 50.00th=[ 486], 60.00th=[ 644],
| 70.00th=[ 876], 80.00th=[ 1448], 90.00th=[ 2416], 95.00th=[ 3152],
| 99.00th=[ 7520], 99.50th=[12608], 99.90th=[32640], 99.95th=[44288],
| 99.99th=[102912]
bw (KB/s): min= 9217, max=186336, per=100.00%, avg=58845.27, stdev=42973.67
lat (usec): 50=0.01%, 100=0.01%, 250=15.86%, 500=35.34%, 750=14.15%
lat (usec): 1000=8.01%
lat (msec): 2=11.71%, 4=12.10%, 10=2.14%, 20=0.45%, 50=0.20%
lat (msec): 100=0.01%, 250=0.03%
cpu: usr=5.93%, sys=14.82%, ctx=14883559, majf=0, minf=39
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=29305206/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=48825: Mon Oct 19 10:21:30 2015
write: io=108250MB, bw=54151KB/s, iops=13537, runt=2047006msec
slat (usec): min=1, max=584, avg= 5.26, stdev= 4.92
clat (usec): min=33, max=108679, avg=1175.07, stdev=8859.49
lat (usec): min=50, max=108682, avg=1180.65, stdev=8859.59
clat percentiles (usec):
| 1.00th=[ 89], 5.00th=[ 104], 10.00th=[ 119], 20.00th=[ 145],
| 30.00th=[ 171], 40.00th=[ 191], 50.00th=[ 215], 60.00th=[ 241],
| 70.00th=[ 282], 80.00th=[ 354], 90.00th=[ 516], 95.00th=[ 820],
| 99.00th=[19584], 99.50th=[99840], 99.90th=[99840], 99.95th=[100864],
| 99.99th=[101888]
bw (KB/s): min= 9348, max=237608, per=100.00%, avg=55214.32, stdev=39468.27
lat (usec): 50=0.02%, 100=3.37%, 250=59.29%, 500=26.77%, 750=5.07%
lat (usec): 1000=1.18%
lat (msec): 2=1.74%, 4=0.99%, 10=0.40%, 20=0.16%, 50=0.22%
lat (msec): 100=0.59%, 250=0.19%
cpu: usr=2.15%, sys=11.56%, ctx=12028211, majf=0, minf=24
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=27712022/d=0, short=r=0/w=0/d=0
?
Run status group 0 (all jobs):
READ: io=114473MB, aggrb=58649KB/s, minb=58649KB/s, maxb=58649KB/s, mint=1998682msec, maxt=1998682msec
WRITE: io=108250MB, aggrb=54151KB/s, minb=54151KB/s, maxb=54151KB/s, mint=2047006msec, maxt=2047006msec
?
Disk stats (read/write):
sdb: ios=29022088/27640063, merge=283184/58260, ticks=30502266/32183243, in_queue=62664453, util=100.00%
Cloud VPS iodepth8 (total-16)
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=8
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=8
fio-2.0.13
Starting 2 processes
?
readtest: (groupid=0, jobs=1): err= 0: pid=30210: Thu Oct 22 09:21:47 2015
read: io=35332MB, bw=3080.8KB/s, iops=770, runt=11746463msec
slat (usec): min=4, max=77802, avg=45.05, stdev=183.93
clat (usec): min=113, max=570144, avg=10335.64, stdev=8453.21
lat (usec): min=246, max=570164, avg=10381.99, stdev=8460.58
clat percentiles (usec):
| 1.00th=[ 1464], 5.00th=[ 2608], 10.00th=[ 3440], 20.00th=[ 4704],
| 30.00th=[ 5856], 40.00th=[ 7072], 50.00th=[ 8640], 60.00th=[10176],
| 70.00th=[11968], 80.00th=[14400], 90.00th=[18560], 95.00th=[23168],
| 99.00th=[39168], 99.50th=[50944], 99.90th=[90624], 99.95th=[113152],
| 99.99th=[171008]
bw (KB/s): min= 340, max=26064, per=100.00%, avg=3084.85, stdev=1689.55
lat (usec): 250=0.01%, 500=0.03%, 750=0.06%, 1000=0.19%
lat (msec): 2=2.14%, 4=11.83%, 10=44.25%, 20=33.66%, 50=7.32%
lat (msec): 100=0.45%, 250=0.07%, 500=0.01%, 750=0.01%
cpu: usr=1.35%, sys=5.31%, ctx=5868148, majf=0, minf=31
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=9044992/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=30211: Thu Oct 22 09:21:47 2015
write: io=33974MB, bw=3362.4KB/s, iops=840, runt=10346655msec
slat (usec): min=6, max=179705, avg=58.15, stdev=246.59
clat (usec): min=345, max=569382, avg=9449.59, stdev=7800.55
lat (usec): min=569, max=569395, avg=9509.13, stdev=7807.58
clat percentiles (msec):
| 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 6],
| 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 8], 60.00th=[ 9],
| 70.00th=[ 11], 80.00th=[ 13], 90.00th=[ 16], 95.00th=[ 21],
| 99.00th=[ 39], 99.50th=[ 53], 99.90th=[ 92], 99.95th=[ 113],
| 99.99th=[ 165]
bw (KB/s): min= 288, max= 6400, per=100.00%, avg=3367.49, stdev=957.57
lat (usec): 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2=0.09%, 4=7.34%, 10=61.59%, 20=25.64%, 50=4.76%
lat (msec): 100=0.49%, 250=0.07%, 500=0.01%, 750=0.01%
cpu: usr=1.69%, sys=7.06%, ctx=7042644, majf=0, minf=24
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=8697222/d=0, short=r=0/w=0/d=0
?
Run status group 0 (all jobs):
READ: io=35332MB, aggrb=3080KB/s, minb=3080KB/s, maxb=3080KB/s, mint=11746463msec, maxt=11746463msec
WRITE: io=33974MB, aggrb=3362KB/s, minb=3362KB/s, maxb=3362KB/s, mint=10346655msec, maxt=10346655msec
?
Disk stats (read/write):
vda: ios=9044992/8753174, merge=0/40616, ticks=92946153/82117863, in_queue=175050555, util=100.00%
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=8
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=8
fio-2.0.13
Starting 2 processes
?
readtest: (groupid=0, jobs=1): err= 0: pid=30210: Thu Oct 22 09:21:47 2015
read: io=35332MB, bw=3080.8KB/s, iops=770, runt=11746463msec
slat (usec): min=4, max=77802, avg=45.05, stdev=183.93
clat (usec): min=113, max=570144, avg=10335.64, stdev=8453.21
lat (usec): min=246, max=570164, avg=10381.99, stdev=8460.58
clat percentiles (usec):
| 1.00th=[ 1464], 5.00th=[ 2608], 10.00th=[ 3440], 20.00th=[ 4704],
| 30.00th=[ 5856], 40.00th=[ 7072], 50.00th=[ 8640], 60.00th=[10176],
| 70.00th=[11968], 80.00th=[14400], 90.00th=[18560], 95.00th=[23168],
| 99.00th=[39168], 99.50th=[50944], 99.90th=[90624], 99.95th=[113152],
| 99.99th=[171008]
bw (KB/s): min= 340, max=26064, per=100.00%, avg=3084.85, stdev=1689.55
lat (usec): 250=0.01%, 500=0.03%, 750=0.06%, 1000=0.19%
lat (msec): 2=2.14%, 4=11.83%, 10=44.25%, 20=33.66%, 50=7.32%
lat (msec): 100=0.45%, 250=0.07%, 500=0.01%, 750=0.01%
cpu: usr=1.35%, sys=5.31%, ctx=5868148, majf=0, minf=31
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=9044992/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=30211: Thu Oct 22 09:21:47 2015
write: io=33974MB, bw=3362.4KB/s, iops=840, runt=10346655msec
slat (usec): min=6, max=179705, avg=58.15, stdev=246.59
clat (usec): min=345, max=569382, avg=9449.59, stdev=7800.55
lat (usec): min=569, max=569395, avg=9509.13, stdev=7807.58
clat percentiles (msec):
| 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 6],
| 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 8], 60.00th=[ 9],
| 70.00th=[ 11], 80.00th=[ 13], 90.00th=[ 16], 95.00th=[ 21],
| 99.00th=[ 39], 99.50th=[ 53], 99.90th=[ 92], 99.95th=[ 113],
| 99.99th=[ 165]
bw (KB/s): min= 288, max= 6400, per=100.00%, avg=3367.49, stdev=957.57
lat (usec): 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2=0.09%, 4=7.34%, 10=61.59%, 20=25.64%, 50=4.76%
lat (msec): 100=0.49%, 250=0.07%, 500=0.01%, 750=0.01%
cpu: usr=1.69%, sys=7.06%, ctx=7042644, majf=0, minf=24
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=8697222/d=0, short=r=0/w=0/d=0
?
Run status group 0 (all jobs):
READ: io=35332MB, aggrb=3080KB/s, minb=3080KB/s, maxb=3080KB/s, mint=11746463msec, maxt=11746463msec
WRITE: io=33974MB, aggrb=3362KB/s, minb=3362KB/s, maxb=3362KB/s, mint=10346655msec, maxt=10346655msec
?
Disk stats (read/write):
vda: ios=9044992/8753174, merge=0/40616, ticks=92946153/82117863, in_queue=175050555, util=100.00%
VPS/KVM iodepth8 (total-16)
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=8
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=8
fio-2.1.3
Starting 2 processes
?
readtest: (groupid=0, jobs=1): err= 0: pid=1463: Mon Oct 19 16:38:39 2015
read: io=91538MB, bw=32778KB/s, iops=8194, runt=2859724msec
slat (usec): min=0, max=31874, avg=17.76, stdev=35.02
clat (usec): min=0, max=57086, avg=953.41, stdev=1014.69
lat (usec): min=16, max=57110, avg=971.83, stdev=1015.66
clat percentiles (usec):
| 1.00th=[ 36], 5.00th=[ 57], 10.00th=[ 78], 20.00th=[ 187],
| 30.00th=[ 338], 40.00th=[ 482], 50.00th=[ 652], 60.00th=[ 844],
| 70.00th=[ 1096], 80.00th=[ 1464], 90.00th=[ 2288], 95.00th=[ 3024],
| 99.00th=[ 4640], 99.50th=[ 5280], 99.90th=[ 7200], 99.95th=[ 8256],
| 99.99th=[12224]
bw (KB /s): min=12207, max=318816, per=100.00%, avg=32791.35, stdev=13943.15
lat (usec): 2=0.04%, 4=0.01%, 10=0.01%, 20=0.06%, 50=3.16%
lat (usec): 100=11.26%, 250=8.21%, 500=18.32%, 750=14.31%, 1000=11.13%
lat (msec): 2=21.03%, 4=10.54%, 10=1.93%, 20=0.02%, 50=0.01%
lat (msec): 100=0.01%
cpu: usr=5.06%, sys=18.52%, ctx=12619958, majf=0, minf=35
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=23433828/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=1464: Mon Oct 19 16:38:39 2015
write: io=91538MB, bw=30169KB/s, iops=7542, runt=3106998msec
slat (usec): min=0, max=34539, avg=17.23, stdev=32.63
clat (usec): min=1, max=56882, avg=1039.39, stdev=1108.74
lat (usec): min=94, max=56908, avg=1057.23, stdev=1110.97
clat percentiles (usec):
| 1.00th=[ 173], 5.00th=[ 225], 10.00th=[ 243], 20.00th=[ 266],
| 30.00th=[ 298], 40.00th=[ 398], 50.00th=[ 612], 60.00th=[ 860],
| 70.00th=[ 1176], 80.00th=[ 1624], 90.00th=[ 2512], 95.00th=[ 3344],
| 99.00th=[ 5088], 99.50th=[ 5792], 99.90th=[ 7776], 99.95th=[ 8896],
| 99.99th=[12864]
bw (KB /s): min= 8822, max=119480, per=100.00%, avg=30182.57, stdev=25851.83
lat (usec): 2=0.01%, 50=0.01%, 100=0.01%, 250=12.59%, 500=32.33%
lat (usec): 750=10.93%, 1000=8.99%
lat (msec): 2=20.44%, 4=11.89%, 10=2.80%, 20=0.03%, 50=0.01%
lat (msec): 100=0.01%
cpu: usr=4.09%, sys=15.70%, ctx=6891783, majf=0, minf=28
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=23433828/d=0, short=r=0/w=0/d=0
?
Run status group 0 (all jobs):
READ: io=91538MB, aggrb=32777KB/s, minb=32777KB/s, maxb=32777KB/s, mint=2859724msec, maxt=2859724msec
WRITE: io=91538MB, aggrb=30169KB/s, minb=30169KB/s, maxb=30169KB/s, mint=3106998msec, maxt=3106998msec
?
Disk stats (read/write):
vda: ios=23433856/23428672, merge=0/642, ticks=21337472/23232656, in_queue=44555512, util=99.86%
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=8
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=8
fio-2.1.3
Starting 2 processes
?
readtest: (groupid=0, jobs=1): err= 0: pid=1463: Mon Oct 19 16:38:39 2015
read: io=91538MB, bw=32778KB/s, iops=8194, runt=2859724msec
slat (usec): min=0, max=31874, avg=17.76, stdev=35.02
clat (usec): min=0, max=57086, avg=953.41, stdev=1014.69
lat (usec): min=16, max=57110, avg=971.83, stdev=1015.66
clat percentiles (usec):
| 1.00th=[ 36], 5.00th=[ 57], 10.00th=[ 78], 20.00th=[ 187],
| 30.00th=[ 338], 40.00th=[ 482], 50.00th=[ 652], 60.00th=[ 844],
| 70.00th=[ 1096], 80.00th=[ 1464], 90.00th=[ 2288], 95.00th=[ 3024],
| 99.00th=[ 4640], 99.50th=[ 5280], 99.90th=[ 7200], 99.95th=[ 8256],
| 99.99th=[12224]
bw (KB /s): min=12207, max=318816, per=100.00%, avg=32791.35, stdev=13943.15
lat (usec): 2=0.04%, 4=0.01%, 10=0.01%, 20=0.06%, 50=3.16%
lat (usec): 100=11.26%, 250=8.21%, 500=18.32%, 750=14.31%, 1000=11.13%
lat (msec): 2=21.03%, 4=10.54%, 10=1.93%, 20=0.02%, 50=0.01%
lat (msec): 100=0.01%
cpu: usr=5.06%, sys=18.52%, ctx=12619958, majf=0, minf=35
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=23433828/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=1464: Mon Oct 19 16:38:39 2015
write: io=91538MB, bw=30169KB/s, iops=7542, runt=3106998msec
slat (usec): min=0, max=34539, avg=17.23, stdev=32.63
clat (usec): min=1, max=56882, avg=1039.39, stdev=1108.74
lat (usec): min=94, max=56908, avg=1057.23, stdev=1110.97
clat percentiles (usec):
| 1.00th=[ 173], 5.00th=[ 225], 10.00th=[ 243], 20.00th=[ 266],
| 30.00th=[ 298], 40.00th=[ 398], 50.00th=[ 612], 60.00th=[ 860],
| 70.00th=[ 1176], 80.00th=[ 1624], 90.00th=[ 2512], 95.00th=[ 3344],
| 99.00th=[ 5088], 99.50th=[ 5792], 99.90th=[ 7776], 99.95th=[ 8896],
| 99.99th=[12864]
bw (KB /s): min= 8822, max=119480, per=100.00%, avg=30182.57, stdev=25851.83
lat (usec): 2=0.01%, 50=0.01%, 100=0.01%, 250=12.59%, 500=32.33%
lat (usec): 750=10.93%, 1000=8.99%
lat (msec): 2=20.44%, 4=11.89%, 10=2.80%, 20=0.03%, 50=0.01%
lat (msec): 100=0.01%
cpu: usr=4.09%, sys=15.70%, ctx=6891783, majf=0, minf=28
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=23433828/d=0, short=r=0/w=0/d=0
?
Run status group 0 (all jobs):
READ: io=91538MB, aggrb=32777KB/s, minb=32777KB/s, maxb=32777KB/s, mint=2859724msec, maxt=2859724msec
WRITE: io=91538MB, aggrb=30169KB/s, minb=30169KB/s, maxb=30169KB/s, mint=3106998msec, maxt=3106998msec
?
Disk stats (read/write):
vda: ios=23433856/23428672, merge=0/642, ticks=21337472/23232656, in_queue=44555512, util=99.86%
Dedicated server iodepth8 (total-16)
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=8
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=8
fio-2.0.13
Starting 2 processes
?
readtest: (groupid=0, jobs=1): err= 0: pid=55495: Mon Oct 19 15:02:54 2015
read: io=114473MB, bw=126371KB/s, iops=31592, runt=927591msec
slat (usec): min=1, max=694, avg= 7.42, stdev= 6.16
clat (usec): min=13, max=13576, avg=244.08, stdev=417.08
lat (usec): min=25, max=13579, avg=251.72, stdev=417.34
clat percentiles (usec):
| 1.00th=[ 102], 5.00th=[ 112], 10.00th=[ 118], 20.00th=[ 126],
| 30.00th=[ 135], 40.00th=[ 141], 50.00th=[ 149], 60.00th=[ 159],
| 70.00th=[ 169], 80.00th=[ 187], 90.00th=[ 237], 95.00th=[ 612],
| 99.00th=[ 2544], 99.50th=[ 2768], 99.90th=[ 3600], 99.95th=[ 4448],
| 99.99th=[ 5216]
bw (KB/s): min=23920, max=213552, per=100.00%, avg=126413.13, stdev=83921.67
lat (usec): 20=0.01%, 50=0.01%, 100=0.45%, 250=90.53%, 500=3.64%
lat (usec): 750=0.75%, 1000=0.64%
lat (msec): 2=1.64%, 4=2.27%, 10=0.07%, 20=0.01%
cpu: usr=6.83%, sys=31.95%, ctx=11047569, majf=0, minf=31
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=29305206/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=55496: Mon Oct 19 15:02:54 2015
write: io=108250MB, bw=269423KB/s, iops=30452, runt=813427msec
slat (usec): min=1, max=623, avg= 6.66, stdev= 4.22
clat (usec): min=8, max=15976, avg=211.82, stdev=421.99
lat (usec): min=40, max=14050, avg=267.71, stdev=421.10
clat percentiles (usec):
| 1.00th=[ 54], 5.00th=[ 74], 10.00th=[ 82], 20.00th=[ 87],
| 30.00th=[ 90], 40.00th=[ 93], 50.00th=[ 96], 60.00th=[ 100],
| 70.00th=[ 107], 80.00th=[ 119], 90.00th=[ 141], 95.00th=[ 175],
| 99.00th=[ 286], 99.50th=[ 358], 99.90th=[ 2256], 99.95th=[ 3760],
| 99.99th=[ 4256]
bw (KB/s): min=219096, max=274496, per=100.00%, avg=269439.69, stdev=4021.57
lat (usec): 10=0.01%, 20=0.01%, 50=0.34%, 100=59.50%, 250=38.53%
lat (usec): 500=1.41%, 750=0.07%, 1000=0.02%
lat (msec): 2=0.02%, 4=0.07%, 10=0.03%, 100=0.01%
cpu: usr=9.09%, sys=59.34%, ctx=13000854, majf=0, minf=24
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=27712022/d=0, short=r=0/w=0/d=0
?
Run status group 0 (all jobs):
READ: io=114473MB, aggrb=126371KB/s, minb=126371KB/s, maxb=126371KB/s, mint=927591msec, maxt=927591msec
WRITE: io=108250MB, aggrb=269423KB/s, minb=269423KB/s, maxb=269423KB/s, mint=411427msec, maxt=411427msec
?
Disk stats (read/write):
sdb: ios=29295474/27711898, merge=112/124, ticks=6834383/3005350, in_queue=9819443, util=100.00%
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=8
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=8
fio-2.0.13
Starting 2 processes
?
readtest: (groupid=0, jobs=1): err= 0: pid=55495: Mon Oct 19 15:02:54 2015
read: io=114473MB, bw=126371KB/s, iops=31592, runt=927591msec
slat (usec): min=1, max=694, avg= 7.42, stdev= 6.16
clat (usec): min=13, max=13576, avg=244.08, stdev=417.08
lat (usec): min=25, max=13579, avg=251.72, stdev=417.34
clat percentiles (usec):
| 1.00th=[ 102], 5.00th=[ 112], 10.00th=[ 118], 20.00th=[ 126],
| 30.00th=[ 135], 40.00th=[ 141], 50.00th=[ 149], 60.00th=[ 159],
| 70.00th=[ 169], 80.00th=[ 187], 90.00th=[ 237], 95.00th=[ 612],
| 99.00th=[ 2544], 99.50th=[ 2768], 99.90th=[ 3600], 99.95th=[ 4448],
| 99.99th=[ 5216]
bw (KB/s): min=23920, max=213552, per=100.00%, avg=126413.13, stdev=83921.67
lat (usec): 20=0.01%, 50=0.01%, 100=0.45%, 250=90.53%, 500=3.64%
lat (usec): 750=0.75%, 1000=0.64%
lat (msec): 2=1.64%, 4=2.27%, 10=0.07%, 20=0.01%
cpu: usr=6.83%, sys=31.95%, ctx=11047569, majf=0, minf=31
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=29305206/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=55496: Mon Oct 19 15:02:54 2015
write: io=108250MB, bw=269423KB/s, iops=30452, runt=813427msec
slat (usec): min=1, max=623, avg= 6.66, stdev= 4.22
clat (usec): min=8, max=15976, avg=211.82, stdev=421.99
lat (usec): min=40, max=14050, avg=267.71, stdev=421.10
clat percentiles (usec):
| 1.00th=[ 54], 5.00th=[ 74], 10.00th=[ 82], 20.00th=[ 87],
| 30.00th=[ 90], 40.00th=[ 93], 50.00th=[ 96], 60.00th=[ 100],
| 70.00th=[ 107], 80.00th=[ 119], 90.00th=[ 141], 95.00th=[ 175],
| 99.00th=[ 286], 99.50th=[ 358], 99.90th=[ 2256], 99.95th=[ 3760],
| 99.99th=[ 4256]
bw (KB/s): min=219096, max=274496, per=100.00%, avg=269439.69, stdev=4021.57
lat (usec): 10=0.01%, 20=0.01%, 50=0.34%, 100=59.50%, 250=38.53%
lat (usec): 500=1.41%, 750=0.07%, 1000=0.02%
lat (msec): 2=0.02%, 4=0.07%, 10=0.03%, 100=0.01%
cpu: usr=9.09%, sys=59.34%, ctx=13000854, majf=0, minf=24
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=27712022/d=0, short=r=0/w=0/d=0
?
Run status group 0 (all jobs):
READ: io=114473MB, aggrb=126371KB/s, minb=126371KB/s, maxb=126371KB/s, mint=927591msec, maxt=927591msec
WRITE: io=108250MB, aggrb=269423KB/s, minb=269423KB/s, maxb=269423KB/s, mint=411427msec, maxt=411427msec
?
Disk stats (read/write):
sdb: ios=29295474/27711898, merge=112/124, ticks=6834383/3005350, in_queue=9819443, util=100.00%
Cloud VPS iodepth1 (total-2)
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
fio-2.0.13
Starting 2 processes
?
readtest: (groupid=0, jobs=1): err= 0: pid=27403: Thu Oct 22 01:38:19 2015
read: io=35332MB, bw=1769.9KB/s, iops=442, runt=20442421msec
slat (usec): min=7, max=1346.7K, avg=43.59, stdev=465.29
clat (usec): min=2, max=2129.1K, avg=2207.14, stdev=3529.84
lat (usec): min=234, max=2129.1K, avg=2251.91, stdev=3563.17
clat percentiles (usec):
| 1.00th=[ 414], 5.00th=[ 684], 10.00th=[ 940], 20.00th=[ 1224],
| 30.00th=[ 1464], 40.00th=[ 1640], 50.00th=[ 1800], 60.00th=[ 1992],
| 70.00th=[ 2224], 80.00th=[ 2608], 90.00th=[ 3536], 95.00th=[ 4896],
| 99.00th=[ 9920], 99.50th=[14528], 99.90th=[22144], 99.95th=[27264],
| 99.99th=[42240]
bw (KB/s): min= 2, max= 3904, per=100.00%, avg=1775.66, stdev=494.15
lat (usec): 4=0.01%, 10=0.01%, 20=0.01%, 100=0.01%, 250=0.01%
lat (usec): 500=1.94%, 750=3.94%, 1000=5.98%
lat (msec): 2=48.66%, 4=31.67%, 10=6.80%, 20=0.85%, 50=0.14%
lat (msec): 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.01%, >=2000=0.01%
cpu: usr=0.89%, sys=3.24%, ctx=9086183, majf=0, minf=24
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=9044992/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=27404: Thu Oct 22 01:38:19 2015
write: io=33974MB, bw=1948.3KB/s, iops=487, runt=17856150msec
slat (usec): min=10, max=1346.7K, avg=54.45, stdev=480.12
clat (usec): min=2, max=2536.6K, avg=1989.12, stdev=7303.31
lat (usec): min=300, max=2536.6K, avg=2044.72, stdev=7320.93
clat percentiles (usec):
| 1.00th=[ 438], 5.00th=[ 660], 10.00th=[ 844], 20.00th=[ 1128],
| 30.00th=[ 1336], 40.00th=[ 1512], 50.00th=[ 1672], 60.00th=[ 1832],
| 70.00th=[ 2064], 80.00th=[ 2352], 90.00th=[ 2928], 95.00th=[ 3952],
| 99.00th=[ 7392], 99.50th=[10816], 99.90th=[20352], 99.95th=[26752],
| 99.99th=[68096]
bw (KB/s): min= 3, max= 3944, per=100.00%, avg=1975.93, stdev=535.23
lat (usec): 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01%
lat (usec): 250=0.01%, 500=1.83%, 750=5.53%, 1000=7.75%
lat (msec): 2=52.43%, 4=27.61%, 10=4.30%, 20=0.44%, 50=0.09%
lat (msec): 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.01%, >=2000=0.01%
cpu: usr=0.97%, sys=4.05%, ctx=8746041, majf=0, minf=25
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=8697222/d=0, short=r=0/w=0/d=0
?
Run status group 0 (all jobs):
READ: io=35332MB, aggrb=1769KB/s, minb=1769KB/s, maxb=1769KB/s, mint=20442421msec, maxt=20442421msec
WRITE: io=33974MB, aggrb=1948KB/s, minb=1948KB/s, maxb=1948KB/s, mint=17856150msec, maxt=17856150msec
?
Disk stats (read/write):
vda: ios=9044993/8781992, merge=0/67048, ticks=19920252/17540300, in_queue=37448210, util=99.46%
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
fio-2.0.13
Starting 2 processes
?
readtest: (groupid=0, jobs=1): err= 0: pid=27403: Thu Oct 22 01:38:19 2015
read: io=35332MB, bw=1769.9KB/s, iops=442, runt=20442421msec
slat (usec): min=7, max=1346.7K, avg=43.59, stdev=465.29
clat (usec): min=2, max=2129.1K, avg=2207.14, stdev=3529.84
lat (usec): min=234, max=2129.1K, avg=2251.91, stdev=3563.17
clat percentiles (usec):
| 1.00th=[ 414], 5.00th=[ 684], 10.00th=[ 940], 20.00th=[ 1224],
| 30.00th=[ 1464], 40.00th=[ 1640], 50.00th=[ 1800], 60.00th=[ 1992],
| 70.00th=[ 2224], 80.00th=[ 2608], 90.00th=[ 3536], 95.00th=[ 4896],
| 99.00th=[ 9920], 99.50th=[14528], 99.90th=[22144], 99.95th=[27264],
| 99.99th=[42240]
bw (KB/s): min= 2, max= 3904, per=100.00%, avg=1775.66, stdev=494.15
lat (usec): 4=0.01%, 10=0.01%, 20=0.01%, 100=0.01%, 250=0.01%
lat (usec): 500=1.94%, 750=3.94%, 1000=5.98%
lat (msec): 2=48.66%, 4=31.67%, 10=6.80%, 20=0.85%, 50=0.14%
lat (msec): 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.01%, >=2000=0.01%
cpu: usr=0.89%, sys=3.24%, ctx=9086183, majf=0, minf=24
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=9044992/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=27404: Thu Oct 22 01:38:19 2015
write: io=33974MB, bw=1948.3KB/s, iops=487, runt=17856150msec
slat (usec): min=10, max=1346.7K, avg=54.45, stdev=480.12
clat (usec): min=2, max=2536.6K, avg=1989.12, stdev=7303.31
lat (usec): min=300, max=2536.6K, avg=2044.72, stdev=7320.93
clat percentiles (usec):
| 1.00th=[ 438], 5.00th=[ 660], 10.00th=[ 844], 20.00th=[ 1128],
| 30.00th=[ 1336], 40.00th=[ 1512], 50.00th=[ 1672], 60.00th=[ 1832],
| 70.00th=[ 2064], 80.00th=[ 2352], 90.00th=[ 2928], 95.00th=[ 3952],
| 99.00th=[ 7392], 99.50th=[10816], 99.90th=[20352], 99.95th=[26752],
| 99.99th=[68096]
bw (KB/s): min= 3, max= 3944, per=100.00%, avg=1975.93, stdev=535.23
lat (usec): 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01%
lat (usec): 250=0.01%, 500=1.83%, 750=5.53%, 1000=7.75%
lat (msec): 2=52.43%, 4=27.61%, 10=4.30%, 20=0.44%, 50=0.09%
lat (msec): 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.01%, >=2000=0.01%
cpu: usr=0.97%, sys=4.05%, ctx=8746041, majf=0, minf=25
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=8697222/d=0, short=r=0/w=0/d=0
?
Run status group 0 (all jobs):
READ: io=35332MB, aggrb=1769KB/s, minb=1769KB/s, maxb=1769KB/s, mint=20442421msec, maxt=20442421msec
WRITE: io=33974MB, aggrb=1948KB/s, minb=1948KB/s, maxb=1948KB/s, mint=17856150msec, maxt=17856150msec
?
Disk stats (read/write):
vda: ios=9044993/8781992, merge=0/67048, ticks=19920252/17540300, in_queue=37448210, util=99.46%
VPS/KVM iodepth1 (total-2)
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
fio-2.1.3
Starting 2 processes
?
readtest: (groupid=0, jobs=1): err= 0: pid=10202: Wed Oct 21 18:27:57 2015
read: io=91804MB, bw=12597KB/s, iops=3149, runt=7462654msec
slat (usec): min=0, max=18026, avg=16.01, stdev=25.97
clat (usec): min=0, max=39633, avg=295.76, stdev=288.49
lat (usec): min=22, max=39649, avg=312.49, stdev=290.19
clat percentiles (usec):
| 1.00th=[ 55], 5.00th=[ 72], 10.00th=[ 85], 20.00th=[ 191],
| 30.00th=[ 217], 40.00th=[ 231], 50.00th=[ 243], 60.00th=[ 262],
| 70.00th=[ 290], 80.00th=[ 342], 90.00th=[ 462], 95.00th=[ 604],
| 99.00th=[ 1960], 99.50th=[ 2256], 99.90th=[ 2672], 99.95th=[ 2960],
| 99.99th=[ 4448]
bw (KB /s): min= 4248, max=19624, per=100.00%, avg=12602.52, stdev=2781.20
lat (usec): 2=0.03%, 4=0.03%, 10=0.01%, 20=0.01%, 50=0.39%
lat (usec): 100=13.84%, 250=39.40%, 500=38.49%, 750=4.52%, 1000=1.23%
lat (msec): 2=1.10%, 4=0.94%, 10=0.02%, 20=0.01%, 50=0.01%
cpu: usr=2.97%, sys=8.98%, ctx=23926406, majf=0, minf=28
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=23501824/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=10203: Wed Oct 21 18:27:57 2015
write: io=91804MB, bw=11924KB/s, iops=2981, runt=7883671msec
slat (usec): min=0, max=19988, avg=17.69, stdev=15.33
clat (usec): min=0, max=39690, avg=312.09, stdev=293.47
lat (usec): min=91, max=39709, avg=330.41, stdev=293.94
clat percentiles (usec):
| 1.00th=[ 143], 5.00th=[ 159], 10.00th=[ 173], 20.00th=[ 193],
| 30.00th=[ 205], 40.00th=[ 217], 50.00th=[ 237], 60.00th=[ 262],
| 70.00th=[ 290], 80.00th=[ 338], 90.00th=[ 478], 95.00th=[ 652],
| 99.00th=[ 1976], 99.50th=[ 2288], 99.90th=[ 2800], 99.95th=[ 3248],
| 99.99th=[ 4576]
bw (KB /s): min= 4008, max=23640, per=100.00%, avg=11929.79, stdev=2715.99
lat (usec): 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
lat (usec): 100=0.01%, 250=55.81%, 500=35.40%, 750=4.93%, 1000=1.41%
lat (msec): 2=1.46%, 4=0.95%, 10=0.02%, 20=0.01%, 50=0.01%
cpu: usr=2.98%, sys=9.02%, ctx=23580827, majf=0, minf=28
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=23501824/d=0, short=r=0/w=0/d=0
?
Run status group 0 (all jobs):
READ: io=91804MB, aggrb=12597KB/s, minb=12597KB/s, maxb=12597KB/s, mint=7462654msec, maxt=7462654msec
WRITE: io=91804MB, aggrb=11924KB/s, minb=11924KB/s, maxb=11924KB/s, mint=7883671msec, maxt=7883671msec
?
Disk stats (read/write):
vda: ios=23501824/23504931, merge=0/3622, ticks=6398492/7003300, in_queue=13376172, util=94.33%
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
fio-2.1.3
Starting 2 processes
?
readtest: (groupid=0, jobs=1): err= 0: pid=10202: Wed Oct 21 18:27:57 2015
read: io=91804MB, bw=12597KB/s, iops=3149, runt=7462654msec
slat (usec): min=0, max=18026, avg=16.01, stdev=25.97
clat (usec): min=0, max=39633, avg=295.76, stdev=288.49
lat (usec): min=22, max=39649, avg=312.49, stdev=290.19
clat percentiles (usec):
| 1.00th=[ 55], 5.00th=[ 72], 10.00th=[ 85], 20.00th=[ 191],
| 30.00th=[ 217], 40.00th=[ 231], 50.00th=[ 243], 60.00th=[ 262],
| 70.00th=[ 290], 80.00th=[ 342], 90.00th=[ 462], 95.00th=[ 604],
| 99.00th=[ 1960], 99.50th=[ 2256], 99.90th=[ 2672], 99.95th=[ 2960],
| 99.99th=[ 4448]
bw (KB /s): min= 4248, max=19624, per=100.00%, avg=12602.52, stdev=2781.20
lat (usec): 2=0.03%, 4=0.03%, 10=0.01%, 20=0.01%, 50=0.39%
lat (usec): 100=13.84%, 250=39.40%, 500=38.49%, 750=4.52%, 1000=1.23%
lat (msec): 2=1.10%, 4=0.94%, 10=0.02%, 20=0.01%, 50=0.01%
cpu: usr=2.97%, sys=8.98%, ctx=23926406, majf=0, minf=28
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=23501824/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=10203: Wed Oct 21 18:27:57 2015
write: io=91804MB, bw=11924KB/s, iops=2981, runt=7883671msec
slat (usec): min=0, max=19988, avg=17.69, stdev=15.33
clat (usec): min=0, max=39690, avg=312.09, stdev=293.47
lat (usec): min=91, max=39709, avg=330.41, stdev=293.94
clat percentiles (usec):
| 1.00th=[ 143], 5.00th=[ 159], 10.00th=[ 173], 20.00th=[ 193],
| 30.00th=[ 205], 40.00th=[ 217], 50.00th=[ 237], 60.00th=[ 262],
| 70.00th=[ 290], 80.00th=[ 338], 90.00th=[ 478], 95.00th=[ 652],
| 99.00th=[ 1976], 99.50th=[ 2288], 99.90th=[ 2800], 99.95th=[ 3248],
| 99.99th=[ 4576]
bw (KB /s): min= 4008, max=23640, per=100.00%, avg=11929.79, stdev=2715.99
lat (usec): 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
lat (usec): 100=0.01%, 250=55.81%, 500=35.40%, 750=4.93%, 1000=1.41%
lat (msec): 2=1.46%, 4=0.95%, 10=0.02%, 20=0.01%, 50=0.01%
cpu: usr=2.98%, sys=9.02%, ctx=23580827, majf=0, minf=28
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=23501824/d=0, short=r=0/w=0/d=0
?
Run status group 0 (all jobs):
READ: io=91804MB, aggrb=12597KB/s, minb=12597KB/s, maxb=12597KB/s, mint=7462654msec, maxt=7462654msec
WRITE: io=91804MB, aggrb=11924KB/s, minb=11924KB/s, maxb=11924KB/s, mint=7883671msec, maxt=7883671msec
?
Disk stats (read/write):
vda: ios=23501824/23504931, merge=0/3622, ticks=6398492/7003300, in_queue=13376172, util=94.33%
Dedicated server iodepth1 (total-2)
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
fio-2.0.13
Starting 2 processes
readtest: (groupid=0, jobs=1): err= 0: pid=6347: Wed Oct 21 16:50:54 2015
read: io=114473MB, bw=22923KB/s, iops=5730, runt=5113676msec
slat (usec): min=3, max=982, avg= 9.59, stdev= 2.67
clat (usec): min=0, max=17045, avg=162.57, stdev=270.11
lat (usec): min=29, max=17062, avg=172.44, stdev=270.59
clat percentiles (usec):
| 1.00th=[ 104], 5.00th=[ 106], 10.00th=[ 107], 20.00th=[ 109],
| 30.00th=[ 110], 40.00th=[ 111], 50.00th=[ 121], 60.00th=[ 123],
| 70.00th=[ 125], 80.00th=[ 126], 90.00th=[ 127], 95.00th=[ 229],
| 99.00th=[ 1816], 99.50th=[ 2128], 99.90th=[ 3216], 99.95th=[ 3856],
| 99.99th=[ 4896]
bw (KB/s): min= 3105, max=31880, per=100.00%, avg=22934.65, stdev=11661.67
lat (usec): 2=0.01%, 10=0.01%, 20=0.01%, 50=0.04%, 100=0.17%
lat (usec): 250=95.01%, 500=1.66%, 750=0.89%, 1000=0.39%
lat (msec): 2=1.10%, 4=0.69%, 10=0.04%, 20=0.01%
cpu: usr=2.03%, sys=6.65%, ctx=29562276, majf=0, minf=27
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=29305206/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=6348: Wed Oct 21 16:50:54 2015
write: io=108250MB, bw=21653KB/s, iops=5360, runt=1796575msec
slat (usec): min=3, max=979, avg= 9.66, stdev= 4.02
clat (usec): min=0, max=16879, avg=52.80, stdev=136.77
lat (usec): min=26, max=16883, avg=62.74, stdev=136.77
clat percentiles (usec):
| 1.00th=[ 20], 5.00th=[ 27], 10.00th=[ 29], 20.00th=[ 35],
| 30.00th=[ 38], 40.00th=[ 39], 50.00th=[ 40], 60.00th=[ 40],
| 70.00th=[ 40], 80.00th=[ 41], 90.00th=[ 52], 95.00th=[ 107],
| 99.00th=[ 145], 99.50th=[ 804], 99.90th=[ 2128], 99.95th=[ 2672],
| 99.99th=[ 4320]
bw (KB/s): min=3003, max=30740, per=100.00%, avg=22435.35, stdev=11315.77
lat (usec): 2=0.01%, 4=0.01%, 10=0.01%, 20=0.99%, 50=88.01%
lat (usec): 100=4.56%, 250=5.58%, 500=0.18%, 750=0.15%, 1000=0.11%
lat (msec): 2=0.29%, 4=0.11%, 10=0.01%, 20=0.01%
cpu: usr=9.20%, sys=19.06%, ctx=28484939, majf=0, minf=23
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=27712022/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=114473MB, aggrb=22923KB/s, minb=22923KB/s, maxb=22923KB/s, mint=5113676msec, maxt=5113676msec
WRITE: io=108250MB, aggrb=21923KB/s, minb=21923KB/s, maxb=21923KB/s, mint=5111376msec, maxt=5111376msec
Disk stats (read/write):
sdb: ios=29304641/27712022, merge=0/0, ticks=4606063/1297249, in_queue=5873504, util=90.24%
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
fio-2.0.13
Starting 2 processes
readtest: (groupid=0, jobs=1): err= 0: pid=6347: Wed Oct 21 16:50:54 2015
read: io=114473MB, bw=22923KB/s, iops=5730, runt=5113676msec
slat (usec): min=3, max=982, avg= 9.59, stdev= 2.67
clat (usec): min=0, max=17045, avg=162.57, stdev=270.11
lat (usec): min=29, max=17062, avg=172.44, stdev=270.59
clat percentiles (usec):
| 1.00th=[ 104], 5.00th=[ 106], 10.00th=[ 107], 20.00th=[ 109],
| 30.00th=[ 110], 40.00th=[ 111], 50.00th=[ 121], 60.00th=[ 123],
| 70.00th=[ 125], 80.00th=[ 126], 90.00th=[ 127], 95.00th=[ 229],
| 99.00th=[ 1816], 99.50th=[ 2128], 99.90th=[ 3216], 99.95th=[ 3856],
| 99.99th=[ 4896]
bw (KB/s): min= 3105, max=31880, per=100.00%, avg=22934.65, stdev=11661.67
lat (usec): 2=0.01%, 10=0.01%, 20=0.01%, 50=0.04%, 100=0.17%
lat (usec): 250=95.01%, 500=1.66%, 750=0.89%, 1000=0.39%
lat (msec): 2=1.10%, 4=0.69%, 10=0.04%, 20=0.01%
cpu: usr=2.03%, sys=6.65%, ctx=29562276, majf=0, minf=27
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=29305206/w=0/d=0, short=r=0/w=0/d=0
writetest: (groupid=0, jobs=1): err= 0: pid=6348: Wed Oct 21 16:50:54 2015
write: io=108250MB, bw=21653KB/s, iops=5360, runt=1796575msec
slat (usec): min=3, max=979, avg= 9.66, stdev= 4.02
clat (usec): min=0, max=16879, avg=52.80, stdev=136.77
lat (usec): min=26, max=16883, avg=62.74, stdev=136.77
clat percentiles (usec):
| 1.00th=[ 20], 5.00th=[ 27], 10.00th=[ 29], 20.00th=[ 35],
| 30.00th=[ 38], 40.00th=[ 39], 50.00th=[ 40], 60.00th=[ 40],
| 70.00th=[ 40], 80.00th=[ 41], 90.00th=[ 52], 95.00th=[ 107],
| 99.00th=[ 145], 99.50th=[ 804], 99.90th=[ 2128], 99.95th=[ 2672],
| 99.99th=[ 4320]
bw (KB/s): min=3003, max=30740, per=100.00%, avg=22435.35, stdev=11315.77
lat (usec): 2=0.01%, 4=0.01%, 10=0.01%, 20=0.99%, 50=88.01%
lat (usec): 100=4.56%, 250=5.58%, 500=0.18%, 750=0.15%, 1000=0.11%
lat (msec): 2=0.29%, 4=0.11%, 10=0.01%, 20=0.01%
cpu: usr=9.20%, sys=19.06%, ctx=28484939, majf=0, minf=23
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=27712022/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=114473MB, aggrb=22923KB/s, minb=22923KB/s, maxb=22923KB/s, mint=5113676msec, maxt=5113676msec
WRITE: io=108250MB, aggrb=21923KB/s, minb=21923KB/s, maxb=21923KB/s, mint=5111376msec, maxt=5111376msec
Disk stats (read/write):
sdb: ios=29304641/27712022, merge=0/0, ticks=4606063/1297249, in_queue=5873504, util=90.24%