Initialize the system environment and modify the kernel parameters failed

[xujingbin]

Question

I have a problem when i execute 'ansible-playbook bootstrap.yml'

four virtual machines:CentOS Linux release 7.4.1708 (Core)

TASK [machine_benchmark : include_tasks] *************************************************
included: /home/tidb/tidb-ansible/roles/machine_benchmark/tasks/fio_randread.yml for 192.168.62.140, 192.168.62.196, 192.168.62.202

TASK [machine_benchmark : fio randread benchmark on tikv_data_dir disk] ******************
fatal: [192.168.62.196]: FAILED! => {"changed": true, "cmd": "cd /data1/deploy/data/fio && ./fio -ioengine=psync -bs=32k -fdatasync=1 -thread -rw=randread -size=10G -filename=fio_randread_test.txt -name='fio randread test' -iodepth=4 -runtime=60 -numjobs=4 -group_reporting --output-format=json --output=fio_randread_result.json", "delta": "0:00:00.075924", "end": "2018-10-09 09:54:19.343905", "msg": "non-zero return code", "rc": 132, "start": "2018-10-09 09:54:19.267981", "stderr": "/bin/sh: line 1: 29283 Illegal instruction     ./fio -ioengine=psync -bs=32k -fdatasync=1 -thread -rw=randread -size=10G -filename=fio_randread_test.txt -name='fio randread test' -iodepth=4 -runtime=60 -numjobs=4 -group_reporting --output-format=json --output=fio_randread_result.json", "stderr_lines": ["/bin/sh: line 1: 29283 Illegal instruction     ./fio -ioengine=psync -bs=32k -fdatasync=1 -thread -rw=randread -size=10G -filename=fio_randread_test.txt -name='fio randread test' -iodepth=4 -runtime=60 -numjobs=4 -group_reporting --output-format=json --output=fio_randread_result.json"], "stdout": "", "stdout_lines": []}
fatal: [192.168.62.202]: FAILED! => {"changed": true, "cmd": "cd /data1/deploy/data/fio && ./fio -ioengine=psync -bs=32k -fdatasync=1 -thread -rw=randread -size=10G -filename=fio_randread_test.txt -name='fio randread test' -iodepth=4 -runtime=60 -numjobs=4 -group_reporting --output-format=json --output=fio_randread_result.json", "delta": "0:00:00.083828", "end": "2018-10-09 09:54:19.415909", "msg": "non-zero return code", "rc": 132, "start": "2018-10-09 09:54:19.332081", "stderr": "/bin/sh: line 1: 29424 Illegal instruction     ./fio -ioengine=psync -bs=32k -fdatasync=1 -thread -rw=randread -size=10G -filename=fio_randread_test.txt -name='fio randread test' -iodepth=4 -runtime=60 -numjobs=4 -group_reporting --output-format=json --output=fio_randread_result.json", "stderr_lines": ["/bin/sh: line 1: 29424 Illegal instruction     ./fio -ioengine=psync -bs=32k -fdatasync=1 -thread -rw=randread -size=10G -filename=fio_randread_test.txt -name='fio randread test' -iodepth=4 -runtime=60 -numjobs=4 -group_reporting --output-format=json --output=fio_randread_result.json"], "stdout": "", "stdout_lines": []}
fatal: [192.168.62.140]: FAILED! => {"changed": true, "cmd": "cd /data1/deploy/data/fio && ./fio -ioengine=psync -bs=32k -fdatasync=1 -thread -rw=randread -size=10G -filename=fio_randread_test.txt -name='fio randread test' -iodepth=4 -runtime=60 -numjobs=4 -group_reporting --output-format=json --output=fio_randread_result.json", "delta": "0:00:00.092554", "end": "2018-10-09 09:54:19.426816", "msg": "non-zero return code", "rc": 132, "start": "2018-10-09 09:54:19.334262", "stderr": "/bin/sh: line 1: 22245 Illegal instruction     ./fio -ioengine=psync -bs=32k -fdatasync=1 -thread -rw=randread -size=10G -filename=fio_randread_test.txt -name='fio randread test' -iodepth=4 -runtime=60 -numjobs=4 -group_reporting --output-format=json --output=fio_randread_result.json", "stderr_lines": ["/bin/sh: line 1: 22245 Illegal instruction     ./fio -ioengine=psync -bs=32k -fdatasync=1 -thread -rw=randread -size=10G -filename=fio_randread_test.txt -name='fio randread test' -iodepth=4 -runtime=60 -numjobs=4 -group_reporting --output-format=json --output=fio_randread_result.json"], "stdout": "", "stdout_lines": []}
	to retry, use: --limit @/home/tidb/tidb-ansible/retry_files/bootstrap.retry

PLAY RECAP *******************************************************************************
192.168.62.140             : ok=41   changed=5    unreachable=0    failed=1
192.168.62.196             : ok=37   changed=5    unreachable=0    failed=1
192.168.62.202             : ok=37   changed=5    unreachable=0    failed=1
localhost                  : ok=1    changed=0    unreachable=0    failed=0


ERROR MESSAGE SUMMARY ********************************************************************
[192.168.62.196]: Ansible FAILED! => playbook: bootstrap.yml; TASK: machine_benchmark : fio randread benchmark on tikv_data_dir disk; message: {"changed": true, "cmd": "cd /data1/deploy/data/fio && ./fio -ioengine=psync -bs=32k -fdatasync=1 -thread -rw=randread -size=10G -filename=fio_randread_test.txt -name='fio randread test' -iodepth=4 -runtime=60 -numjobs=4 -group_reporting --output-format=json --output=fio_randread_result.json", "delta": "0:00:00.075924", "end": "2018-10-09 09:54:19.343905", "msg": "non-zero return code", "rc": 132, "start": "2018-10-09 09:54:19.267981", "stderr": "/bin/sh: line 1: 29283 Illegal instruction     ./fio -ioengine=psync -bs=32k -fdatasync=1 -thread -rw=randread -size=10G -filename=fio_randread_test.txt -name='fio randread test' -iodepth=4 -runtime=60 -numjobs=4 -group_reporting --output-format=json --output=fio_randread_result.json", "stderr_lines": ["/bin/sh: line 1: 29283 Illegal instruction     ./fio -ioengine=psync -bs=32k -fdatasync=1 -thread -rw=randread -size=10G -filename=fio_randread_test.txt -name='fio randread test' -iodepth=4 -runtime=60 -numjobs=4 -group_reporting --output-format=json --output=fio_randread_result.json"], "stdout": "", "stdout_lines": []}

[192.168.62.202]: Ansible FAILED! => playbook: bootstrap.yml; TASK: machine_benchmark : fio randread benchmark on tikv_data_dir disk; message: {"changed": true, "cmd": "cd /data1/deploy/data/fio && ./fio -ioengine=psync -bs=32k -fdatasync=1 -thread -rw=randread -size=10G -filename=fio_randread_test.txt -name='fio randread test' -iodepth=4 -runtime=60 -numjobs=4 -group_reporting --output-format=json --output=fio_randread_result.json", "delta": "0:00:00.083828", "end": "2018-10-09 09:54:19.415909", "msg": "non-zero return code", "rc": 132, "start": "2018-10-09 09:54:19.332081", "stderr": "/bin/sh: line 1: 29424 Illegal instruction     ./fio -ioengine=psync -bs=32k -fdatasync=1 -thread -rw=randread -size=10G -filename=fio_randread_test.txt -name='fio randread test' -iodepth=4 -runtime=60 -numjobs=4 -group_reporting --output-format=json --output=fio_randread_result.json", "stderr_lines": ["/bin/sh: line 1: 29424 Illegal instruction     ./fio -ioengine=psync -bs=32k -fdatasync=1 -thread -rw=randread -size=10G -filename=fio_randread_test.txt -name='fio randread test' -iodepth=4 -runtime=60 -numjobs=4 -group_reporting --output-format=json --output=fio_randread_result.json"], "stdout": "", "stdout_lines": []}

[192.168.62.140]: Ansible FAILED! => playbook: bootstrap.yml; TASK: machine_benchmark : fio randread benchmark on tikv_data_dir disk; message: {"changed": true, "cmd": "cd /data1/deploy/data/fio && ./fio -ioengine=psync -bs=32k -fdatasync=1 -thread -rw=randread -size=10G -filename=fio_randread_test.txt -name='fio randread test' -iodepth=4 -runtime=60 -numjobs=4 -group_reporting --output-format=json --output=fio_randread_result.json", "delta": "0:00:00.092554", "end": "2018-10-09 09:54:19.426816", "msg": "non-zero return code", "rc": 132, "start": "2018-10-09 09:54:19.334262", "stderr": "/bin/sh: line 1: 22245 Illegal instruction     ./fio -ioengine=psync -bs=32k -fdatasync=1 -thread -rw=randread -size=10G -filename=fio_randread_test.txt -name='fio randread test' -iodepth=4 -runtime=60 -numjobs=4 -group_reporting --output-format=json --output=fio_randread_result.json", "stderr_lines": ["/bin/sh: line 1: 22245 Illegal instruction     ./fio -ioengine=psync -bs=32k -fdatasync=1 -thread -rw=randread -size=10G -filename=fio_randread_test.txt -name='fio randread test' -iodepth=4 -runtime=60 -numjobs=4 -group_reporting --output-format=json --output=fio_randread_result.json"], "stdout": "", "stdout_lines": []}

[lysu]

@dinggel Thanks for your feedback, @LinuxGit PTAL~

[LinuxGit]

Could you provide the output of uname -a command?

[xujingbin]

Linux nn1 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

[LinuxGit]

Thanks. What's your virtual machines type, KVM, XEN or Cloud? It's seems that our static build fio binary can not run on your system.
Here's our Software and Hardware Requirements:
https://github.com/pingcap/docs/blob/master/op-guide/recommendation.md
We use fio to benchmark disk performance, you need to use SSD disks for the data directory of TiKV and PD nodes. Otherwise, it cannot pass the check.

[xujingbin]

openstack

[xujingbin]

cpu 8core,
memory 20G,
3 TiDBs: /data1 15G

[xujingbin]

Can't I use a mechanical hard drive？

[LinuxGit]

When you deploy TiDB using Ansible, you need to use SSD disks for the data directory of TiKV and PD nodes. Otherwise, it cannot pass the check. If you only want to try TiDB out and explore the features, it is recommended to deploy TiDB using Docker Compose on a single machine.
see https://github.com/pingcap/docs/blob/master/op-guide/docker-compose.md.

[LinuxGit]

Hi, @dinggel
fio Illegal instruction issue has been resloved, you could update tidb-ansible.
pingcap/tidb-ansible@99cee8b