【 TiDB 使用环境】生产环境
【 TiDB 版本】 V2.1.10
【复现路径】由于TIKV磁盘满了，导致tidb
【遇到的问题：问题现象及影响】 tidb-server 无法启动
【资源配置】日志信息
启动报错信息：
ERROR MESSAGE SUMMARY ****************************************************************************************************************************************************************************************
[TiKV2-2]: Ansible FAILED! => playbook: start.yml; TASK: wait until the TiKV port is up; message: {“changed”: false, “elapsed”: 300, “msg”: “the TiKV port 20172 is not up”}
[TiKV1-2]: Ansible FAILED! => playbook: start.yml; TASK: wait until the TiKV port is up; message: {“changed”: false, “elapsed”: 300, “msg”: “the TiKV port 20172 is not up”}
[10.3.0.147]: Ansible FAILED! => playbook: start.yml; TASK: wait until the TiDB port is up; message: {“changed”: false, “elapsed”: 300, “msg”: “the TiDB port 4000 is not up”}
[10.3.0.149]: Ansible FAILED! => playbook: start.yml; TASK: wait until the TiDB port is up; message: {“changed”: false, “elapsed”: 300, “msg”: “the TiDB port 4000 is not up”}
[10.3.0.148]: Ansible FAILED! => playbook: start.yml; TASK: wait until the TiDB port is up; message: {“changed”: false, “elapsed”: 300, “msg”: “the TiDB port 4000 is not up”}
启动不了的tikv信息
---------tikv_stderr.out----------
thread ‘main’ panicked at ‘invalid auto generated configuration file “/lsb/data2/deploy/data/last_tikv.toml”, err expected an equals, found eof at line 290’, src/config.rs:1122:17
stack backtrace:
0: std::sys::unix::backtrace::tracing:
:unwind_backtrace
at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
1: std::panicking::default_hook::{{closure}}
at libstd/sys_common/backtrace.rs:71
2: std::panicking::default_hook
at libstd/panicking.rs:227
3: std::panicking::rust_panic_with_hook
at libstd/panicking.rs:475
4: std::panicking::continue_panic_fmt
at libstd/panicking.rs:390
5: std::panicking::begin_panic_fmt
at libstd/panicking.rs:345
6: tikv_server::main
at src/config.rs:1122
7: std::rt::lang_start::{{closure}}
at /checkout/src/libstd/rt.rs:74
8: main
9: __libc_start_main
10:
tidb 日志信息
2023/10/19 12:08:13.036 +08:00] [FATAL] [session.go:1368] [“check bootstrapped failed”] [error=“[tikv:9005]Region is unavailable[try again later]”] [errorVerbose="[tikv:9005]Region is unavailable[try again later]
查看ps进程
[tidb@TiDB-test09 ~]$ ps -ef|grep tidb-server
tidb 29604 1 1 12:17 ? 00:00:00 bin/tidb-server -P 4000 --status=10080 --advertise-address=10.3.0.147 --path=10.3.0.147:2379,10.3.0.148:2379,10.3.0.149:2379 --config=conf/tidb.toml --log-slow-query=/lsb/tidb/debloy/log/tidb_slow_query.log --log-file=/lsb/tidb/debloy/log/tidb.log
tidb 29639 29560 0 12:18 pts/2 00:00:00 grep --color=auto tidb-server
[tidb@TiDB-test09 ~]$ netstat -an|grep 4000
[tidb@TiDB-test09 ~]$