【 TiDB 使用环境`】Ubuntu20虚拟机测试环境
【 TiDB 版本】6.1.0
【遇到的问题】tidb 4000端口启动失败
Error: failed to start tidb: failed to start: 192.168.186.131 tidb-4000.service, please check the instance’s log(/data/tidb-deploy/tidb-4000/log) for more detail.: timed out waiting for port 4000 to be started after 2m0s
【复现路径】pd和tikv可以成功启动，tidb无法成功启动，错误描述和日志如上如下。
配置：
pd_servers:
- host: 192.168.186.131
tidb_servers:
- host: 192.168.186.131
tikv_servers:
- host: 192.168.186.131
port: 20160
status_port: 20180
config:
server.labels: { host: "logic-host-1" }
- host: 192.168.186.132
port: 20161
status_port: 20181
config:
server.labels: { host: "logic-host-2" }
- host: 192.168.186.133
port: 20162
status_port: 20182
config:
server.labels: { host: "logic-host-3" }
tiflash_servers:
- host: 192.168.186.131
monitoring_servers:
- host: 192.168.186.131
grafana_servers:
- host: 192.168.186.131
错误相关的日志如下：
[2022/10/03 13:40:34.352 +00:00] [INFO] [session.go:3274] ["CRUCIAL OPERATION"] [conn=0] [schemaVersion=19] [cur_db=mysql] [sql="CREATE TABLE IF NOT EXISTS mysql.bind_info (\
\t\toriginal_sql TEXT NOT NULL,\
\t\tbind_sql TEXT NOT NULL,\
\t\tdefault_db TEXT NOT NULL,\
\t\tstatus TEXT NOT NULL,\
\t\tcreate_time TIMESTAMP(3) NOT NULL,\
\t\tupdate_time TIMESTAMP(3) NOT NULL,\
\t\tcharset TEXT NOT NULL,\
\t\tcollation TEXT NOT NULL,\
\t\tsource VARCHAR(10) NOT NULL DEFAULT 'unknown',\
\t\tINDEX sql_index(original_sql(700),default_db(68)) COMMENT \"accelerate the speed when add global binding query\",\
\t\tINDEX time_index(update_time) COMMENT \"accelerate the speed when querying with last update time\"\
\t) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;"] [user=]
[2022/10/03 13:41:56.896 +00:00] [WARN] [backoff.go:158] ["tikvDiskFull backoffer.maxSleep 80000ms is exceeded, errors:\
tikv disk full: store_id:1 reason:\"propose failed: tikv disk full, cmd diskFullOpt=NotAllowedOnFull, leader diskUsage=AlmostFull\" ctx: region ID: 3, meta: id:3 start_key:\"t\\200\\000\\000\\000\\000\\000\\000%\" region_epoch:<conf_ver:5 version:18 > peers:<id:4 store_id:1 > peers:<id:51 store_id:2 > peers:<id:77 store_id:7 > , peer: id:4 store_id:1 , addr: 192.168.186.131:20160, idx: 0, reqStoreType: TiKvOnly, runStoreType: tikv at 2022-10-03T13:41:41.891026592Z\
tikv disk full: store_id:1 reason:\"propose failed: tikv disk full, cmd diskFullOpt=NotAllowedOnFull, leader diskUsage=AlmostFull\" ctx: region ID: 3, meta: id:3 start_key:\"t\\200\\000\\000\\000\\000\\000\\000%\" region_epoch:<conf_ver:5 version:18 > peers:<id:4 store_id:1 > peers:<id:51 store_id:2 > peers:<id:77 store_id:7 > , peer: id:4 store_id:1 , addr: 192.168.186.131:20160, idx: 0, reqStoreType: TiKvOnly, runStoreType: tikv at 2022-10-03T13:41:46.892636767Z\
tikv disk full: store_id:1 reason:\"propose failed: tikv disk full, cmd diskFullOpt=NotAllowedOnFull, leader diskUsage=AlmostFull\" ctx: region ID: 3, meta: id:3 start_key:\"t\\200\\000\\000\\000\\000\\000\\000%\" region_epoch:<conf_ver:5 version:18 > peers:<id:4 store_id:1 > peers:<id:51 store_id:2 > peers:<id:77 store_id:7 > , peer: id:4 store_id:1 , addr: 192.168.186.131:20160, idx: 0, reqStoreType: TiKvOnly, runStoreType: tikv at 2022-10-03T13:41:51.895488103Z\
longest sleep type: tikvDiskFull, time: 82500ms"]
[2022/10/03 13:41:56.896 +00:00] [WARN] [backoff.go:158] ["regionMiss backoffer.maxSleep 80000ms is exceeded, errors:\
tikv disk full: store_id:1 reason:\"propose failed: tikv disk full, cmd diskFullOpt=NotAllowedOnFull, leader diskUsage=AlmostFull\" ctx: region ID: 3, meta: id:3 start_key:\"t\\200\\000\\000\\000\\000\\000\\000%\" region_epoch:<conf_ver:5 version:18 > peers:<id:4 store_id:1 > peers:<id:51 store_id:2 > peers:<id:77 store_id:7 > , peer: id:4 store_id:1 , addr: 192.168.186.131:20160, idx: 0, reqStoreType: TiKvOnly, runStoreType: tikv at 2022-10-03T13:41:41.891026592Z\
tikv disk full: store_id:1 reason:\"propose failed: tikv disk full, cmd diskFullOpt=NotAllowedOnFull, leader diskUsage=AlmostFull\" ctx: region ID: 3, meta: id:3 start_key:\"t\\200\\000\\000\\000\\000\\000\\000%\" region_epoch:<conf_ver:5 version:18 > peers:<id:4 store_id:1 > peers:<id:51 store_id:2 > peers:<id:77 store_id:7 > , peer: id:4 store_id:1 , addr: 192.168.186.131:20160, idx: 0, reqStoreType: TiKvOnly, runStoreType: tikv at 2022-10-03T13:41:46.892636767Z\
tikv disk full: store_id:1 reason:\"propose failed: tikv disk full, cmd diskFullOpt=NotAllowedOnFull, leader diskUsage=AlmostFull\" ctx: region ID: 3, meta: id:3 start_key:\"t\\200\\000\\000\\000\\000\\000\\000%\" region_epoch:<conf_ver:5 version:18 > peers:<id:4 store_id:1 > peers:<id:51 store_id:2 > peers:<id:77 store_id:7 > , peer: id:4 store_id:1 , addr: 192.168.186.131:20160, idx: 0, reqStoreType: TiKvOnly, runStoreType: tikv at 2022-10-03T13:41:51.895488103Z\
longest sleep type: tikvDiskFull, time: 82500ms"]
[2022/10/03 13:41:56.896 +00:00] [WARN] [session.go:881] ["can not retry txn"] [label=internal] [error="tikv disk full"] [IsBatchInsert=false] [IsPessimistic=false] [InRestrictedSQL=true] [tidb_retry_limit=10] [tidb_disable_txn_auto_retry=true]
[2022/10/03 13:41:56.896 +00:00] [WARN] [session.go:897] ["commit failed"] ["finished txn"="Txn{state=invalid}"] [error="tikv disk full"]
[2022/10/03 13:41:56.896 +00:00] [WARN] [session.go:1941] ["run statement failed"] [schemaVersion=19] [error="tikv disk full"] [session="{\
\"currDBName\": \"mysql\",\
\"id\": 0,\
\"status\": 2,\
\"strictMode\": true,\
\"user\": null\
}"]
[2022/10/03 13:41:56.896 +00:00] [FATAL] [bootstrap.go:2088] ["mustExecute error"] [error="tikv disk full"] [stack="github.com/pingcap/tidb/session.mustExecute\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/session/bootstrap.go:2088\
github.com/pingcap/tidb/session.insertBuiltinBindInfoRow\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/session/bootstrap.go:1461\
github.com/pingcap/tidb/session.initBindInfoTable\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/session/bootstrap.go:1457\
github.com/pingcap/tidb/session.doDDLWorks\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/session/bootstrap.go:1941\
github.com/pingcap/tidb/session.bootstrap\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/session/bootstrap.go:445\
github.com/pingcap/tidb/session.runInBootstrapSession\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/session/session.go:2941\
github.com/pingcap/tidb/session.BootstrapSession\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/session/session.go:2829\
main.createStoreAndDomain\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/tidb-server/main.go:296\
main.main\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/tidb-server/main.go:202\
runtime.main\
\t/usr/local/go/src/runtime/proc.go:250"]