【求助】机器重装系统 新建表 SQL执行异常

【 TiDB 使用环境】生产环境 /测试/ Poc
【 TiDB 版本】 v7.1.1
【复现路径】做过哪些操作出现的问题
pd和tidb是在同一台机器,系统重装后,使用pd-recover恢复pd,重新新建集群 版本从6.5.2升级7.1.1
【遇到的问题:问题现象及影响】
新建表执行SQL报错PD server timeout,旧表正常
[2023/09/26 10:04:42.147 +08:00] [ERROR] [manager.go:328] [“task manager error”] [dist_task_manager=192.168.1.70:4000] [error=“[tikv:9001]PD server timeout: “]
[2023/09/26 10:04:42.330 +08:00] [WARN] [backoff.go:158] [“pdRPC backoffer.maxSleep 10000ms is exceeded, errors:\nregion not found for key "748000000000002B0E5F720000000000000000" at 2023-09-26T10:04:34.27196196+08:00\nregion not found for key "748000000000002B0E5F720000000000000000" at 2023-09-26T10:04:36.85500499+08:00\nregion not found for key "748000000000002B0E5F720000000000000000" at 2023-09-26T10:04:39.470411601+08:00\nlongest sleep type: pdRPC, time: 12058ms”]
[2023/09/26 10:04:42.331 +08:00] [INFO] [conn.go:1184] [“command dispatched failed”] [conn=5165622175524192701] [connInfo=“id:5165622175524192701, addr:192.168.1.1:55895 status:10, collation:utf8_general_ci, user:root”] [command=Query] [status=“inTxn:0, autocommit:1”] [sql=“SELECT * FROM yqc_data_count.kafka_date_check LIMIT 0,1000”] [txn_mode=PESSIMISTIC] [timestamp=444515973925699585] [err=”[tikv:9001]PD server timeout: \ngithub.com/pingcap/errors.AddStack\n\t/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20221009092201-b66cddb77c32/errors.go:174\ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs\n\t/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20221009092201-b66cddb77c32/normalize.go:164\ngithub.com/pingcap/tidb/store/driver/error.ToTiDBErr\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/driver/error/error.go:112\ngithub.com/pingcap/tidb/store/copr.(*RegionCache).SplitKeyRangesByLocations\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/region_cache.go:139\ngithub.com/pingcap/tidb/store/copr.(*RegionCache).SplitKeyRangesByBuckets\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/region_cache.go:187\ngithub.com/pingcap/tidb/store/copr.buildCopTasks\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:335\ngithub.com/pingcap/tidb/store/copr.(*CopClient).BuildCopIterator.func3\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:148\ngithub.com/pingcap/tidb/kv.(*KeyRanges).ForEachPartitionWithErr\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/kv/kv.go:456\ngithub.com/pingcap/tidb/store/copr.(*CopClient).BuildCopIterator\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:162\ngithub.com/pingcap/tidb/store/copr.(*CopClient).Send\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:92\ngithub.com/pingcap/tidb/distsql.Select\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/distsql/distsql.go:99\ngithub.com/pingcap/tidb/distsql.SelectWithRuntimeStats\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/distsql/distsql.go:149\ngithub.com/pingcap/tidb/executor.selectResultHook.SelectResult\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/table_reader.go:59\ngithub.com/pingcap/tidb/executor.(*TableReaderExecutor).buildResp\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/table_reader.go:347\ngithub.com/pingcap/tidb/executor.(*TableReaderExecutor).Open\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/table_reader.go:224\ngithub.com/pingcap/tidb/executor.(*baseExecutor).Open\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:204\ngithub.com/pingcap/tidb/executor.(*LimitExec).Open\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:1492\ngithub.com/pingcap/tidb/executor.(*ExecStmt).openExecutor\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/adapter.go:1195\ngithub.com/pingcap/tidb/executor.(*ExecStmt).Exec\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/adapter.go:531\ngithub.com/pingcap/tidb/session.runStmt\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/session/session.go:2394\ngithub.com/pingcap/tidb/session.(*session).ExecuteStmt\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/session/session.go:2251\ngithub.com/pingcap/tidb/server.(*TiDBContext).ExecuteStmt\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/driver_tidb.go:294\ngithub.com/pingcap/tidb/server.(*clientConn).handleStmt\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:2094\ngithub.com/pingcap/tidb/server.(*clientConn).handleQuery\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1885\ngithub.com/pingcap/tidb/server.(*clientConn).dispatch\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1372\ngithub.com/pingcap/tidb/server.(*clientConn).Run\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1153\ngithub.com/pingcap/tidb/server.(*Server).onConn\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/server.go:677\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1598”]

【资源配置】
【附件:截图/日志/监控】
pd.log (763.6 KB)
tidb.log (1.3 MB)

你是确定恢复正常了才升级的吗?升级之前ddl操作能正常做吗?

升级前的正常使用的,是执行tiup cluster deploy yqc v7.1.1 ./prod.yaml --user mcjt -p后才进行恢复的

你的意思是你原来用的集群版本是6.5.2,然后你通过tiup cluster deploy yqc v7.1.1 ./prod.yaml 生成了7.1.1的tiup来管理组件吗?不是通过tiup update cluster这种方式升级的吗?

是的 因为机器系统重做 数据被被格式化了 后来我就从头开始 执行
#检查集群潜在风险
tiup cluster check ./prod.yaml --user mcjt -p
#首次部署
tiup cluster deploy yqc v7.1.1 ./prod.yaml --user mcjt -p
#安全启动
tiup cluster start yqc --init
在启动时候发现pd和tikv无法连接就找官网查到pd-recover可以恢复 按照操作后 集群正常启动 今天新建表执行SQL就出现PD Server Timeout

你TIDB日志不是写的么
/ext/tmp/tmp_ddl-4000硬盘空间
[error=“the available disk space(45788008448) in /ext/tmp/tmp_ddl-4000 should be greater than @@tidb_ddl_disk_quota(107374182400)”]

你的SQL
[sql=“select pid,count(id) dim_num,‘certificationsummary’ as dim_type,DATE_FORMAT(CURDATE(), ‘%Y-%m-%d’) as pdate\r\nfrom bu_cert_info\r\nwhere is_deleted = 0\r\ngroup by pid”]

报错内存不足
[err="Your query has been cancelled due to exceeding the allowed memory limit for a single SQL query. Please try narrowing your query scope or increase the tidb_mem_quota_query limit and try again.

完整输出日志
[2023/09/26 09:59:17.196 +08:00] [INFO] [conn.go:1184] [“command dispatched failed”] [conn=5165622175524192659] [connInfo=“id:5165622175524192659, addr:192.168.1.243:57240 status:10, collation:utf8_general_ci, user:root”] [command=Query] [status=“inTxn:0, autocommit:1”] [sql=“select pid,count(id) dim_num,‘certificationsummary’ as dim_type,DATE_FORMAT(CURDATE(), ‘%Y-%m-%d’) as pdate\r\nfrom bu_cert_info\r\nwhere is_deleted = 0\r\ngroup by pid”] [txn_mode=PESSIMISTIC] [timestamp=444515888217980929] [err="Your query has been cancelled due to exceeding the allowed memory limit for a single SQL query. Please try narrowing your query scope or increase the tidb_mem_quota_query limit and try again.


内存使用率10%,而且只有新建的表yqc_data_count.kafka_date_check才会出现,旧表是正常

你看下/ext目录的空间是不是不足了,df -h看下,挂在个大一点的磁盘

这样子弄很容易出问题,建议集群按照官方推荐的 tiup 升级,不建议集群的版本和各个组件的版本不一致。

tiup list --verbose
看一下你各个组件的版本号师傅存在不一致的情况
information_schema.cluster_info 也可以看看这个视图!

先把不一致的组件先升级上来

set global tidb_ddl_enable_fast_reorg=off;先把索引加速功能关闭吧,这个功能开启要求tidb_ddl_disk_quota这个参数起码100G,我看你的空间完全不够。。。。

谢谢你的帮忙 我关闭了 还是不行

还报错吗?你看下执行ddl语句的时候,看看日志报什么错。
另外上面的参数改了需要重新登录下,新的session才生效,想当前session生效set tidb_ddl_enable_fast_reorg=off;不能加global

[2023/09/27 08:42:16.115 +08:00] [WARN] [backoff.go:158] [“pdRPC backoffer.maxSleep 10000ms is exceeded, errors:\nregion not found for key "748000000000002B1A5F720000000000000000" at 2023-09-27T08:42:09.556344803+08:00\nregion not found for key "748000000000002B1A5F720000000000000000" at 2023-09-27T08:42:11.663201037+08:00\nregion not found for key "748000000000002B1A5F720000000000000000" at 2023-09-27T08:42:14.544404707+08:00\nlongest sleep type: pdRPC, time: 10712ms”]
[2023/09/27 08:42:16.116 +08:00] [INFO] [conn.go:1184] [“command dispatched failed”] [conn=5165622175524193315] [connInfo=“id:5165622175524193315, addr:192.168.1.1:61524 status:10, collation:utf8_general_ci, user:root”] [command=Query] [status=“inTxn:0, autocommit:1”] [sql=“SELECT * FROM yqc_data_count.kafka_date_check LIMIT 0,1000”] [txn_mode=PESSIMISTIC] [timestamp=444537326891696131] [err=“[tikv:9001]PD server timeout: \ngithub.com/pingcap/errors.AddStack\n\t/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20221009092201-b66cddb77c32/errors.go:174\ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs\n\t/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20221009092201-b66cddb77c32/normalize.go:164\ngithub.com/pingcap/tidb/store/driver/error.ToTiDBErr\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/driver/error/error.go:112\ngithub.com/pingcap/tidb/store/copr.(*RegionCache).SplitKeyRangesByLocations\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/region_cache.go:139\ngithub.com/pingcap/tidb/store/copr.(*RegionCache).SplitKeyRangesByBuckets\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/region_cache.go:187\ngithub.com/pingcap/tidb/store/copr.buildCopTasks\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:335\ngithub.com/pingcap/tidb/store/copr.(*CopClient).BuildCopIterator.func3\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:148\ngithub.com/pingcap/tidb/kv.(*KeyRanges).ForEachPartitionWithErr\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/kv/kv.go:456\ngithub.com/pingcap/tidb/store/copr.(*CopClient).BuildCopIterator\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:162\ngithub.com/pingcap/tidb/store/copr.(*CopClient).Send\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:92\ngithub.com/pingcap/tidb/distsql.Select\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/distsql/distsql.go:99\ngithub.com/pingcap/tidb/distsql.SelectWithRuntimeStats\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/distsql/distsql.go:149\ngithub.com/pingcap/tidb/executor.selectResultHook.SelectResult\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/table_reader.go:59\ngithub.com/pingcap/tidb/executor.(*TableReaderExecutor).buildResp\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/table_reader.go:347\ngithub.com/pingcap/tidb/executor.(*TableReaderExecutor).Open\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/table_reader.go:224\ngithub.com/pingcap/tidb/executor.(*baseExecutor).Open\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:204\ngithub.com/pingcap/tidb/executor.(*LimitExec).Open\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:1492\ngithub.com/pingcap/tidb/executor.(*ExecStmt).openExecutor\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/adapter.go:1195\ngithub.com/pingcap/tidb/executor.(*ExecStmt).Exec\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/adapter.go:531\ngithub.com/pingcap/tidb/session.runStmt\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/session/session.go:2394\ngithub.com/pingcap/tidb/session.(*session).ExecuteStmt\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/session/session.go:2251\ngithub.com/pingcap/tidb/server.(*TiDBContext).ExecuteStmt\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/driver_tidb.go:294\ngithub.com/pingcap/tidb/server.(*clientConn).handleStmt\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:2094\ngithub.com/pingcap/tidb/server.(*clientConn).handleQuery\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1885\ngithub.com/pingcap/tidb/server.(*clientConn).dispatch\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1372\ngithub.com/pingcap/tidb/server.(*clientConn).Run\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1153\ngithub.com/pingcap/tidb/server.(*Server).onConn\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/server.go:677\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1598”]
[2023/09/27 08:42:16.383 +08:00] [WARN] [backoff.go:158] [“pdRPC backoffer.maxSleep 10000ms is exceeded, errors:\nregion not found for key "748000000000001D6C5F69800000000000000101726576657274696EFF6700000000000000F8" at 2023-09-27T08:42:09.952832129+08:00\nregion not found for key "748000000000001D6C5F69800000000000000101726576657274696EFF6700000000000000F8" at 2023-09-27T08:42:11.926583009+08:00\nregion not found for key "748000000000001D6C5F69800000000000000101726576657274696EFF6700000000000000F8" at 2023-09-27T08:42:14.759066156+08:00\nlongest sleep type: pdRPC, time: 11427ms”]
[2023/09/27 08:42:16.384 +08:00] [ERROR] [manager.go:328] [“task manager error”] [dist_task_manager=192.168.1.70:4000] [error=“[tikv:9001]PD server timeout: “]
[2023/09/27 08:42:16.594 +08:00] [WARN] [backoff.go:158] [“pdRPC backoffer.maxSleep 10000ms is exceeded, errors:\nregion not found for key "748000000000001D6C5F6980000000000000010163616E63656C6C69FF6E67000000000000F9" at 2023-09-27T08:42:09.711039983+08:00\nregion not found for key "748000000000001D6C5F6980000000000000010163616E63656C6C69FF6E67000000000000F9" at 2023-09-27T08:42:11.279578991+08:00\nregion not found for key "748000000000001D6C5F6980000000000000010163616E63656C6C69FF6E67000000000000F9" at 2023-09-27T08:42:14.048524485+08:00\nlongest sleep type: pdRPC, time: 12035ms”]
[2023/09/27 08:42:16.594 +08:00] [WARN] [dispatcher.go:160] [“get unfinished(pending, running or reverting) tasks failed”] [error=”[tikv:9001]PD server timeout: “]
[2023/09/27 08:42:16.948 +08:00] [WARN] [backoff.go:158] [“pdRPC backoffer.maxSleep 10000ms is exceeded, errors:\nregion not found for key "748000000000001D6C5F69800000000000000101726576657274696EFF6700000000000000F8" at 2023-09-27T08:42:10.879404659+08:00\nregion not found for key "748000000000001D6C5F69800000000000000101726576657274696EFF6700000000000000F8" at 2023-09-27T08:42:13.18534525+08:00\nregion not found for key "748000000000001D6C5F69800000000000000101726576657274696EFF6700000000000000F8" at 2023-09-27T08:42:14.743313155+08:00\nlongest sleep type: pdRPC, time: 10965ms”]
[2023/09/27 08:42:16.949 +08:00] [ERROR] [manager.go:328] [“task manager error”] [dist_task_manager=192.168.1.70:4000] [error=”[tikv:9001]PD server timeout: “]
[2023/09/27 08:42:23.022 +08:00] [INFO] [gc_worker.go:442] [”[gc worker] starts the whole job”] [uuid=62b3cd5e3e00005] [safePoint=444537174219030528] [concurrency=3]
[2023/09/27 08:42:23.025 +08:00] [INFO] [gc_worker.go:1250] [“[gc worker] start resolve locks”] [uuid=62b3cd5e3e00005] [safePoint=444537174219030528] [try-resolve-locks-ts=444537174219030528] [concurrency=3]
[2023/09/27 08:42:23.025 +08:00] [INFO] [range_task.go:137] [“range task started”] [name=resolve-locks-runner] [startKey=] [endKey=] [concurrency=3]
[2023/09/27 08:42:27.640 +08:00] [WARN] [backoff.go:158] [“pdRPC backoffer.maxSleep 10000ms is exceeded, errors:\nregion not found for key "748000000000001D6C5F69800000000000000101726576657274696EFF6700000000000000F8" at 2023-09-27T08:42:20.591541654+08:00\nregion not found for key "748000000000001D6C5F69800000000000000101726576657274696EFF6700000000000000F8" at 2023-09-27T08:42:22.6222926+08:00\nregion not found for key "748000000000001D6C5F69800000000000000101726576657274696EFF6700000000000000F8" at 2023-09-27T08:42:25.547932947+08:00\nlongest sleep type: pdRPC, time: 11246ms”]
[2023/09/27 08:42:27.640 +08:00] [ERROR] [manager.go:328] [“task manager error”] [dist_task_manager=192.168.1.70:4000] [error="[tikv:9001]PD server timeout: “]
[2023/09/27 08:42:27.762 +08:00] [WARN] [backoff.go:158] [“pdRPC backoffer.maxSleep 10000ms is exceeded, errors:\nregion not found for key "748000000000001D6C5F69800000000000000101726576657274696EFF6700000000000000F8" at 2023-09-27T08:42:20.212388224+08:00\nregion not found for key "748000000000001D6C5F69800000000000000101726576657274696EFF6700000000000000F8" at 2023-09-27T08:42:23.17796901+08:00\nregion not found for key "748000000000001D6C5F69800000000000000101726576657274696EFF6700000000000000F8" at 2023-09-27T08:42:26.146820825+08:00\nlongest sleep type: pdRPC, time: 10805ms”]
[2023/09/27 08:42:27.762 +08:00] [ERROR] [manager.go:328] [“task manager error”] [dist_task_manager=192.168.1.70:4000] [error=”[tikv:9001]PD server timeout: “]
[2023/09/27 08:42:29.351 +08:00] [WARN] [backoff.go:158] [“pdRPC backoffer.maxSleep 10000ms is exceeded, errors:\nregion not found for key "748000000000001D6C5F6980000000000000010163616E63656C6C69FF6E67000000000000F9" at 2023-09-27T08:42:20.580868359+08:00\nregion not found for key "748000000000001D6C5F6980000000000000010163616E63656C6C69FF6E67000000000000F9" at 2023-09-27T08:42:23.509420725+08:00\nregion not found for key "748000000000001D6C5F6980000000000000010163616E63656C6C69FF6E67000000000000F9" at 2023-09-27T08:42:26.399567688+08:00\nlongest sleep type: pdRPC, time: 12748ms”]
[2023/09/27 08:42:29.352 +08:00] [WARN] [dispatcher.go:160] [“get unfinished(pending, running or reverting) tasks failed”] [error=”[tikv:9001]PD server timeout: "]
[2023/09/27 08:42:33.584 +08:00] [WARN] [backoff.go:158] [“pdRPC backoffer.maxSleep 10000ms is exceeded, errors:\nregion not found for key "748000000000001D665F720000000000000000" at 2023-09-27T08:42:26.891839985+08:00\nregion not found for key "748000000000001D665F720000000000000000" at 2023-09-27T08:42:28.764991961+08:00\nregion not found for key "748000000000001D665F720000000000000000" at 2023-09-27T08:42:30.607323161+08:00\nlongest sleep type: pdRPC, time: 10842ms”]
[2023/09/27 08:42:33.584 +08:00] [WARN] [task_manager.go:280] [“fail to peek scan task”] [ttl-worker=job-manager] [ttl-worker=task-manager] [error="execute sql: SELECT LOW_PRIORITY\n\tjob_id,\n\ttable_id,\n\tscan_id,\n\tscan_range_start,\n\tscan_range_end,\n\texpire_time,\n\towner_id,\n\towner_addr,\n\towner_hb_time,\n\tstatus,\n\tstatus_update_time,\n\tstate,\n\tcreated_time FROM mysql.tidb_ttl_task WHERE status = ‘waiting’ OR (owner_hb_time < %? AND status = ‘running’) ORDER BY created_time ASC: [tikv:9001]PD server timeout: “]
[2023/09/27 08:42:39.024 +08:00] [WARN] [backoff.go:158] [“pdRPC backoffer.maxSleep 10000ms is exceeded, errors:\nregion not found for key "748000000000001D6C5F69800000000000000101726576657274696EFF6700000000000000F8" at 2023-09-27T08:42:32.149109126+08:00\nregion not found for key "748000000000001D6C5F69800000000000000101726576657274696EFF6700000000000000F8" at 2023-09-27T08:42:34.038721168+08:00\nregion not found for key "748000000000001D6C5F69800000000000000101726576657274696EFF6700000000000000F8" at 2023-09-27T08:42:36.597258641+08:00\nlongest sleep type: pdRPC, time: 11375ms”]
[2023/09/27 08:42:39.025 +08:00] [ERROR] [manager.go:328] [“task manager error”] [dist_task_manager=192.168.1.70:4000] [error=”[tikv:9001]PD server timeout: "]

节点踢出然后再加入呢

踢出pd还是tidb还是tikv?pd和tidb都是单机

可能还是你集群的版本已经有问题了,建议导出数据,重新创建集群后导入数据吧。。。

好吧 .