机房意外断电，集群宿主机服务器宕机后，重启整个tidb集群，发现2个tidb-server和1个tikv启动失败。

vcdog · 2025 年10 月 13 日 07:29

【TiDB 使用环境】生产环境
【TiDB 版本】v6.5.0
【操作系统】CentOS Linux release 7.9
【部署方式】云下本地机房部署
【集群数据量】1个
【集群节点数】5个节点
【问题复现路径】机房意外断电，集群宿主机服务器宕机后，重启整个tidb集群，发现2个tidb-server和1个tikv启动失败。
【遇到的问题：问题现象及影响】
【资源配置】进入到 TiDB Dashboard -集群信息 (Cluster Info) -主机(Hosts) 截图此页面
【复制黏贴 ERROR 报错的日志】

1、tidb的错误日志如下：

[2025/10/13 15:23:46.399 +08:00] [FATAL] [terror.go:300] ["unexpected error"] [error="[tikv:9005]Region is unavailable"] [stack="github.com/pingcap/tidb/parser/terror.MustNil\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/parser/terror/terror.go:300\nmain.createStoreAndDomain\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/tidb-server/main.go:315\nmain.main\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/tidb-server/main.go:214\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250"] [stack="github.com/pingcap/tidb/parser/terror.MustNil\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/parser/terror/terror.go:300\nmain.createStoreAndDomain\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/tidb-server/main.go:315\nmain.main\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/tidb-server/main.go:214\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250"]

2、tikv的错误日志如下：

025/10/13 15:22:57.869 +08:00] [INFO] [engine.rs:91] ["Recovering raft logs takes 9.924494026s"]
[2025/10/13 15:22:58.814 +08:00] [ERROR] [engine_factory.rs:164] ["failed to create kv engine"] [err="Engine(Status { code: IoError, sub_code: None, sev: NoError, state: \"IO error: While renaming a file to /acdata/tidb-data/tikv-20163/db/CURRENT: /acdata/tidb-data/tikv-20163/db/26226887.dbtmp.plain: Structure needs cleaning\" })"] [path=/acdata/tidb-data/tikv-20163/db]
[2025/10/13 15:22:58.814 +08:00] [FATAL] [server.rs:1828] ["failed to create kv engine: Storage Engine Status { code: IoError, sub_code: None, sev: NoError, state: \"IO error: While renaming a file to /acdata/tidb-data/tikv-20163/db/CURRENT: /acdata/tidb-data/tikv-20163/db/26226887.dbtmp.plain: Structure needs cleaning\" }"]

tidb-server一共有2个节点，均无法启动。tikv节点有2个，1个可以正常启动，另一个无法启动。

【其他附件：截图/日志/监控】

vcdog · 2025 年10 月 13 日 07:44

故障节点tikv的最新日志：

[2025/10/13 15:43:20.789 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=328718087] [region_id=3495887]
[2025/10/13 15:43:20.789 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=979029532] [region_id=3495915]
[2025/10/13 15:43:20.789 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=78767001] [region_id=3495913]
[2025/10/13 15:43:20.789 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=435851421] [region_id=3495881]
[2025/10/13 15:43:20.789 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=980277002] [region_id=3495917]
[2025/10/13 15:43:20.789 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=1059114457] [region_id=3495957]
[2025/10/13 15:43:20.789 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=1119981127] [region_id=3495947]
[2025/10/13 15:43:20.789 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=979408434] [region_id=3495929]
[2025/10/13 15:43:20.789 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=1119982236] [region_id=3495941]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1550] ["starting a new election"] [term=9] [raft_id=585066369] [region_id=585066367]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1170] ["became pre-candidate at term 9"] [term=9] [raft_id=585066369] [region_id=585066367]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1299] ["broadcasting vote request"] [to="[585066368]"] [log_index=21567] [log_term=9] [term=9] [type=MsgRequestPreVote] [raft_id=585066369] [region_id=585066367]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1550] ["starting a new election"] [term=8] [raft_id=1162394003] [region_id=1162394001]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1170] ["became pre-candidate at term 8"] [term=8] [raft_id=1162394003] [region_id=1162394001]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1550] ["starting a new election"] [term=8] [raft_id=1162391849] [region_id=1162391847]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1170] ["became pre-candidate at term 8"] [term=8] [raft_id=1162391849] [region_id=1162391847]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1299] ["broadcasting vote request"] [to="[1162394002]"] [log_index=218049] [log_term=8] [term=8] [type=MsgRequestPreVote] [raft_id=1162394003] [region_id=1162394001]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1299] ["broadcasting vote request"] [to="[1162391848]"] [log_index=861300] [log_term=8] [term=8] [type=MsgRequestPreVote] [raft_id=1162391849] [region_id=1162391847]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1550] ["starting a new election"] [term=9] [raft_id=1162319129] [region_id=3603673]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1170] ["became pre-candidate at term 9"] [term=9] [raft_id=1162319129] [region_id=3603673]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1299] ["broadcasting vote request"] [to="[3603674]"] [log_index=4980] [log_term=9] [term=9] [type=MsgRequestPreVote] [raft_id=1162319129] [region_id=3603673]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1550] ["starting a new election"] [term=26] [raft_id=523424940] [region_id=3247701]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1170] ["became pre-candidate at term 26"] [term=26] [raft_id=523424940] [region_id=3247701]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1299] ["broadcasting vote request"] [to="[3247702]"] [log_index=45] [log_term=26] [term=26] [type=MsgRequestPreVote] [raft_id=523424940] [region_id=3247701]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1550] ["starting a new election"] [term=9] [raft_id=737732309] [region_id=737732307]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1170] ["became pre-candidate at term 9"] [term=9] [raft_id=737732309] [region_id=737732307]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1550] ["starting a new election"] [term=27] [raft_id=761650804] [region_id=3255335]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1170] ["became pre-candidate at term 27"] [term=27] [raft_id=761650804] [region_id=3255335]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1299] ["broadcasting vote request"] [to="[3255336]"] [log_index=569756] [log_term=27] [term=27] [type=MsgRequestPreVote] [raft_id=761650804] [region_id=3255335]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1299] ["broadcasting vote request"] [to="[737732308]"] [log_index=992081] [log_term=9] [term=9] [type=MsgRequestPreVote] [raft_id=737732309] [region_id=737732307]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1550] ["starting a new election"] [term=9] [raft_id=204707433] [region_id=3604859]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1170] ["became pre-candidate at term 9"] [term=9] [raft_id=204707433] [region_id=3604859]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1299] ["broadcasting vote request"] [to="[3604860]"] [log_index=12] [log_term=9] [term=9] [type=MsgRequestPreVote] [raft_id=204707433] [region_id=3604859]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1550] ["starting a new election"] [term=9] [raft_id=919413202] [region_id=3604655]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1170] ["became pre-candidate at term 9"] [term=9] [raft_id=919413202] [region_id=3604655]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1299] ["broadcasting vote request"] [to="[3604656]"] [log_index=13] [log_term=9] [term=9] [type=MsgRequestPreVote] [raft_id=919413202] [region_id=3604655]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1550] ["starting a new election"] [term=9] [raft_id=743694001] [region_id=3604437]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1170] ["became pre-candidate at term 9"] [term=9] [raft_id=743694001] [region_id=3604437]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1299] ["broadcasting vote request"] [to="[3604438]"] [log_index=1069] [log_term=9] [term=9] [type=MsgRequestPreVote] [raft_id=743694001] [region_id=3604437]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1550] ["starting a new election"] [term=10] [raft_id=900757602] [region_id=3602781]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1170] ["became pre-candidate at term 10"] [term=10] [raft_id=900757602] [region_id=3602781]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1299] ["broadcasting vote request"] [to="[3602782]"] [log_index=25659] [log_term=10] [term=10] [type=MsgRequestPreVote] [raft_id=900757602] [region_id=3602781]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1550] ["starting a new election"] [term=27] [raft_id=1096042106] [region_id=3252263]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1170] ["became pre-candidate at term 27"] [term=27] [raft_id=1096042106] [region_id=3252263]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1299] ["broadcasting vote request"] [to="[3252264]"] [log_index=12204] [log_term=27] [term=27] [type=MsgRequestPreVote] [raft_id=1096042106] [region_id=3252263]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1550] ["starting a new election"] [term=11] [raft_id=906569044] [region_id=3598869]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1170] ["became pre-candidate at term 11"] [term=11] [raft_id=906569044] [region_id=3598869]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1299] ["broadcasting vote request"] [to="[3598870]"] [log_index=478939] [log_term=11] [term=11] [type=MsgRequestPreVote] [raft_id=906569044] [region_id=3598869]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1550] ["starting a new election"] [term=10] [raft_id=1142442313] [region_id=3602425]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1170] ["became pre-candidate at term 10"] [term=10] [raft_id=1142442313] [region_id=3602425]
[2025/10/13 15:43:20.789 +08:00] [INFO] [raft.rs:1299] ["broadcasting vote request"] [to="[3602426]"] [log_index=199925] [log_term=10] [term=10] [type=MsgRequestPreVote] [raft_id=1142442313] [region_id=3602425]
[2025/10/13 15:43:20.790 +08:00] [INFO] [raft.rs:1550] ["starting a new election"] [term=26] [raft_id=935124766] [region_id=3247657]
[2025/10/13 15:43:20.790 +08:00] [INFO] [raft.rs:1170] ["became pre-candidate at term 26"] [term=26] [raft_id=935124766] [region_id=3247657]
[2025/10/13 15:43:20.790 +08:00] [INFO] [raft.rs:1550] ["starting a new election"] [term=26] [raft_id=1119981495] [region_id=3251547]
[2025/10/13 15:43:20.790 +08:00] [INFO] [raft.rs:1299] ["broadcasting vote request"] [to="[3247658]"] [log_index=289] [log_term=26] [term=26] [type=MsgRequestPreVote] [raft_id=935124766] [region_id=3247657]
[2025/10/13 15:43:20.790 +08:00] [INFO] [raft.rs:1170] ["became pre-candidate at term 26"] [term=26] [raft_id=1119981495] [region_id=3251547]
[2025/10/13 15:43:20.790 +08:00] [INFO] [raft.rs:1299] ["broadcasting vote request"] [to="[3251548]"] [log_index=43029] [log_term=26] [term=26] [type=MsgRequestPreVote] [raft_id=1119981495] [region_id=3251547]
[2025/10/13 15:43:20.790 +08:00] [INFO] [raft.rs:1550] ["starting a new election"] [term=17] [raft_id=1046009125] [region_id=3597197]
[2025/10/13 15:43:20.790 +08:00] [INFO] [raft.rs:1170] ["became pre-candidate at term 17"] [term=17] [raft_id=1046009125] [region_id=3597197]
[2025/10/13 15:43:20.790 +08:00] [INFO] [raft.rs:1299] ["broadcasting vote request"] [to="[3597198]"] [log_index=178288] [log_term=17] [term=17] [type=MsgRequestPreVote] [raft_id=1046009125] [region_id=3597197]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=291774538] [region_id=3496299]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=837294496] [region_id=3496429]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=355644053] [region_id=3496359]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=1162356915] [region_id=3496315]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=44027001] [region_id=3496381]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=1087287749] [region_id=3496409]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=727612110] [region_id=3496363]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=762771361] [region_id=3496379]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=671980673] [region_id=3496295]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=1162332696] [region_id=3496421]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=672915766] [region_id=3496373]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=836195504] [region_id=3496377]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=1162333355] [region_id=3496383]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=839476129] [region_id=3496375]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=1162342076] [region_id=3496343]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=1162333053] [region_id=3496371]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=957843319] [region_id=3496297]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=968402238] [region_id=3496357]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=1021907932] [region_id=3496369]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=1162357368] [region_id=3496367]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=1162341810] [region_id=3496387]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=673526994] [region_id=3496311]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=43688291] [region_id=3496313]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=948116867] [region_id=3496309]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=1162353891] [region_id=3496307]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=477133071] [region_id=3496331]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=1162340648] [region_id=3496301]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=1162341935] [region_id=3496325]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=978873816] [region_id=3496329]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=1162320884] [region_id=3496335]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=1162348303] [region_id=3496303]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=1162347273] [region_id=3496349]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=868733984] [region_id=3496327]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=1162358363] [region_id=3496365]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=728143056] [region_id=3496305]
[2025/10/13 15:43:20.790 +08:00] [WARN] [peer.rs:6093] ["leader missing longer than abnormal_leader_missing_duration"] [expect=10m] [peer_id=1162355019] [region_id=3496319]

像风一样的男子 · 2025 年10 月 13 日 07:46

2个kv节点是怎么部署的？单副本？pd节点状态是什么？

wzf0072 · 2025 年10 月 13 日 07:53

tidb-server是无状态的，可以重新部署。无法启动是指启动失败还是反复重启？
tikv 2个节点？数据是单副本的吗？

vcdog · 2025 年10 月 13 日 07:54

2个kv节点，是2副本模式。pd节点状态正常。

vcdog · 2025 年10 月 13 日 07:57

tidb-server启动后，只有进程，没有监听端口。

说明没有启动成功。从日志报错，也可以看到报错误[error=“[tikv:9005]Region is unavailable”]，说明要读取的这个region恰好分布在有问题的那个tikv节点上。

wzf0072 · 2025 年10 月 13 日 07:58

多副本失败恢复

vcdog · 2025 年10 月 13 日 08:12

收到，我试试大佬的这个方法

vcdog · 2025 年10 月 13 日 09:35

这么大的量级的region恢复，应该怎么操作呢？

TiDBer_HErMeXDz · 2025 年10 月 14 日 01:06

可以哟。收藏了。

麻辣机师 · 2025 年10 月 14 日 01:27

"IO error: While renaming a file to /acdata/tidb-data/tikv-20163/db/CURRENT: /acdata/tidb-data/tikv-20163/db/26226887.dbtmp.plain: Structure needs cleaning
– 文件损坏或磁盘有问题

小龙虾爱大龙虾 · 2025 年10 月 14 日 01:27

需要做 online unsafe recovery 了，有数据丢失了，参考：https://docs.pingcap.com/zh/tidb/stable/online-unsafe-recovery/#online-unsafe-recovery-使用文档

你的集群很奇怪 4 PD、2 TiKV，2 TiKV、2 副本不是正确使用 tidb 的姿势，这个跟 Mysql 不一样

BraveChen · 2025 年10 月 14 日 01:58

看起来你这个有次节点文件损坏，但是两副本又导致必须要全部没问题才可以，没有容错。

舞动梦灵 · 2025 年10 月 14 日 06:17

你这个设计，我不知道你们的理由是什么，感觉很搞笑，设置2副本模式，没有一点容错性，和1副本没什么区别，都是tikv挂掉任意一台都需要恢复数据的。

zhanggame1 · 2025 年10 月 14 日 06:25

tikv节点有2个只要一个出问题就得进行
unsafe recovery

vcdog · 2025 年10 月 15 日 01:02

最初的时候，是单独一个tikv节点呢，数据量比较大，4.5T。可能考虑节省成本。

麻辣机师 · 2025 年10 月 15 日 01:10

我宁愿混部，保证高可用，反正磁盘费用还行

北极星DB · 2025 年10 月 15 日 02:55

有道理，一般至少3副本才对吧

舞动梦灵 · 2025 年10 月 15 日 09:49

不管几个节点。要么1副本要么3副本，你2副本占空间又多余，就算1个tikv节点。也可以3副本呀，只不过三个副本都在同一个机器上面，要么这个机器一块盘放三个副本，要么你挂3个盘，当成3个tikv来用。

gaozu · 2025 年10 月 17 日 07:30

我也试下.