【 TiDB 使用环境】生产环境
【 TiDB 版本】v7.5.0
【复现路径】TIKV 缩容报错
[2024/10/25 12:26:12.508 +08:00] [INFO] [peer.rs:5489] [“failed to propose”] [err=“EpochNotMatch current epoch of region 15309 is conf_ver: 2 version: 242, but you sent conf_ver: 1 version: 242”] [message=“header { region_id: 15309 peer { id: 15310 store_id: 1 } region_epoch { conf_ver: 1 version: 242 } } admin_request { cmd_type: ChangePeerV2 change_peer_v2 { changes { change_type: AddLearnerNode peer { id: 31297651274 store_id: 75003 role: Learner } } } }”] [peer_id=15310] [region_id=15309] [thread_id=0x5]
[2024/10/25 12:26:12.508 +08:00] [INFO] [pd.rs:1620] [“try to change peer”] [changes=“[peer { id: 31287713205 store_id: 75003 role: Learner } change_type: AddLearnerNode]”] [region_id=15089] [thread_id=0x5]
[2024/10/25 12:26:12.508 +08:00] [INFO] [peer.rs:5489] [“failed to propose”] [err=“EpochNotMatch current epoch of region 14135 is conf_ver: 2 version: 251, but you sent conf_ver: 1 version: 251”] [message=“header { region_id: 14135 peer { id: 14136 store_id: 1 } region_epoch { conf_ver: 1 version: 251 } } admin_request { cmd_type: ChangePeerV2 change_peer_v2 { changes { change_type: AddLearnerNode peer { id: 31299885371 store_id: 75003 role: Learner } } } }”] [peer_id=14136] [region_id=14135] [thread_id=0x5]
[2024/10/25 12:26:12.508 +08:00] [INFO] [peer.rs:5489] [“failed to propose”] [err=“EpochNotMatch current epoch of region 15089 is conf_ver: 2 version: 932, but you sent conf_ver: 1 version: 932”] [message=“header { region_id: 15089 peer { id: 15090 store_id: 1 } region_epoch { conf_ver: 1 version: 932 } } admin_request { cmd_type: ChangePeerV2 change_peer_v2 { changes { change_type: AddLearnerNode peer { id: 31287713205 store_id: 75003 role: Learner } } } }”] [peer_id=15090] [region_id=15089] [thread_id=0x5]
[2024/10/25 12:26:12.508 +08:00] [INFO] [pd.rs:1620] [“try to change peer”] [changes=“[peer { id: 31288085839 store_id: 75003 role: Learner } change_type: AddLearnerNode]”] [region_id=15191] [thread_id=0x5]
[2024/10/25 12:26:12.508 +08:00] [INFO] [pd.rs:1620] [“try to change peer”] [changes=“[peer { id: 31292208393 store_id: 75003 role: Learner } change_type: AddLearnerNode]”] [region_id=173] [thread_id=0x5]
[2024/10/25 12:26:12.508 +08:00] [INFO] [peer.rs:5489] [“failed to propose”] [err=“EpochNotMatch current epoch of region 15191 is conf_ver: 2 version: 969, but you sent conf_ver: 1 version: 969”] [message=“header { region_id: 15191 peer { id: 15192 store_id: 1 } region_epoch { conf_ver: 1 version: 969 } } admin_request { cmd_type: ChangePeerV2 change_peer_v2 { changes { change_type: AddLearnerNode peer { id: 31288085839 store_id: 75003 role: Learner } } } }”] [peer_id=15192] [region_id=15191] [thread_id=0x5]
[2024/10/25 12:26:12.508 +08:00] [INFO] [peer.rs:5489] [“failed to propose”] [err=“EpochNotMatch current epoch of region 173 is conf_ver: 2 version: 80, but you sent conf_ver: 1 version: 80”] [message=“header { region_id: 173 peer { id: 174 store_id: 1 } region_epoch { conf_ver: 1 version: 80 } } admin_request { cmd_type: ChangePeerV2 change_peer_v2 { changes { change_type: AddLearnerNode peer { id: 31292208393 store_id: 75003 role: Learner } } } }”] [peer_id=174] [region_id=173] [thread_id=0x5]
[2024/10/25 12:26:12.508 +08:00] [INFO] [pd.rs:1620] [“try to change peer”] [changes=“[peer { id: 31309848674 store_id: 75003 role: Learner } change_type: AddLearnerNode]”] [region_id=10091] [thread_id=0x5]
[2024/10/25 12:26:12.508 +08:00] [INFO] [pd.rs:1620] [“try to change peer”] [changes=“[peer { id: 31303250654 store_id: 75003 role: Learner } change_type: AddLearnerNode]”] [region_id=10383] [thread_id=0x5]
[2024/10/25 12:26:12.508 +08:00] [INFO] [peer.rs:5489] [“failed to propose”] [err=“EpochNotMatch current epoch of region 10091 is conf_ver: 2 version: 724, but you sent conf_ver: 1 version: 724”] [message=“header { region_id: 10091 peer { id: 10092 store_id: 1 } region_epoch { conf_ver: 1 version: 724 } } admin_request { cmd_type: ChangePeerV2 change_peer_v2 { changes { change_type: AddLearnerNode peer { id: 31309848674 store_id: 75003 role: Learner } } } }”] [peer_id=10092] [region_id=10091] [thread_id=0x5]
[2024/10/25 12:26:12.508 +08:00] [INFO] [peer.rs:5489] [“failed to propose”] [err=“EpochNotMatch current epoch of region 10383 is conf_ver: 2 version: 827, but you sent conf_ver: 1 version: 827”] [message=“header { region_id: 10383 peer { id: 10384 store_id: 1 } region_epoch { conf_ver: 1 version: 827 } } admin_request { cmd_type: ChangePeerV2 change_peer_v2 { changes { change_type: AddLearnerNode peer { id: 31303250654 store_id: 75003 role: Learner } } } }”] [peer_id=10384] [region_id=10383] [thread_id=0x5
分析一下这些tikv的日志
有一个点:store_id = 1 的节点上的副本 都是单副本,其他四个tikv节点都是 3副本 ;
是不是因为 store_id = 1上面的 region 都是单副本 ,所以下线不了。
现在这个store_id节点处于下线中,我要怎么处理呢?