TiKV 日志中存在 [INFO] err="sending on a disconnected channel"

wl21787 · 2020 年2 月 10 日 02:41

【TiDB 版本】：3.0.7
【问题描述】：tikv.log出现[“failed to schedule peer tick”] [err=“sending on a disconnected channel”]

tikv.log中频繁出现此类信息具体表示什么意思，该日志信息是否为正常现象？

具体日志内容如下：

[2020/02/10 09:52:50.629 +08:00] [INFO] [apply.rs:1034] ["execute admin command"] [command="cmd_type: ChangePeer change_peer { change_type: AddLearnerNode peer { id: 242619 store_id: 6 is_learner: true } }"] [index=219280] [term=8] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:50.629 +08:00] [INFO] [apply.rs:1358] ["exec ConfChange"] [epoch="conf_ver: 23 version: 5404"] [type=AddLearner] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:50.629 +08:00] [INFO] [apply.rs:1498] ["add learner successfully"] [region="id: 173963 start_key: 7480000000000008FF675F72E000000000FF0000000000000000FA end_key: 7480000000000008FF675F72F000000000FFB45FB70000000000FA region_epoch { conf_ver: 23 version: 5404 } peers { id: 173964 store_id: 14 } peers { id: 173966 store_id: 9 } peers { id: 242112 store_id: 4 }"] [peer="id: 242619 store_id: 6 is_learner: true"] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:54.844 +08:00] [INFO] [apply.rs:1034] ["execute admin command"] [command="cmd_type: ChangePeer change_peer { peer { id: 242619 store_id: 6 } }"] [index=219281] [term=8] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:54.844 +08:00] [INFO] [apply.rs:1358] ["exec ConfChange"] [epoch="conf_ver: 24 version: 5404"] [type=AddNode] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:54.844 +08:00] [INFO] [apply.rs:1413] ["add peer successfully"] [region="id: 173963 start_key: 7480000000000008FF675F72E000000000FF0000000000000000FA end_key: 7480000000000008FF675F72F000000000FFB45FB70000000000FA region_epoch { conf_ver: 24 version: 5404 } peers { id: 173964 store_id: 14 } peers { id: 173966 store_id: 9 } peers { id: 242112 store_id: 4 } peers { id: 242619 store_id: 6 is_learner: true }"] [peer="id: 242619 store_id: 6"] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:54.847 +08:00] [INFO] [apply.rs:1034] ["execute admin command"] [command="cmd_type: ChangePeer change_peer { change_type: RemoveNode peer { id: 173964 store_id: 14 } }"] [index=219282] [term=8] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:54.847 +08:00] [INFO] [apply.rs:1358] ["exec ConfChange"] [epoch="conf_ver: 25 version: 5404"] [type=RemoveNode] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:54.847 +08:00] [INFO] [apply.rs:1466] ["remove peer successfully"] [region="id: 173963 start_key: 7480000000000008FF675F72E000000000FF0000000000000000FA end_key: 7480000000000008FF675F72F000000000FFB45FB70000000000FA region_epoch { conf_ver: 25 version: 5404 } peers { id: 173964 store_id: 14 } peers { id: 173966 store_id: 9 } peers { id: 242112 store_id: 4 } peers { id: 242619 store_id: 6 }"] [peer="id: 173964 store_id: 14"] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:54.849 +08:00] [INFO] [apply.rs:1034] ["execute admin command"] [command="cmd_type: ChangePeer change_peer { change_type: AddLearnerNode peer { id: 242620 store_id: 10 is_learner: true } }"] [index=219283] [term=8] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:54.849 +08:00] [INFO] [apply.rs:1358] ["exec ConfChange"] [epoch="conf_ver: 26 version: 5404"] [type=AddLearner] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:54.849 +08:00] [INFO] [apply.rs:1498] ["add learner successfully"] [region="id: 173963 start_key: 7480000000000008FF675F72E000000000FF0000000000000000FA end_key: 7480000000000008FF675F72F000000000FFB45FB70000000000FA region_epoch { conf_ver: 26 version: 5404 } peers { id: 173966 store_id: 9 } peers { id: 242112 store_id: 4 } peers { id: 242619 store_id: 6 }"] [peer="id: 242620 store_id: 10 is_learner: true"] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:58.855 +08:00] [INFO] [apply.rs:1034] ["execute admin command"] [command="cmd_type: ChangePeer change_peer { peer { id: 242620 store_id: 10 } }"] [index=219284] [term=8] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:58.855 +08:00] [INFO] [apply.rs:1358] ["exec ConfChange"] [epoch="conf_ver: 27 version: 5404"] [type=AddNode] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:58.855 +08:00] [INFO] [apply.rs:1413] ["add peer successfully"] [region="id: 173963 start_key: 7480000000000008FF675F72E000000000FF0000000000000000FA end_key: 7480000000000008FF675F72F000000000FFB45FB70000000000FA region_epoch { conf_ver: 27 version: 5404 } peers { id: 173966 store_id: 9 } peers { id: 242112 store_id: 4 } peers { id: 242619 store_id: 6 } peers { id: 242620 store_id: 10 is_learner: true }"] [peer="id: 242620 store_id: 10"] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:58.855 +08:00] [INFO] [apply.rs:1034] ["execute admin command"] [command="cmd_type: ChangePeer change_peer { change_type: RemoveNode peer { id: 242112 store_id: 4 } }"] [index=219285] [term=8] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:58.855 +08:00] [INFO] [apply.rs:1358] ["exec ConfChange"] [epoch="conf_ver: 28 version: 5404"] [type=RemoveNode] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:58.855 +08:00] [INFO] [apply.rs:1466] ["remove peer successfully"] [region="id: 173963 start_key: 7480000000000008FF675F72E000000000FF0000000000000000FA end_key: 7480000000000008FF675F72F000000000FFB45FB70000000000FA region_epoch { conf_ver: 28 version: 5404 } peers { id: 173966 store_id: 9 } peers { id: 242112 store_id: 4 } peers { id: 242619 store_id: 6 } peers { id: 242620 store_id: 10 }"] [peer="id: 242112 store_id: 4"] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:58.855 +08:00] [INFO] [peer.rs:1343] ["starts destroy"] [merged_by_target=false] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:58.855 +08:00] [INFO] [peer.rs:541] ["begin to destroy"] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:58.856 +08:00] [INFO] [peer_storage.rs:1325] ["finish clear peer meta"] [takes=487.678µs] [raft_logs=47] [raft_key=1] [apply_key=1] [meta_key=1] [region_id=173963]
[2020/02/10 09:52:58.856 +08:00] [INFO] [peer.rs:588] ["peer destroy itself"] [takes=981.389µs] [peer_id=242112] [region_id=173963]
[2020/02/10 09:52:58.856 +08:00] [INFO] [region.rs:451] ["register deleting data in range"] [end_key=7A7480000000000008FF675F72F000000000FFB45FB70000000000FA] [start_key=7A7480000000000008FF675F72E000000000FF0000000000000000FA] [region_id=173963]
[2020/02/10 09:52:59.186 +08:00] [INFO] [peer.rs:724] ["failed to schedule peer tick"] [err="sending on a disconnected channel"] [tick=RAFT] [peer_id=242112] [region_id=173963]
[2020/02/10 09:53:04.489 +08:00] [INFO] [peer.rs:724] ["failed to schedule peer tick"] [err="sending on a disconnected channel"] [tick=RAFT_LOG_GC] [peer_id=242112] [region_id=173963]
[2020/02/10 09:53:04.492 +08:00] [INFO] [peer.rs:724] ["failed to schedule peer tick"] [err="sending on a disconnected channel"] [tick=SPLIT_REGION_CHECK] [peer_id=242112] [region_id=173963]
[2020/02/10 09:53:41.774 +08:00] [INFO] [peer.rs:724] ["failed to schedule peer tick"] [err="sending on a disconnected channel"] [tick=PD_HEARTBEAT] [peer_id=242112] [region_id=173963]
[2020/02/10 09:56:41.356 +08:00] [INFO] [peer.rs:724] ["failed to schedule peer tick"] [err="sending on a disconnected channel"] [tick=CHECK_PEER_STALE_STATE] [peer_id=242112] [region_id=173963]
[2020/02/10 10:04:03.918 +08:00] [INFO] [region.rs:418] ["succeed in deleting data in range"] [end_key=7A7480000000000008FF675F72F000000000FFB45FB70000000000FA] [start_key=7A7480000000000008FF675F72E000000000FF0000000000000000FA] [region_id=173963]

zzzzzz · 2020 年2 月 10 日 04:36

该日志是 INFO 级别的日志。

通常发生在 close TiKV 或 remove peer 的时候

可以先查看下当前集群状态是否正常。

wl21787 · 2020 年2 月 10 日 05:24

集群状态一切正常。

zzzzzz · 2020 年2 月 10 日 06:07

嗯，可以关注下 tikv 的状态。

如果集群没有其他问题的话，且日志不再出现的， INFO 信息可以忽略。

system · 2022 年10 月 31 日 19:05

此话题已在最后回复的 1 分钟后被自动关闭。不再允许新回复。