down-peer-region-count和pending-peer-region-count很多如何排查处理

[2021/12/23 09:18:57.465 +08:00] [INFO] [operator_controller.go:424] [“add operator”] [region-id=1322901] [operator="“replace-rule-offline-peer {mv peer: store [2] to [1]} (kind:region,replica, region:1322901(25531,82), createAt:2021-12-23 09:18:57.465084102 +0800 CST m=+11056033.322593522, startAt:0001-01-01 00:00:00 +0000 UTC, currentStep:0, steps:[add learner peer 1629124 on store 1, promote learner peer 1629124 on store 1 to voter, remove peer on store 2])”"] [“additional info”=]
[2021/12/23 09:18:57.465 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=create]
[2021/12/23 09:18:57.466 +08:00] [INFO] [cluster.go:551] [“region ConfVer changed”] [region-id=1322901] [detail=“Add peer:{id:1629124 store_id:1 is_learner:true }”] [old-confver=82] [new-confver=83]
[2021/12/23 09:18:57.466 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=heartbeat]
[2021/12/23 09:19:02.741 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:19:07.741 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:19:12.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:19:18.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:19:23.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:19:28.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:19:28.744 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=heartbeat]
[2021/12/23 09:19:33.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:19:38.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:19:43.243 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:19:48.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:19:54.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:19:59.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:20:05.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:20:10.743 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:20:16.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:20:21.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:20:26.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:20:28.743 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=heartbeat]
[2021/12/23 09:20:32.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:20:37.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:20:42.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:20:48.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:20:53.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:20:58.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:21:03.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:21:09.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:21:14.743 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:21:20.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:21:25.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:21:28.750 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=heartbeat]
[2021/12/23 09:21:30.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:21:36.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:21:41.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:21:46.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:21:51.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:21:57.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:22:02.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:22:07.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:22:12.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:22:17.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:22:23.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:22:28.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:22:28.754 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=heartbeat]
[2021/12/23 09:22:33.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:22:39.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:22:44.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:22:50.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:22:55.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:23:00.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:23:05.747 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:23:11.244 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:23:16.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:23:21.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:23:27.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:23:28.753 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=heartbeat]
[2021/12/23 09:23:32.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:23:37.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:23:42.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:23:47.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:23:53.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:23:58.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:24:03.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:24:08.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:24:14.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:24:19.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:24:24.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:24:28.761 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=heartbeat]
[2021/12/23 09:24:29.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:24:34.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:24:40.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:24:45.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:24:51.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:24:56.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:25:02.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:25:07.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:25:12.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:25:18.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:25:23.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:25:28.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:25:28.759 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=heartbeat]
[2021/12/23 09:25:33.743 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:25:38.743 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:25:44.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:25:49.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:25:55.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:26:00.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:26:06.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:26:11.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:26:17.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:26:22.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:26:28.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:26:28.758 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=heartbeat]
[2021/12/23 09:26:33.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:26:39.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:26:44.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:26:50.241 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:26:55.741 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:27:00.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:27:06.241 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:27:11.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:27:16.741 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:27:21.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:27:27.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:27:28.760 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=heartbeat]
[2021/12/23 09:27:32.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:27:38.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:27:43.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:27:48.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:27:54.243 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:27:59.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:28:05.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:28:10.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:28:15.741 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:28:20.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:28:25.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:28:28.768 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=heartbeat]
[2021/12/23 09:28:31.241 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:28:36.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:28:41.741 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:28:47.242 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:28:52.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:28:58.241 +08:00] [INFO] [operator_controller.go:560] [“operator timeout”] [region-id=1322901] [takes=10m0.776768253s] [operator="“replace-rule-offline-peer {mv peer: store [2] to [1]} (kind:region,replica, region:1322901(25531,82), createAt:2021-12-23 09:18:57.465084102 +0800 CST m=+11056033.322593522, startAt:2021-12-23 09:18:57.465212602 +0800 CST m=+11056033.322722028, currentStep:0, steps:[add learner peer 1629124 on store 1, promote learner peer 1629124 on store 1 to voter, remove peer on store 2]) timeout”"]
[2021/12/23 09:29:05.128 +08:00] [INFO] [operator_controller.go:424] [“add operator”] [region-id=1322901] [operator="“remove-orphan-peer {rm peer: store [1]} (kind:region, region:1322901(25531,83), createAt:2021-12-23 09:29:05.12860822 +0800 CST m=+11056640.986117639, startAt:0001-01-01 00:00:00 +0000 UTC, currentStep:0, steps:[remove peer on store 1])”"] [“additional info”=]
[2021/12/23 09:29:05.128 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“remove peer on store 1”] [source=create]
[2021/12/23 09:29:05.130 +08:00] [INFO] [cluster.go:551] [“region ConfVer changed”] [region-id=1322901] [detail=“Remove peer:{id:1629124 store_id:1 is_learner:true }”] [old-confver=83] [new-confver=84]
[2021/12/23 09:29:05.130 +08:00] [INFO] [operator_controller.go:537] [“operator finish”] [region-id=1322901] [takes=1.960798ms] [operator="“remove-orphan-peer {rm peer: store [1]} (kind:region, region:1322901(25531,83), createAt:2021-12-23 09:29:05.12860822 +0800 CST m=+11056640.986117639, startAt:2021-12-23 09:29:05.128700448 +0800 CST m=+11056640.986209869, currentStep:1, steps:[remove peer on store 1]) finished”"] [“additional info”=]

这是一个peer完整的日志,十分钟之后超时了

这个就是它想从节点2上调度一个peer到节点1上,它先在节点1上生成了一个peer是pending状态,是learner,然后过了十分钟就报超时, 然后又从1上把这个新生成的pending状态的learner的peer给删了。