down-peer-region-count和pending-peer-region-count很多如何排查处理

现在的情况是这样:
我们要缩容的那个节点2上现在只剩下副本,没有leader了,但现在副本迁不走。从pd上的调度日志看,是有在调度的,但是过了十分钟之后就超时了。节点2上的副本也没任何变化,然后前面我们新增的那个节点上,就只有leader的副本,没有非leader的副本,好像也不太正常??
具体日志如下(很多这样的日志):

[2021/12/23 09:24:34.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1385813] [step=“add learner peer 1629105 on store 1498967”] [source=“active push”]
[2021/12/23 09:24:34.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1476621] [step=“add learner peer 1629113 on store 1498967”] [source=“active push”]
[2021/12/23 09:24:34.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1118565] [step=“add learner peer 1629111 on store 1498967”] [source=“active push”]
[2021/12/23 09:24:34.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1415657] [step=“add learner peer 1629101 on store 1498967”] [source=“active push”]
[2021/12/23 09:24:34.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1322901] [step=“add learner peer 1629124 on store 1”] [source=“active push”]
[2021/12/23 09:24:34.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=948784] [step=“add learner peer 1629120 on store 1”] [source=“active push”]
[2021/12/23 09:24:34.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=240960] [step=“add learner peer 1629127 on store 1”] [source=“active push”]
[2021/12/23 09:24:34.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1418145] [step=“add learner peer 1629144 on store 1”] [source=“active push”]
[2021/12/23 09:24:34.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=112108] [step=“add learner peer 1629148 on store 1”] [source=“active push”]
[2021/12/23 09:24:35.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=244416] [step=“add learner peer 1629147 on store 1”] [source=“active push”]
[2021/12/23 09:24:37.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1088397] [step=“merge region 1418145 into region 1088397”] [source=“active push”]
[2021/12/23 09:24:37.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=5016] [step=“add learner peer 1629121 on store 1”] [source=“active push”]
[2021/12/23 09:24:37.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=5060] [step=“merge region 5016 into region 5060”] [source=“active push”]
[2021/12/23 09:24:37.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=18636] [step=“add learner peer 1629130 on store 1”] [source=“active push”]
[2021/12/23 09:24:37.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=107880] [step=“add learner peer 1629131 on store 1”] [source=“active push”]
[2021/12/23 09:24:37.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=108052] [step=“add learner peer 1629145 on store 1”] [source=“active push”]
[2021/12/23 09:24:37.742 +08:00] [INFO] [operator_controller.go:620] [“send schedule command”] [region-id=1388401] [step=“add learner peer 1629126 on store 1”] [source=“active push”]

2021/12/23 09:18:42.742 +08:00] [INFO] [operator_controller.go:560] [“operator timeout”] [region-id=64368] [takes=10m3.927152479s] [operator="“replace-rule-offline-peer {mv peer: store [2] to [1]} (kind:region,replica, region:64368(567,84), createAt:2021-12-23 09:08:38.8150239 +0800 CST m=+11055414.672533317, startAt:2021-12-23 09:08:38.815106696 +0800 CST m=+11055414.672616123, currentStep:0, steps:[add learner peer 1629084 on store 1, promote learner peer 1629084 on store 1 to voter, remove peer on store 2]) timeout”"]
[2021/12/23 09:18:47.742 +08:00] [INFO] [operator_controller.go:560] [“operator timeout”] [region-id=119016] [takes=10m4.920553184s] [operator="“replace-rule-offline-peer {mv peer: store [2] to [1]} (kind:region,replica, region:119016(1110,92), createAt:2021-12-23 09:08:42.821604911 +0800 CST m=+11055418.679114334, startAt:2021-12-23 09:08:42.821682328 +0800 CST m=+11055418.679191751, currentStep:0, steps:[add learner peer 1629085 on store 1, promote learner peer 1629085 on store 1 to voter, remove peer on store 2]) timeout”"]
[2021/12/23 09:18:53.242 +08:00] [INFO] [operator_controller.go:560] [“operator timeout”] [region-id=935716] [takes=10m3.412135396s] [operator="“replace-rule-offline-peer {mv peer: store [2] to [1]} (kind:region,replica, region:935716(25367,68), createAt:2021-12-23 09:08:49.829832062 +0800 CST m=+11055425.687341486, startAt:2021-12-23 09:08:49.829929786 +0800 CST m=+11055425.687439213, currentStep:0, steps:[add learner peer 1629086 on store 1, promote learner peer 1629086 on store 1 to voter, remove peer on store 2]) timeout”"]
[2021/12/23 09:18:58.242 +08:00] [INFO] [operator_controller.go:560] [“operator timeout”] [region-id=1546131] [takes=10m4.407551709s] [operator="“replace-rule-offline-peer {mv peer: store [2] to [1]} (kind:region,replica, region:1546131(33859,192), createAt:2021-12-23 09:08:53.834417272 +0800 CST m=+11055429.691926694, startAt:2021-12-23 09:08:53.834489681 +0800 CST m=+11055429.691999211, currentStep:0, steps:[add learner peer 1629087 on store 1, promote learner peer 1629087 on store 1 to voter, remove peer on store 2]) timeout”"]
[2021/12/23 09:19:01.242 +08:00] [INFO] [operator_controller.go:560] [“operator timeout”] [region-id=1487021] [takes=10m3.405494558s] [operator="“replace-rule-offline-peer {mv peer: store [2] to [1]} (kind:region,replica, region:1487021(40178,294), createAt:2021-12-23 09:08:57.836705036 +0800 CST m=+11055433.694214454, startAt:2021-12-23 09:08:57.83676788 +0800 CST m=+11055433.694277303, currentStep:0, steps:[add learner peer 1629088 on store 1, promote learner peer 1629088 on store 1 to voter, remove peer on store 2]) timeout”"]
[2021/12/23 09:19:08.742 +08:00] [INFO] [operator_controller.go:560] [“operator timeout”] [region-id=6656] [takes=10m2.194256487s] [operator="“replace-rule-offline-peer {mv peer: store [2] to [1]} (kind:region,replica, region:6656(327,360), createAt:2021-12-23 09:09:06.547814545 +0800 CST m=+11055442.405323967, startAt:2021-12-23 09:09:06.547900103 +0800 CST m=+11055442.405409527, currentStep:0, steps:[add learner peer 1629089 on store 1, promote learner peer 1629089 on store 1 to voter, remove peer on store 2]) timeout”"]
[2021/12/23 09:19:08.742 +08:00] [INFO] [operator_controller.go:560] [“operator timeout”] [region-id=399272] [takes=10m2.096587074s] [operator="“replace-rule-offline-peer {mv peer: store [2] to [1]} (kind:region,replica, region:399272(337,422), createAt:2021-12-23 09:09:06.645561614 +0800 CST m=+11055442.503071037, startAt:2021-12-23 09:09:06.645625423 +0800 CST m=+11055442.503134848, currentStep:0, steps:[add learner peer 1629090 on store 1, promote learner peer 1629090 on store 1 to voter, remove peer on store 2]) timeout”"]
[2021/12/23 09:19:14.242 +08:00] [INFO] [operator_controller.go:560] [“operator timeout”] [region-id=15684] [takes=10m3.692422571s] [operator="“replace-rule-offline-peer {mv peer: store [2] to [1]} (kind:region,replica, region:15684(412,124), createAt:2021-12-23 09:09:10.549561564 +0800 CST m=+11055446.407070984, startAt:2021-12-23 09:09:10.549608236 +0800 CST m=+11055446.407117658, currentStep:0, steps:[add learner peer 1629091 on store 1, promote learner peer 1629091 on store 1 to voter, remove peer on store 2]) timeout”"]
[2021/12/23 09:19:16.742 +08:00] [INFO] [operator_controller.go:560] [“operator timeout”] [region-id=112176] [takes=10m2.187618002s] [operator="“replace-rule-offline-peer {mv peer: store [2] to [1]} (kind:region,replica, region:112176(1055,180), createAt:2021-12-23 09:09:14.55445867 +0800 CST m=+11055450.411968090, startAt:2021-12-23 09:09:14.554547477 +0800 CST m=+11055450.412056904, currentStep:0, steps:[add learner peer 1629092 on store 1, promote learner peer 1629092 on store 1 to voter, remove peer on store 2]) timeout”"]