启动tidb 集群时,某台tikv启动超时

为提高效率,请提供以下信息,问题描述清晰能够更快得到解决:

【TiDB 版本】
v4.0.0

Cluster type: tidb
Cluster name: hxgk-tidb
Cluster version: v4.0.0
SSH type: builtin

【问题描述】
[tidb@TiUP ~]$ tiup cluster start hxgk-tidb
Starting component cluster: /home/tidb/.tiup/components/cluster/v1.2.5/tiup-cluster start hxgk-tidb
Starting cluster hxgk-tidb…

  • [ Serial ] - SSHKeySet: privateKey=/home/tidb/.tiup/storage/cluster/clusters/hxgk-tidb/ssh/id_rsa, publicKey=/home/tidb/.tiup/storage/cluster/clusters/hxgk-tidb/ssh/id_rsa.pub
  • [Parallel] - UserSSH: user=tidb, host=172.16.21.162
  • [Parallel] - UserSSH: user=tidb, host=172.16.21.158
  • [Parallel] - UserSSH: user=tidb, host=172.16.21.156
  • [Parallel] - UserSSH: user=tidb, host=172.16.21.161
  • [Parallel] - UserSSH: user=tidb, host=172.16.21.162
  • [Parallel] - UserSSH: user=tidb, host=172.16.21.159
  • [Parallel] - UserSSH: user=tidb, host=172.16.21.157
  • [Parallel] - UserSSH: user=tidb, host=172.16.21.159
  • [Parallel] - UserSSH: user=tidb, host=172.16.21.160
  • [Parallel] - UserSSH: user=tidb, host=172.16.21.162
  • [Parallel] - UserSSH: user=tidb, host=172.16.21.160
  • [Parallel] - UserSSH: user=tidb, host=172.16.21.161
  • [Parallel] - UserSSH: user=tidb, host=172.16.21.159
  • [ Serial ] - StartCluster
    Starting component pd
    Starting instance pd 172.16.21.161:2379
    Starting instance pd 172.16.21.160:2379
    Starting instance pd 172.16.21.159:2379
    Start pd 172.16.21.161:2379 success
    Start pd 172.16.21.159:2379 success
    Start pd 172.16.21.160:2379 success
    Starting component node_exporter
    Starting instance 172.16.21.159
    Start 172.16.21.159 success
    Starting component blackbox_exporter
    Starting instance 172.16.21.159
    Start 172.16.21.159 success
    Starting component node_exporter
    Starting instance 172.16.21.160
    Start 172.16.21.160 success
    Starting component blackbox_exporter
    Starting instance 172.16.21.160
    Start 172.16.21.160 success
    Starting component node_exporter
    Starting instance 172.16.21.161
    Start 172.16.21.161 success
    Starting component blackbox_exporter
    Starting instance 172.16.21.161
    Start 172.16.21.161 success
    Starting component tikv
    Starting instance tikv 172.16.21.158:20160
    Starting instance tikv 172.16.21.156:20160
    Starting instance tikv 172.16.21.157:20160
    Start tikv 172.16.21.157:20160 success
    Start tikv 172.16.21.156:20160 success
    retry error: operation timed out after 2m0s
    tikv 172.16.21.158:20160 failed to start: timed out waiting for port 20160 to be started after 2m0s, please check the log of the instance

Error: failed to start tikv: tikv 172.16.21.158:20160 failed to start: timed out waiting for port 20160 to be started after 2m0s, please check the log of the instance: timed out waiting for port 20160 to be started after 2m0s

Verbose debug logs has been written to /home/tidb/logs/tiup-cluster-debug-2021-05-18-10-24-41.log.
Error: run /home/tidb/.tiup/components/cluster/v1.2.5/tiup-cluster (wd:/home/tidb/.tiup/data/SXj2fiQ) failed: exit status 1
[tidb@TiUP ~]$


若提问为性能优化、故障排查类问题,请下载脚本运行。终端输出的打印结果,请务必全选并复制粘贴上传。

麻烦拿一下172.16.21.158:20160的错误日志

[2021/05/18 11:46:32.738 +08:00] [INFO] [raft.rs:285] [newRaft] [peers="[(76036, Progress { matched: 1029, next_idx: 1030, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76034, Progress { matched: 0, next_idx: 1030, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76035, Progress { matched: 0, next_idx: 1030, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } })]"] [“last term”=1012] [“last index”=1029] [applied=1029] [commit=1029] [term=1012] [raft_id=76036] [region_id=76033]
[2021/05/18 11:46:32.738 +08:00] [INFO] [raw_node.rs:222] [“RawNode created with id 76036.”] [id=76036] [raft_id=76036] [region_id=76033]
[2021/05/18 11:46:32.738 +08:00] [INFO] [peer.rs:147] [“create peer”] [peer_id=76040] [region_id=76037]
[2021/05/18 11:46:32.738 +08:00] [INFO] [raft.rs:783] [“became follower at term 12458”] [term=12458] [raft_id=76040] [region_id=76037]
[2021/05/18 11:46:32.738 +08:00] [INFO] [raft.rs:285] [newRaft] [peers="[(76040, Progress { matched: 97196641, next_idx: 97196642, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76038, Progress { matched: 0, next_idx: 97196642, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76039, Progress { matched: 0, next_idx: 97196642, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } })]"] [“last term”=12458] [“last index”=97196641] [applied=97196219] [commit=97196640] [term=12458] [raft_id=76040] [region_id=76037]
[2021/05/18 11:46:32.738 +08:00] [INFO] [raw_node.rs:222] [“RawNode created with id 76040.”] [id=76040] [raft_id=76040] [region_id=76037]
[2021/05/18 11:46:32.738 +08:00] [INFO] [peer.rs:147] [“create peer”] [peer_id=76044] [region_id=76041]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raft.rs:783] [“became follower at term 1213”] [term=1213] [raft_id=76044] [region_id=76041]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raft.rs:285] [newRaft] [peers="[(76044, Progress { matched: 9275275, next_idx: 9275276, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76042, Progress { matched: 0, next_idx: 9275276, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76043, Progress { matched: 0, next_idx: 9275276, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } })]"] [“last term”=1213] [“last index”=9275275] [applied=9275202] [commit=9275275] [term=1213] [raft_id=76044] [region_id=76041]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raw_node.rs:222] [“RawNode created with id 76044.”] [id=76044] [raft_id=76044] [region_id=76041]
[2021/05/18 11:46:32.739 +08:00] [INFO] [peer.rs:147] [“create peer”] [peer_id=76048] [region_id=76045]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raft.rs:783] [“became follower at term 966”] [term=966] [raft_id=76048] [region_id=76045]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raft.rs:285] [newRaft] [peers="[(76048, Progress { matched: 88856, next_idx: 88857, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76046, Progress { matched: 0, next_idx: 88857, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76047, Progress { matched: 0, next_idx: 88857, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } })]"] [“last term”=966] [“last index”=88856] [applied=88856] [commit=88856] [term=966] [raft_id=76048] [region_id=76045]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raw_node.rs:222] [“RawNode created with id 76048.”] [id=76048] [raft_id=76048] [region_id=76045]
[2021/05/18 11:46:32.739 +08:00] [INFO] [peer.rs:147] [“create peer”] [peer_id=76052] [region_id=76049]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raft.rs:783] [“became follower at term 856”] [term=856] [raft_id=76052] [region_id=76049]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raft.rs:285] [newRaft] [peers="[(76052, Progress { matched: 869, next_idx: 870, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76050, Progress { matched: 0, next_idx: 870, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76051, Progress { matched: 0, next_idx: 870, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } })]"] [“last term”=856] [“last index”=869] [applied=869] [commit=869] [term=856] [raft_id=76052] [region_id=76049]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raw_node.rs:222] [“RawNode created with id 76052.”] [id=76052] [raft_id=76052] [region_id=76049]
[2021/05/18 11:46:32.739 +08:00] [INFO] [peer.rs:147] [“create peer”] [peer_id=76056] [region_id=76053]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raft.rs:783] [“became follower at term 942”] [term=942] [raft_id=76056] [region_id=76053]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raft.rs:285] [newRaft] [peers="[(76056, Progress { matched: 957, next_idx: 958, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76054, Progress { matched: 0, next_idx: 958, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76055, Progress { matched: 0, next_idx: 958, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } })]"] [“last term”=942] [“last index”=957] [applied=957] [commit=957] [term=942] [raft_id=76056] [region_id=76053]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raw_node.rs:222] [“RawNode created with id 76056.”] [id=76056] [raft_id=76056] [region_id=76053]
[2021/05/18 11:46:32.739 +08:00] [INFO] [peer.rs:147] [“create peer”] [peer_id=76060] [region_id=76057]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raft.rs:783] [“became follower at term 982”] [term=982] [raft_id=76060] [region_id=76057]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raft.rs:285] [newRaft] [peers="[(76060, Progress { matched: 52438502, next_idx: 52438503, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76058, Progress { matched: 0, next_idx: 52438503, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76059, Progress { matched: 0, next_idx: 52438503, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } })]"] [“last term”=982] [“last index”=52438502] [applied=52438412] [commit=52438502] [term=982] [raft_id=76060] [region_id=76057]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raw_node.rs:222] [“RawNode created with id 76060.”] [id=76060] [raft_id=76060] [region_id=76057]
[2021/05/18 11:46:32.739 +08:00] [INFO] [peer.rs:147] [“create peer”] [peer_id=76064] [region_id=76061]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raft.rs:783] [“became follower at term 867”] [term=867] [raft_id=76064] [region_id=76061]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raft.rs:285] [newRaft] [peers="[(76064, Progress { matched: 893, next_idx: 894, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76062, Progress { matched: 0, next_idx: 894, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76063, Progress { matched: 0, next_idx: 894, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } })]"] [“last term”=867] [“last index”=893] [applied=893] [commit=893] [term=867] [raft_id=76064] [region_id=76061]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raw_node.rs:222] [“RawNode created with id 76064.”] [id=76064] [raft_id=76064] [region_id=76061]
[2021/05/18 11:46:32.739 +08:00] [INFO] [peer.rs:147] [“create peer”] [peer_id=76068] [region_id=76065]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raft.rs:783] [“became follower at term 941”] [term=941] [raft_id=76068] [region_id=76065]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raft.rs:285] [newRaft] [peers="[(76068, Progress { matched: 983, next_idx: 984, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76066, Progress { matched: 0, next_idx: 984, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76067, Progress { matched: 0, next_idx: 984, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } })]"] [“last term”=941] [“last index”=983] [applied=983] [commit=983] [term=941] [raft_id=76068] [region_id=76065]
[2021/05/18 11:46:32.739 +08:00] [INFO] [raw_node.rs:222] [“RawNode created with id 76068.”] [id=76068] [raft_id=76068] [region_id=76065]
[2021/05/18 11:46:32.739 +08:00] [INFO] [peer.rs:147] [“create peer”] [peer_id=76072] [region_id=76069]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raft.rs:783] [“became follower at term 923”] [term=923] [raft_id=76072] [region_id=76069]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raft.rs:285] [newRaft] [peers="[(76072, Progress { matched: 934, next_idx: 935, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76070, Progress { matched: 0, next_idx: 935, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76071, Progress { matched: 0, next_idx: 935, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } })]"] [“last term”=923] [“last index”=934] [applied=934] [commit=934] [term=923] [raft_id=76072] [region_id=76069]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raw_node.rs:222] [“RawNode created with id 76072.”] [id=76072] [raft_id=76072] [region_id=76069]
[2021/05/18 11:46:32.740 +08:00] [INFO] [peer.rs:147] [“create peer”] [peer_id=76076] [region_id=76073]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raft.rs:783] [“became follower at term 977”] [term=977] [raft_id=76076] [region_id=76073]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raft.rs:285] [newRaft] [peers="[(76076, Progress { matched: 996, next_idx: 997, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76074, Progress { matched: 0, next_idx: 997, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76075, Progress { matched: 0, next_idx: 997, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } })]"] [“last term”=977] [“last index”=996] [applied=996] [commit=996] [term=977] [raft_id=76076] [region_id=76073]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raw_node.rs:222] [“RawNode created with id 76076.”] [id=76076] [raft_id=76076] [region_id=76073]
[2021/05/18 11:46:32.740 +08:00] [INFO] [peer.rs:147] [“create peer”] [peer_id=76080] [region_id=76077]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raft.rs:783] [“became follower at term 958”] [term=958] [raft_id=76080] [region_id=76077]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raft.rs:285] [newRaft] [peers="[(76080, Progress { matched: 974, next_idx: 975, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76078, Progress { matched: 0, next_idx: 975, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76079, Progress { matched: 0, next_idx: 975, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } })]"] [“last term”=958] [“last index”=974] [applied=974] [commit=974] [term=958] [raft_id=76080] [region_id=76077]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raw_node.rs:222] [“RawNode created with id 76080.”] [id=76080] [raft_id=76080] [region_id=76077]
[2021/05/18 11:46:32.740 +08:00] [INFO] [peer.rs:147] [“create peer”] [peer_id=76084] [region_id=76081]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raft.rs:783] [“became follower at term 947”] [term=947] [raft_id=76084] [region_id=76081]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raft.rs:285] [newRaft] [peers="[(76084, Progress { matched: 960, next_idx: 961, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76082, Progress { matched: 0, next_idx: 961, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76083, Progress { matched: 0, next_idx: 961, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } })]"] [“last term”=947] [“last index”=960] [applied=960] [commit=960] [term=947] [raft_id=76084] [region_id=76081]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raw_node.rs:222] [“RawNode created with id 76084.”] [id=76084] [raft_id=76084] [region_id=76081]
[2021/05/18 11:46:32.740 +08:00] [INFO] [peer.rs:147] [“create peer”] [peer_id=76088] [region_id=76085]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raft.rs:783] [“became follower at term 922”] [term=922] [raft_id=76088] [region_id=76085]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raft.rs:285] [newRaft] [peers="[(76088, Progress { matched: 936, next_idx: 937, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76086, Progress { matched: 0, next_idx: 937, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76087, Progress { matched: 0, next_idx: 937, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } })]"] [“last term”=922] [“last index”=936] [applied=936] [commit=936] [term=922] [raft_id=76088] [region_id=76085]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raw_node.rs:222] [“RawNode created with id 76088.”] [id=76088] [raft_id=76088] [region_id=76085]
[2021/05/18 11:46:32.740 +08:00] [INFO] [peer.rs:147] [“create peer”] [peer_id=76092] [region_id=76089]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raft.rs:783] [“became follower at term 890”] [term=890] [raft_id=76092] [region_id=76089]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raft.rs:285] [newRaft] [peers="[(76092, Progress { matched: 903, next_idx: 904, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76090, Progress { matched: 0, next_idx: 904, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76091, Progress { matched: 0, next_idx: 904, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } })]"] [“last term”=890] [“last index”=903] [applied=903] [commit=903] [term=890] [raft_id=76092] [region_id=76089]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raw_node.rs:222] [“RawNode created with id 76092.”] [id=76092] [raft_id=76092] [region_id=76089]
[2021/05/18 11:46:32.740 +08:00] [INFO] [peer.rs:147] [“create peer”] [peer_id=76096] [region_id=76093]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raft.rs:783] [“became follower at term 1124”] [term=1124] [raft_id=76096] [region_id=76093]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raft.rs:285] [newRaft] [peers="[(76096, Progress { matched: 8074508, next_idx: 8074509, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76094, Progress { matched: 0, next_idx: 8074509, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } }), (76095, Progress { matched: 0, next_idx: 8074509, state: Probe, paused: false, pending_snapshot: 0, pending_request_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } })]"] [“last term”=1124] [“last index”=8074508] [applied=8074454] [commit=8074508] [term=1124] [raft_id=76096] [region_id=76093]
[2021/05/18 11:46:32.740 +08:00] [INFO] [raw_node.rs:222] [“RawNode created with id 76096.”] [id=76096] [raft_id=76096] [region_id=76093]
[2021/05/18 11:46:32.740 +08:00] [INFO] [peer.rs:147] [“create peer”] [peer_id=76100] [region_id=76097]
[2021/05/18 11:46:32.780 +08:00] [FATAL] [server.rs:576] [“failed to start node: EngineTraits(Other(”[components/raftstore/src/store/fsm/store.rs:812]: \"[components/raftstore/src/store/peer_storage.rs:385]: [region 76097] entry at apply index 357578346 doesn\\\'t exist, may lose data.\""))"]
[2021/05/18 11:46:48.069 +08:00] [INFO] [lib.rs:93] [“Welcome to TiKV”]
[2021/05/18 11:46:48.070 +08:00] [INFO] [lib.rs:95] []
[2021/05/18 11:46:48.070 +08:00] [INFO] [lib.rs:95] [“Release Version: 4.0.0”]
[2021/05/18 11:46:48.070 +08:00] [INFO] [lib.rs:95] [“Edition: Community”]
[2021/05/18 11:46:48.070 +08:00] [INFO] [lib.rs:95] [“Git Commit Hash: 198a2cea01734ce8f46d55a29708f123f9133944”]
[2021/05/18 11:46:48.070 +08:00] [INFO] [lib.rs:95] [“Git Commit Branch: heads/refs/tags/v4.0.0”]
[2021/05/18 11:46:48.070 +08:00] [INFO] [lib.rs:95] [“UTC Build Time: 2020-05-27 12:09:22”]
[2021/05/18 11:46:48.070 +08:00] [INFO] [lib.rs:95] [“Rust Version: rustc 1.42.0-nightly (0de96d37f 2019-12-19)”]
[2021/05/18 11:46:48.070 +08:00] [INFO] [lib.rs:95] [“Enable Features: jemalloc portable sse protobuf-codec”]
[2021/05/18 11:46:48.070 +08:00] [INFO] [lib.rs:95] [“Profile: dist_release”]
[2021/05/18 11:46:48.080 +08:00] [INFO] [mod.rs:48] [“memory limit in bytes: 33529864192, cpu cores quota: 24”]
[2021/05/18 11:46:48.081 +08:00] [WARN] [lib.rs:528] [“environment variable TZ is missing, using /etc/localtime“]
[2021/05/18 11:46:48.081 +08:00] [WARN] [server.rs:826] [“check: kernel”] [err=“kernel parameters net.core.somaxconn got 128, expect 32768”]
[2021/05/18 11:46:48.081 +08:00] [WARN] [server.rs:826] [“check: kernel”] [err=“kernel parameters net.ipv4.tcp_syncookies got 1, expect 0”]
[2021/05/18 11:46:48.081 +08:00] [WARN] [server.rs:826] [“check: kernel”] [err=“kernel parameters vm.swappiness got 60, expect 0”]
[2021/05/18 11:46:48.082 +08:00] [INFO] [util.rs:398] [“connecting to PD endpoint”] [endpoints=172.16.21.159:2379]
[2021/05/18 11:46:48.082 +08:00] [INFO] [] [“New connected subchannel at 0x7f02c7a3a180 for subchannel 0x7f02d5c19a00”]
[2021/05/18 11:46:48.084 +08:00] [INFO] [util.rs:398] [“connecting to PD endpoint”] [endpoints=172.16.21.160:2379]
[2021/05/18 11:46:48.085 +08:00] [INFO] [] [“New connected subchannel at 0x7f02c7a3a240 for subchannel 0x7f02d5c19a00”]
[2021/05/18 11:46:48.086 +08:00] [INFO] [util.rs:398] [“connecting to PD endpoint”] [endpoints=172.16.21.161:2379]
[2021/05/18 11:46:48.086 +08:00] [INFO] [] [“New connected subchannel at 0x7f02c7a3a300 for subchannel 0x7f02d5c19a00”]
[2021/05/18 11:46:48.087 +08:00] [INFO] [util.rs:398] [“connecting to PD endpoint”] [endpoints=http://172.16.21.159:2379]
[2021/05/18 11:46:48.088 +08:00] [INFO] [] [“New connected subchannel at 0x7f02c7a3a3c0 for subchannel 0x7f02d5c19a00”]
[2021/05/18 11:46:48.089 +08:00] [INFO] [util.rs:398] [“connecting to PD endpoint”] [endpoints=http://172.16.21.161:2379]
[2021/05/18 11:46:48.090 +08:00] [INFO] [] [“New connected subchannel at 0x7f02c7a3a480 for subchannel 0x7f02d5c19a00”]
[2021/05/18 11:46:48.091 +08:00] [INFO] [util.rs:457] [“connected to PD leader”] [endpoints=http://172.16.21.161:2379]
[2021/05/18 11:46:48.091 +08:00] [INFO] [util.rs:386] [“all PD endpoints are consistent”] [endpoints=”[“172.16.21.159:2379”, “172.16.21.160:2379”, “172.16.21.161:2379”]”]
[2021/05/18 11:46:48.091 +08:00] [INFO] [server.rs:242] [“connect to PD cluster”] [cluster_id=6904056019054133769]
[2021/05/18 11:46:48.255 +08:00] [INFO] [server.rs:834] [“beginning system configuration check”]
[2021/05/18 11:46:48.256 +08:00] [INFO] [config.rs:709] [“data dir”] [mount_fs=“FsInfo { tp: “xfs”, opts: “rw,relatime,attr2,inode64,noquota”, mnt_dir: “/”, fsname: “/dev/mapper/centos-root” }”] [data_path=/tidb-data/tikv-20160]
[2021/05/18 11:46:48.256 +08:00] [INFO] [config.rs:709] [“data dir”] [mount_fs=“FsInfo { tp: “xfs”, opts: “rw,relatime,attr2,inode64,noquota”, mnt_dir: “/”, fsname: “/dev/mapper/centos-root” }”] [data_path=/tidb-data/tikv-20160/raft]
[2021/05/18 11:46:48.256 +08:00] [INFO] [server.rs:212] [“using config”] [config="{“cfg-path”:“conf/tikv.toml”,“log-level”:“info”,“log-file”:"/tidb-deploy/tikv-20160/log/tikv.log",“slow-log-file”:"",“slow-log-threshold”:“1s”,“log-rotation-timespan”:“1d”,“log-rotation-size”:“300MiB”,“panic-when-unexpected-key-or-data”:false,“readpool”:{“unified”:{“min-thread-count”:1,“max-thread-count”:19,“stack-size”:“10MiB”,“max-tasks-per-worker”:2000},“storage”:{“use-unified-pool”:null,“high-concurrency”:8,“normal-concurrency”:8,“low-concurrency”:8,“max-tasks-per-worker-high”:2000,“max-tasks-per-worker-normal”:2000,“max-tasks-per-worker-low”:2000,“stack-size”:“10MiB”},“coprocessor”:{“use-unified-pool”:null,“high-concurrency”:19,“normal-concurrency”:19,“low-concurrency”:19,“max-tasks-per-worker-high”:2000,“max-tasks-per-worker-normal”:2000,“max-tasks-per-worker-low”:2000,“stack-size”:“10MiB”}},“server”:{“addr”:“0.0.0.0:20160”,“advertise-addr”:“172.16.21.158:20160”,“status-addr”:“0.0.0.0:20180”,“status-thread-pool-size”:1,“max-grpc-send-msg-len”:10485760,“grpc-compression-type”:“none”,“grpc-concurrency”:4,“grpc-concurrent-stream”:1024,“grpc-raft-conn-num”:1,“grpc-memory-pool-quota”:9223372036854775807,“grpc-stream-initial-window-size”:“2MiB”,“grpc-keepalive-time”:“10s”,“grpc-keepalive-timeout”:“3s”,“concurrent-send-snap-limit”:32,“concurrent-recv-snap-limit”:32,“end-point-recursion-limit”:1000,“end-point-stream-channel-size”:8,“end-point-batch-row-limit”:64,“end-point-stream-batch-row-limit”:128,“end-point-enable-batch-if-possible”:true,“end-point-request-max-handle-duration”:“1m”,“end-point-max-concurrency”:24,“snap-max-write-bytes-per-sec”:“100MiB”,“snap-max-total-size”:“0KiB”,“stats-concurrency”:1,“heavy-load-threshold”:300,“heavy-load-wait-duration”:“1ms”,“enable-request-batch”:true,“request-batch-enable-cross-command”:false,“request-batch-wait-duration”:“1ms”,“labels”:{}},“storage”:{“data-dir”:"/tidb-data/tikv-20160",“gc-ratio-threshold”:1.1,“max-key-size”:4096,“scheduler-concurrency”:524288,“scheduler-worker-pool-size”:8,“scheduler-pending-write-threshold”:“100MiB”,“reserve-space”:“2GiB”,“block-cache”:{“shared”:true,“capacity”:“14068MiB”,“num-shard-bits”:6,“strict-capacity-limit”:false,“high-pri-pool-ratio”:0.8,“memory-allocator”:“nodump”}},“pd”:{“endpoints”:[“172.16.21.159:2379”,“172.16.21.160:2379”,“172.16.21.161:2379”],“retry-interval”:“300ms”,“retry-max-count”:9223372036854775807,“retry-log-every”:10},“metric”:{“interval”:“15s”,“address”:"",“job”:“tikv”},“raftstore”:{“sync-log”:true,“prevote”:true,“raftdb-path”:"/tidb-data/tikv-20160/raft",“capacity”:“0KiB”,“raft-base-tick-interval”:“1s”,“raft-heartbeat-ticks”:2,“raft-election-timeout-ticks”:10,“raft-min-election-timeout-ticks”:10,“raft-max-election-timeout-ticks”:20,“raft-max-size-per-msg”:“1MiB”,“raft-max-inflight-msgs”:256,“raft-entry-max-size”:“8MiB”,“raft-log-gc-tick-interval”:“10s”,“raft-log-gc-threshold”:50,“raft-log-gc-count-limit”:73728,“raft-log-gc-size-limit”:“72MiB”,“raft-entry-cache-life-time”:“30s”,“raft-reject-transfer-leader-duration”:“3s”,“split-region-check-tick-interval”:“10s”,“region-split-check-diff”:“6MiB”,“region-compact-check-interval”:“5m”,“clean-stale-peer-delay”:“11m”,“region-compact-check-step”:100,“region-compact-min-tombstones”:10000,“region-compact-tombstones-percent”:30,“pd-heartbeat-tick-interval”:“1m”,“pd-store-heartbeat-tick-interval”:“10s”,“snap-mgr-gc-tick-interval”:“1m”,“snap-gc-timeout”:“4h”,“lock-cf-compact-interval”:“10m”,“lock-cf-compact-bytes-threshold”:“256MiB”,“notify-capacity”:40960,“messages-per-tick”:4096,“max-peer-down-duration”:“5m”,“max-leader-missing-duration”:“2h”,“abnormal-leader-missing-duration”:“10m”,“peer-stale-state-check-interval”:“5m”,“leader-transfer-max-log-lag”:10,“snap-apply-batch-size”:“10MiB”,“consistency-check-interval”:“0s”,“report-region-flow-interval”:“1m”,“raft-store-max-leader-lease”:“9s”,“right-derive-when-split”:true,“allow-remove-leader”:false,“merge-max-log-gap”:10,“merge-check-tick-interval”:“10s”,“use-delete-range”:false,“cleanup-import-sst-interval”:“10m”,“local-read-batch-size”:1024,“apply-max-batch-size”:256,“apply-pool-size”:2,“apply-reschedule-duration”:“5s”,“store-max-batch-size”:256,“store-pool-size”:2,“store-reschedule-duration”:“5s”,“future-poll-size”:1,“hibernate-regions”:false,“early-apply”:true,“apply-yield-duration”:“500ms”},“coprocessor”:{“split-region-on-table”:false,“batch-split-limit”:10,“region-max-size”:“144MiB”,“region-split-size”:“96MiB”,“region-max-keys”:1440000,“region-split-keys”:960000},“rocksdb”:{“wal-recovery-mode”:2,“wal-dir”:"",“wal-ttl-seconds”:0,“wal-size-limit”:“0KiB”,“max-total-wal-size”:“4GiB”,“max-background-jobs”:8,“max-manifest-file-size”:“128MiB”,“create-if-missing”:true,“max-open-files”:40960,“enable-statistics”:true,“stats-dump-period”:“10m”,“compaction-readahead-size”:“0KiB”,“info-log-max-size”:“1GiB”,“info-log-roll-time”:“0s”,“info-log-keep-log-file-num”:10,“info-log-dir”:"",“rate-bytes-per-sec”:“0KiB”,“rate-limiter-mode”:2,“auto-tuned”:false,“bytes-per-sync”:“1MiB”,“wal-bytes-per-sync”:“512KiB”,“max-sub-compactions”:3,“writable-file-max-buffer-size”:“1MiB”,“use-direct-io-for-flush-and-compaction”:false,“enable-pipelined-write”:true,“enable-multi-batch-write”:true,“enable-unordered-write”:false,“defaultcf”:{“block-size”:“64KiB”,“block-cache-size”:“7994MiB”,“disable-block-cache”:false,“cache-index-and-filter-blocks”:true,“pin-l0-filter-and-index-blocks”:true,“use-bloom-filter”:true,“optimize-filters-for-hits”:true,“whole-key-filtering”:true,“bloom-filter-bits-per-key”:10,“block-based-bloom-filter”:false,“read-amp-bytes-per-bit”:0,“compression-per-level”:[“no”,“no”,“lz4”,“lz4”,“lz4”,“zstd”,“zstd”],“write-buffer-size”:“128MiB”,“max-write-buffer-number”:5,“min-write-buffer-number-to-merge”:1,“max-bytes-for-level-base”:“512MiB”,“target-file-size-base”:“8MiB”,“level0-file-num-compaction-trigger”:4,“level0-slowdown-writes-trigger”:20,“level0-stop-writes-trigger”:36,“max-compaction-bytes”:“2GiB”,“compaction-pri”:3,“dynamic-level-bytes”:true,“num-levels”:7,“max-bytes-for-level-multiplier”:10,“compaction-style”:0,“disable-auto-compactions”:false,“soft-pending-compaction-bytes-limit”:“64GiB”,“hard-pending-compaction-bytes-limit”:“256GiB”,“force-consistency-checks”:true,“prop-size-index-distance”:4194304,“prop-keys-index-distance”:40960,“enable-doubly-skiplist”:true,“titan”:{“min-blob-size”:“1KiB”,“blob-file-compression”:“lz4”,“blob-cache-size”:“0KiB”,“min-gc-batch-size”:“16MiB”,“max-gc-batch-size”:“64MiB”,“discardable-ratio”:0.5,“sample-ratio”:0.1,“merge-small-file-threshold”:“8MiB”,“blob-run-mode”:“normal”,“level-merge”:false,“range-merge”:true,“max-sorted-runs”:20,“gc-merge-rewrite”:false}},“writecf”:{“block-size”:“64KiB”,“block-cache-size”:“4796MiB”,“disable-block-cache”:false,“cache-index-and-filter-blocks”:true,“pin-l0-filter-and-index-blocks”:true,“use-bloom-filter”:true,“optimize-filters-for-hits”:false,“whole-key-filtering”:false,“bloom-filter-bits-per-key”:10,“block-based-bloom-filter”:false,“read-amp-bytes-per-bit”:0,“compression-per-level”:[“no”,“no”,“lz4”,“lz4”,“lz4”,“zstd”,“zstd”],“write-buffer-size”:“128MiB”,“max-write-buffer-number”:5,“min-write-buffer-number-to-merge”:1,“max-bytes-for-level-base”:“512MiB”,“target-file-size-base”:“8MiB”,“level0-file-num-compaction-trigger”:4,“level0-slowdown-writes-trigger”:20,“level0-stop-writes-trigger”:36,“max-compaction-bytes”:“2GiB”,“compaction-pri”:3,“dynamic-level-bytes”:true,“num-levels”:7,“max-bytes-for-level-multiplier”:10,“compaction-style”:0,“disable-auto-compactions”:false,“soft-pending-compaction-bytes-limit”:“64GiB”,“hard-pending-compaction-bytes-limit”:“256GiB”,“force-consistency-checks”:true,“prop-size-index-distance”:4194304,“prop-keys-index-distance”:40960,“enable-doubly-skiplist”:true,“titan”:{“min-blob-size”:“1KiB”,“blob-file-compression”:“lz4”,“blob-cache-size”:“0KiB”,“min-gc-batch-size”:“16MiB”,“max-gc-batch-size”:“64MiB”,“discardable-ratio”:0.5,“sample-ratio”:0.1,“merge-small-file-threshold”:“8MiB”,“blob-run-mode”:“read-only”,“level-merge”:false,“range-merge”:true,“max-sorted-runs”:20,“gc-merge-rewrite”:false}},“lockcf”:{“block-size”:“16KiB”,“block-cache-size”:“639MiB”,“disable-block-cache”:false,“cache-index-and-filter-blocks”:true,“pin-l0-filter-and-index-blocks”:true,“use-bloom-filter”:true,“optimize-filters-for-hits”:false,“whole-key-filtering”:true,“bloom-filter-bits-per-key”:10,“block-based-bloom-filter”:false,“read-amp-bytes-per-bit”:0,“compression-per-level”:[“no”,“no”,“no”,“no”,“no”,“no”,“no”],“write-buffer-size”:“32MiB”,“max-write-buffer-number”:5,“min-write-buffer-number-to-merge”:1,“max-bytes-for-level-base”:“128MiB”,“target-file-size-base”:“8MiB”,“level0-file-num-compaction-trigger”:1,“level0-slowdown-writes-trigger”:20,“level0-stop-writes-trigger”:36,“max-compaction-bytes”:“2GiB”,“compaction-pri”:0,“dynamic-level-bytes”:true,“num-levels”:7,“max-bytes-for-level-multiplier”:10,“compaction-style”:0,“disable-auto-compactions”:false,“soft-pending-compaction-bytes-limit”:“64GiB”,“hard-pending-compaction-bytes-limit”:“256GiB”,“force-consistency-checks”:true,“prop-size-index-distance”:4194304,“prop-keys-index-distance”:40960,“enable-doubly-skiplist”:true,“titan”:{“min-blob-size”:“1KiB”,“blob-file-compression”:“lz4”,“blob-cache-size”:“0KiB”,“min-gc-batch-size”:“16MiB”,“max-gc-batch-size”:“64MiB”,“discardable-ratio”:0.5,“sample-ratio”:0.1,“merge-small-file-threshold”:“8MiB”,“blob-run-mode”:“read-only”,“level-merge”:false,“range-merge”:true,“max-sorted-runs”:20,“gc-merge-rewrite”:false}},“raftcf”:{“block-size”:“16KiB”,“block-cache-size”:“128MiB”,“disable-block-cache”:false,“cache-index-and-filter-blocks”:true,“pin-l0-filter-and-index-blocks”:true,“use-bloom-filter”:true,“optimize-filters-for-hits”:true,“whole-key-filtering”:true,“bloom-filter-bits-per-key”:10,“block-based-bloom-filter”:false,“read-amp-bytes-per-bit”:0,“compression-per-level”:[“no”,“no”,“no”,“no”,“no”,“no”,“no”],“write-buffer-size”:“128MiB”,“max-write-buffer-number”:5,“min-write-buffer-number-to-merge”:1,“max-bytes-for-level-base”:“128MiB”,“target-file-size-base”:“8MiB”,“level0-file-num-compaction-trigger”:1,“level0-slowdown-writes-trigger”:20,“level0-stop-writes-trigger”:36,“max-compaction-bytes”:“2GiB”,“compaction-pri”:0,“dynamic-level-bytes”:true,“num-levels”:7,“max-bytes-for-level-multiplier”:10,“compaction-style”:0,“disable-auto-compactions”:false,“soft-pending-compaction-bytes-limit”:“64GiB”,“hard-pending-compaction-bytes-limit”:“256GiB”,“force-consistency-checks”:true,“prop-size-index-distance”:4194304,“prop-keys-index-distance”:40960,“enable-doubly-skiplist”:true,“titan”:{“min-blob-size”:“1KiB”,“blob-file-compression”:“lz4”,“blob-cache-size”:“0KiB”,“min-gc-batch-size”:“16MiB”,“max-gc-batch-size”:“64MiB”,“discardable-ratio”:0.5,“sample-ratio”:0.1,“merge-small-file-threshold”:“8MiB”,“blob-run-mode”:“read-only”,“level-merge”:false,“range-merge”:true,“max-sorted-runs”:20,“gc-merge-rewrite”:false}},“titan”:{“enabled”:false,“dirname”:"",“disable-gc”:false,“max-background-gc”:4,“purge-obsolete-files-period”:“10s”}},“raftdb”:{“wal-recovery-mode”:2,“wal-dir”:"",“wal-ttl-seconds”:0,“wal-size-limit”:“0KiB”,“max-total-wal-size”:“4GiB”,“max-background-jobs”:4,“max-manifest-file-size”:“20MiB”,“create-if-missing”:true,“max-open-files”:40960,“enable-statistics”:true,“stats-dump-period”:“10m”,“compaction-readahead-size”:“0KiB”,“info-log-max-size”:“1GiB”,“info-log-roll-time”:“0s”,“info-log-keep-log-file-num”:10,“info-log-dir”:"",“max-sub-compactions”:2,“writable-file-max-buffer-size”:“1MiB”,“use-direct-io-for-flush-and-compaction”:false,“enable-pipelined-write”:true,“enable-unordered-write”:false,“allow-concurrent-memtable-write”:true,“bytes-per-sync”:“1MiB”,“wal-bytes-per-sync”:“512KiB”,“defaultcf”:{“block-size”:“64KiB”,“block-cache-size”:“639MiB”,“disable-block-cache”:false,“cache-index-and-filter-blocks”:true,“pin-l0-filter-and-index-blocks”:true,“use-bloom-filter”:false,“optimize-filters-for-hits”:true,“whole-key-filtering”:true,“bloom-filter-bits-per-key”:10,“block-based-bloom-filter”:false,“read-amp-bytes-per-bit”:0,“compression-per-level”:[“no”,“no”,“lz4”,“lz4”,“lz4”,“zstd”,“zstd”],“write-buffer-size”:“128MiB”,“max-write-buffer-number”:5,“min-write-buffer-number-to-merge”:1,“max-bytes-for-level-base”:“512MiB”,“target-file-size-base”:“8MiB”,“level0-file-num-compaction-trigger”:4,“level0-slowdown-writes-trigger”:20,“level0-stop-writes-trigger”:36,“max-compaction-bytes”:“2GiB”,“compaction-pri”:0,“dynamic-level-bytes”:true,“num-levels”:7,“max-bytes-for-level-multiplier”:10,“compaction-style”:0,“disable-auto-compactions”:false,“soft-pending-compaction-bytes-limit”:“64GiB”,“hard-pending-compaction-bytes-limit”:“256GiB”,“force-consistency-checks”:true,“prop-size-index-distance”:4194304,“prop-keys-index-distance”:40960,“enable-doubly-skiplist”:true,“titan”:{“min-blob-size”:“1KiB”,“blob-file-compression”:“lz4”,“blob-cache-size”:“0KiB”,“min-gc-batch-size”:“16MiB”,“max-gc-batch-size”:“64MiB”,“discardable-ratio”:0.5,“sample-ratio”:0.1,“merge-small-file-threshold”:“8MiB”,“blob-run-mode”:“normal”,“level-merge”:false,“range-merge”:true,“max-sorted-runs”:20,“gc-merge-rewrite”:false}},“titan”:{“enabled”:false,“dirname”:"",“disable-gc”:false,“max-background-gc”:4,“purge-obsolete-files-period”:“10s”}},“security”:{“ca-path”:"",“cert-path”:"",“key-path”:"",“cert-allowed-cn”:[],“encryption”:{“data-encryption-method”:“plaintext”,“data-key-rotation-period”:“7d”,“master-key”:{“type”:“plaintext”},“previous-master-key”:{“type”:“plaintext”}}},“import”:{“num-threads”:8,“stream-channel-window”:128},“pessimistic-txn”:{“enabled”:true,“wait-for-lock-timeout”:“1s”,“wake-up-delay-duration”:“20ms”,“pipelined”:false},“gc”:{“ratio-threshold”:1.1,“batch-keys”:512,“max-write-bytes-per-sec”:“0KiB”},“split”:{“qps-threshold”:3000,“split-balance-score”:0.25,“split-contained-score”:0.5,“detect-times”:10,“sample-num”:20,“sample-threshold”:100}}"]
[2021/05/18 11:46:48.257 +08:00] [INFO] [mod.rs:335] [“starting working thread”] [worker=addr-resolver]
[2021/05/18 11:46:48.257 +08:00] [INFO] [mod.rs:335] [“starting working thread”] [worker=region-collector-worker]
[2021/05/18 11:46:48.257 +08:00] [INFO] [mod.rs:85] [“encryption: none of key dictionary and file dictionary are found.”]
[2021/05/18 11:46:48.257 +08:00] [INFO] [mod.rs:374] [“encryption is disabled.”]

如果日志不大的话,整个日志上传一下吧。

tikv.log.zip (1.3 MB)
附件中的文件是最新的log,谢谢

  1. 查看日志报错:
    [2021/05/18 11:51:14.339 +08:00] [FATAL] [server.rs:576] [“failed to start node: EngineTraits(Other(”[components/raftstore/src/store/fsm/store.rs:812]: \"[components/raftstore/src/store/peer_storage.rs:385]: [region 76097] entry at apply index 357578346 doesn\\\'t exist, may lose data.\""))"]
  2. 请参考文档确认是否由于掉电等原因导致,可以将这个 region 设置为 tombstone 试试。
    TiKV Control 使用说明 | PingCAP Docs

好的,谢谢!

我也出现了类试的问题,但是在使用pd-ctl的region命令去找报错中出现的region_id,返回的结果是null,
image
就没有办法将其设置为 tombstone。
有没有其他的办法跳过。

此话题已在最后回复的 1 分钟后被自动关闭。不再允许新回复。