tidb中mysql库中元数据清理

Meditator · 2021 年8 月 20 日 03:25

版本：4.0.10
请问mysql.stats_histograms mysql.stats_meta的元数据是否可以清理？

因为：
[WARN] [endpoint.rs:537] [error-response] [err=“Region error (will back off and retry) message: “peer is not leader for region 2541904, leader may Some(id: 2575134 store_id: 247116)” not_leader { region_id: 2541904 leader { id: 2575134 store_id: 247116 } }”]

curl ￥{pd}}/regions/2541904
{
“start_key”: “dIAAAAAAAAAXX3KAAAAAAASa/A==”,
“end_key”: “dIAAAAAAAAAXX3KAAAAAAASdQQ==”,
“start_key_hex”: “7480000000000000175f728000000000049afc”,
“end_key_hex”: “7480000000000000175f728000000000049d41”,
“region_id”: 2541904,
“frames”: [
{
“db_name”: “mysql”,
“table_name”: “stats_histograms”,
“table_id”: 23,
“is_record”: true,
“record_id”: 301820
}
]
}

QBin · 2021 年8 月 20 日 04:08

正常情况下 peer is not leader 的报错。是 TiDB 把请求发到了非 Leader 的副本上，TiDB 会根据该错误更新本地路由（如果错误 response 里携带了最新 Leader 是哪个副本这一信息），并且重新发送请求到最新 Leader，一般情况下业务无感知。不是大量频繁的报错可以先忽略。

Meditator · 2021 年8 月 20 日 06:10

嗯，现在的case是集群空载的情况下还有大量的刷类似的错误，看了几个region发现都是mysql的元数据表。
参考

Meditator · 2021 年8 月 20 日 06:11

都是查询元数据的

QBin · 2021 年8 月 20 日 07:23

如果是测试环境可以关闭掉 PD 的调度看看还会不会有相应的报错。
确认一下目前是不是系统正在做统计信息收集。

Meditator · 2021 年8 月 20 日 08:13

这样后就没有异常日志输出，只有
[2021/08/20 16:13:36.778 +08:00] [WARN] [util.rs:209] [“updating PD client done”] [spend=7.262519ms]
这种

Meditator · 2021 年8 月 20 日 08:23

把某个tikv的debug日志打开，发现一些异常或者有意思的事情

[2021/08/20 15:41:46.704 +08:00] [DEBUG] [peer.rs:1371] [“handle raft ready”] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.704 +08:00] [DEBUG] [peer.rs:2918] [“send raft msg”] [to=549568] [from=549566] [msg_size=12] [msg_type=MsgHeartbeatResponse] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.704 +08:00] [DEBUG] [peer.rs:2918] [“send raft msg”] [to=549568] [from=549566] [msg_size=14] [msg_type=MsgTransferLeader] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.705 +08:00] [DEBUG] [peer.rs:1117] [“handle raft message”] [to_peer_id=549566] [from_peer_id=549568] [message_type=MsgTimeoutNow] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.705 +08:00] [INFO] [raft.rs:1739] [“[term 26] received MsgTimeoutNow from 549568 and starts an election to get leadership.”] [from=549568] [term=26] [raft_id=549566] [region_id=549565]

[2021/08/20 15:41:46.705 +08:00] [INFO] [raft.rs:1177] [“starting a new election”] [term=26] [raft_id=549566] [region_id=549565]
[2021/08/20 15:41:46.705 +08:00] [DEBUG] [raft.rs:2180] [“reset election timeout 15 → 14 at 0”] [election_elapsed=0] [timeout=14] [prev_timeout=15] [raft_id=549566] [region_id=549565]
[2021/08/20 15:41:46.705 +08:00] [INFO] [raft.rs:807] [“became candidate at term 27”] [term=27] [raft_id=549566] [region_id=549565]
[2021/08/20 15:41:46.705 +08:00] [INFO] [raft.rs:902] [“549566 received message from 549566”] [term=27] [msg=MsgRequestVote] [from=549566] [id=549566] [raft_id=549566] [region_id=549565]
[2021/08/20 15:41:46.705 +08:00] [INFO] [raft.rs:923] [“[logterm: 26, index: 26] sent request to 549568”] [msg=MsgRequestVote] [term=27] [id=549568] [log_index=26] [log_term=26] [raft_id=549566] [region_id=549565]
[2021/08/20 15:41:46.705 +08:00] [DEBUG] [raft.rs:400] [“Sending from 549566 to 549568”] [msg=“msg_type: MsgRequestVote to: 549568 term: 27 log_term: 26 index: 26 context: 43616D706169676E5472616E73666572”] [to=549568] [from=549566] [raft_id=549566] [region_id=549565]
[2021/08/20 15:41:46.705 +08:00] [INFO] [raft.rs:923] [“[logterm: 26, index: 26] sent request to 549567”] [msg=MsgRequestVote] [term=27] [id=549567] [log_index=26] [log_term=26] [raft_id=549566] [region_id=549565]
[2021/08/20 15:41:46.705 +08:00] [DEBUG] [raft.rs:400] [“Sending from 549566 to 549567”] [msg=“msg_type: MsgRequestVote to: 549567 term: 27 log_term: 26 index: 26 context: 43616D706169676E5472616E73666572”] [to=549567] [from=549566] [raft_id=549566] [region_id=549565]

[2021/08/20 15:41:46.705 +08:00] [DEBUG] [peer.rs:1371] [“handle raft ready”] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.705 +08:00] [DEBUG] [mod.rs:148] [“scheduling task RaftStoreEvent(RoleChange { region: id: 549565 start_key: 7480000000000001FF945F728000000001FFE2F5A00000000000FA end_key: 7480000000000001FF945F728000000001FFE948AD0000000000FA region_epoch { conf_ver: 305 version: 685 } peers { id: 549566 store_id: 358466 } peers { id: 549567 store_id: 145642 } peers { id: 549568 store_id: 62028 }, role: Candidate })”]
[2021/08/20 15:41:46.705 +08:00] [DEBUG] [peer.rs:2918] [“send raft msg”] [to=549568] [from=549566] [msg_size=34] [msg_type=MsgRequestVote] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.705 +08:00] [DEBUG] [peer.rs:2918] [“send raft msg”] [to=549567] [from=549566] [msg_size=34] [msg_type=MsgRequestVote] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [peer.rs:1117] [“handle raft message”] [to_peer_id=549566] [from_peer_id=549567] [message_type=MsgRequestVoteResponse] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [INFO] [raft.rs:1673] [“received from 549567”] [term=27] [“msg type”=MsgRequestVoteResponse] [from=549567] [raft_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [raft.rs:2180] [“reset election timeout 14 → 12 at 0”] [election_elapsed=0] [timeout=12] [prev_timeout=14] [raft_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [INFO] [raft.rs:874] [“became leader at term 27”] [term=27] [raft_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [raft.rs:400] [“Sending from 549566 to 549568”] [msg=“msg_type: MsgAppend to: 549568 log_term: 26 index: 26 entries { term: 27 index: 27 } commit: 26”] [to=549568] [from=549566] [raft_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [raft.rs:400] [“Sending from 549566 to 549567”] [msg=“msg_type: MsgAppend to: 549567 log_term: 26 index: 26 entries { term: 27 index: 27 } commit: 26”] [to=549567] [from=549566] [raft_id=549566] [region_id=549565]

[2021/08/20 15:41:46.706 +08:00] [DEBUG] [peer.rs:1371] [“handle raft ready”] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [peer.rs:1860] [“update read progress”] [progress=Term(27)] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [peer.rs:1860] [“update read progress”] [progress=“LeaderLease(RemoteLease { expired_time: Timespec { sec: 1549679, nsec: 249561088 }, term: 27 })”] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [peer.rs:1122] [“becomes leader with lease”] [lease=“Lease { valid: Timespec { sec: 1549679, nsec: 249854289 } }”] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [mod.rs:148] [“scheduling task RaftStoreEvent(RoleChange { region: id: 549565 start_key: 7480000000000001FF945F728000000001FFE2F5A00000000000FA end_key: 7480000000000001FF945F728000000001FFE948AD0000000000FA region_epoch { conf_ver: 305 version: 685 } peers { id: 549566 store_id: 358466 } peers { id: 549567 store_id: 145642 } peers { id: 549568 store_id: 62028 }, role: Leader })”]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [peer.rs:2918] [“send raft msg”] [to=549568] [from=549566] [msg_size=24] [msg_type=MsgAppend] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [peer.rs:2918] [“send raft msg”] [to=549567] [from=549566] [msg_size=24] [msg_type=MsgAppend] [peer_id=549566] [region_id=549565]

[2021/08/20 15:41:46.706 +08:00] [DEBUG] [peer_storage.rs:907] [“append entries”] [count=1] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [peer.rs:1006] [“peer start pending”] [time=“Instant { tv_sec: 1549671, tv_nsec: 747612263 }”] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [peer.rs:1006] [“peer start pending”] [time=“Instant { tv_sec: 1549671, tv_nsec: 747613847 }”] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [mod.rs:148] [“scheduling task [split check worker] Get region approximate size and keys for region 549565”]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [peer.rs:1117] [“handle raft message”] [to_peer_id=549566] [from_peer_id=549567] [message_type=MsgAppendResponse] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [raft_log.rs:424] [“committing index 27”] [index=27] [raft_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [raft.rs:400] [“Sending from 549566 to 549567”] [msg=“msg_type: MsgAppend to: 549567 log_term: 27 index: 27 commit: 27”] [to=549567] [from=549566] [raft_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [peer.rs:1051] [“peer has caught up logs”] [takes=0.000156361] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [mod.rs:148] [“scheduling task [split check worker] Get region approximate size and keys for region 549565”]

[2021/08/20 15:41:46.706 +08:00] [DEBUG] [peer.rs:1371] [“handle raft ready”] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [peer.rs:2918] [“send raft msg”] [to=549567] [from=549566] [msg_size=18] [msg_type=MsgAppend] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [peer.rs:1117] [“handle raft message”] [to_peer_id=549566] [from_peer_id=549568] [message_type=MsgRequestVoteResponse] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [peer.rs:1068] [“async apply finish”] [res=“ApplyRes { region_id: 549565, apply_state: applied_index: 27 last_commit_index: 26 commit_index: 27 commit_term: 27 truncated_state { index: 5 term: 5 }, applied_index_term: 27, exec_res: [], metrics: ApplyMetrics { size_diff_hint: 0, delete_keys_hint: 0, written_bytes: 233, written_keys: 1, lock_cf_written_bytes: 0 } }”] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [peer.rs:1860] [“update read progress”] [progress=AppliedIndexTerm(27)] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [peer.rs:1117] [“handle raft message”] [to_peer_id=549566] [from_peer_id=549568] [message_type=MsgAppendResponse] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [raft.rs:400] [“Sending from 549566 to 549568”] [msg=“msg_type: MsgAppend to: 549568 log_term: 27 index: 27 commit: 27”] [to=549568] [from=549566] [raft_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [peer.rs:1051] [“peer has caught up logs”] [takes=0.000298691] [peer_id=549566] [region_id=549565]
[2021/08/20 15:41:46.706 +08:00] [DEBUG] [mod.rs:148] [“scheduling task [split check worker] Get region approximate size and keys for region 549565”]

Meditator · 2021 年8 月 20 日 08:25

region_id=549566的这个region在不断的发起leader选举，但是实际上这个时间点（这个表一周都没有写过数据）集群没有任何写入。

yilong · 2021 年9 月 1 日 03:25

可以看下 admin show ddl 是否有 ddl 任务，或者 SHOW STATS_META 是否有收集统计信息？

Billmay表妹 · 2021 年9 月 7 日 07:59

该问题是否已经解决？如已经解决，请 对问题标记【对我有用】，问题 才能被搜索到，也能帮助他人更高效地找到答案。如果你的问题还没有解决，请继续追问及反馈你遇到的问题，附上操作提示或者截图。

Meditator · 2022 年10 月 31 日 19:15

此话题已在最后回复的 1 分钟后被自动关闭。不再允许新回复。