【 TiDB 使用环境】生产环境
【 TiDB 版本】从v5.4.0升级到v6.5.3
【复现路径】ticdc中存在changefeed,直接remove changefeed后,缩容相关ticdc节点,发现20多个小时没有gc。
【遇到的问题:问题现象及影响】现在是gc不能正常推进,在gc leader中发现如下的相关日志:[2023/07/27 08:53:26.921 +08:00] [INFO] [gc_worker.go:625] ["[gc worker] there's another service in the cluster requires an earlier safe point. gc will continue with the earlier one"] [uuid=625ddd430bc000a] [ourSafePoint=443111232467894272] [minSafePoint=443110872819695626] [2023/07/27 08:53:26.921 +08:00] [INFO] [gc_worker.go:601] ["[gc worker] last safe point is later than current one.No need to gc.This might be caused by manually enlarging gc lifetime"] ["leaderTick on"=625ddd430bc000a] ["last safe point"=2023/07/26 09:10:34.914 +08:00] ["current safe point"=2023/07/26 09:10:34.914 +08:00]
查看service-gc-safepoint可以看到有ticdc的相关safepoint:
{
"service_gc_safe_points": [
{
"service_id": "gc_worker",
"expired_at": 9223372036854775807,
"safe_point": 443111201010876416
},
{
"service_id": "ticdc-default-4589598632202768367",
"expired_at": 1690420235,
"safe_point": 443110872819695626
}
],
"gc_safe_point": 443110872819695626
}
但是扩容后ticdc,查看cdc中,没有changefeed的相关列表。
【资源配置】
【附件:截图/日志/监控】