【 TiDB 使用环境】生产\测试环境\ POC
【 TiDB 版本】
4.0.7
cdc server 以及cdc cli均为4.0.7
【遇到的问题】
基于时间点的同步,新建的复制任务会出现卡住的情况,tso不往前推进
【复现路径】做过哪些操作出现的问题
update
mysql.tidb
set
VARIABLE_VALUE=
'24h'
where
VARIABLE_NAME=
"tikv_gc_life_time"
;
br备份,作完整的数据库恢复,根据br备份文件metabackup中的commitTS作为cdc的tart-ts
./cdc cli changefeed create --pd=http://192.168.0.5:2379 --sink-uri=“mysql://test:test@192.168.0.1:4000/” --changefeed-id “test-2018” --config=changefeed.toml --start-ts=433921993657810984 --sort-engine=unified
复制任务不往前推进,暂停之后在启动也不行
【问题现象及影响】
【附件】
- 相关日志、配置文件、Grafana 监控(https://metricstool.pingcap.com/)
- TiUP Cluster Display 信息
- TiUP CLuster Edit config 信息
- TiDB-Overview 监控
- 对应模块的 Grafana 监控(如有 BR、TiDB-binlog、TiCDC 等)
- 对应模块日志(包含问题前后 1 小时日志)
[2022/06/14 15:18:18.321 +00:00] [DEBUG] [client.go:643] [“dispatching region”] [regionID=22649]
[2022/06/14 15:18:18.321 +00:00] [INFO] [client.go:713] [“creating new stream to store to send request”] [regionID=22649] [requestID=12] [storeID=4] [addr=tidb-z3x18cr0q6-tikv-1.tidb-z3x18cr0q6-tikv-peer.tidb-z3
[2022/06/14 15:18:18.323 +00:00] [DEBUG] [client.go:364] [“created stream to store”] [addr=tidb-z3x18cr0q6-tikv-1.tidb-z3x18cr0q6-tikv-peer.tidb-z3x18cr0q6.svc:20160]
[2022/06/14 15:18:18.323 +00:00] [DEBUG] [client.go:364] [“created stream to store”] [addr=tidb-z3x18cr0q6-tikv-0.tidb-z3x18cr0q6-tikv-peer.tidb-z3x18cr0q6.svc:20160]
[2022/06/14 15:18:18.323 +00:00] [DEBUG] [client.go:748] [“start new request”] [request="{“header”:{“cluster_id”:7102381364438096930,“ticdc_version”:“4.0.7”},“region_id”:22649,“region_epoch”:{“conf_
[2022/06/14 15:18:18.323 +00:00] [DEBUG] [client.go:748] [“start new request”] [request=”{“header”:{“cluster_id”:7102381364438096930,“ticdc_version”:“4.0.7”},“region_id”:16697,“region_epoch”:{“conf_
[2022/06/14 15:18:18.324 +00:00] [INFO] [puller.go:247] [“puller is initialized”] [duration=4.990905ms] [changefeedid=billing-2247] [tableID=594] [spans=”["[7480000000000002ff525f000000000000f9, 748000000000000
[2022/06/14 15:18:18.324 +00:00] [INFO] [puller.go:247] [“puller is initialized”] [duration=5.099369ms] [changefeedid=billing-2247] [tableID=413] [spans="["[7480000000000001ff9d5f000000000000f9, 748000000000000
[2022/06/14 15:18:18.766 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
[2022/06/14 15:18:19.265 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
[2022/06/14 15:18:19.765 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
[2022/06/14 15:18:20.265 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
[2022/06/14 15:18:20.765 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
[2022/06/14 15:18:21.266 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
[2022/06/14 15:18:21.765 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
[2022/06/14 15:18:22.266 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
[2022/06/14 15:18:22.765 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
[2022/06/14 15:18:23.267 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
[2022/06/14 15:18:23.765 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
[2022/06/14 15:18:24.265 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
[2022/06/14 15:18:24.765 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
[2022/06/14 15:18:25.265 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
[2022/06/14 15:18:25.767 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
[2022/06/14 15:18:26.265 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
[2022/06/14 15:18:26.765 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
[2022/06/14 15:18:27.265 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
[2022/06/14 15:18:27.765 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
[2022/06/14 15:18:28.265 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
[2022/06/14 15:18:28.765 +00:00] [DEBUG] [changefeed.go:716] [“skip update resolved ts”] [taskPositions=0] [taskStatus=1]
若提问为性能优化、故障排查类问题,请下载脚本运行。终端输出的打印结果,请务必全选并复制粘贴上传。