为提高效率,提问时请提供以下信息,问题描述清晰可优先响应。
- 【TiDB 版本】:tidb v4.0.9 ticdc v4.0.9
- 【问题描述】:ticdc同步cdclog到s3上时,如果我用sysbench插入1万行数据,那cdc能够正常同步数据到s3上;但是如果我用sysbench插入10万行数据,就只能同步DDL和logmeta,DML全都没有同步上,而且checkpointTs也一直没有更新。
s3上只有ddl和log.meta:
tidb集群如下(只是单机上的一个测试集群):
tidb:
changefeed状态显示正常:
[tidb@localhost bin]$ ./cdc cli changefeed query -c s3-sink
{
"info": {
"sink-uri": "s3://tidb-test/cdclog/?endpoint=http://10.142.90.98:9000\u0026access-Key=xxx\u0026secret-Access-Key=xxx",
"opts": {},
"create-time": "2020-12-24T16:31:18.518593386+08:00",
"start-ts": 421736920779063297,
"target-ts": 0,
"admin-job-type": 0,
"sort-engine": "memory",
"sort-dir": ".",
"config": {
"case-sensitive": true,
"enable-old-value": false,
"force-replicate": false,
"check-gc-safe-point": true,
"filter": {
"rules": [
"*.*"
],
"ignore-txn-start-ts": null,
"ddl-allow-list": null
},
"mounter": {
"worker-num": 16
},
"sink": {
"dispatchers": null,
"protocol": "default"
},
"cyclic-replication": {
"enable": false,
"replica-id": 0,
"filter-replica-ids": null,
"id-buckets": 0,
"sync-ddl": false
},
"scheduler": {
"type": "table-number",
"polling-time": -1
}
},
"state": "normal",
"history": null,
"error": null,
"sync-point-enabled": false,
"sync-point-interval": 600000000000
},
"status": {
"resolved-ts": 421736935326482436,
"checkpoint-ts": 421736930647212033,
"admin-job-type": 0
},
"count": 0,
"task-status": [
{
"capture-id": "a64f9444-50a3-47fa-8cc6-9009140adba4",
"status": {
"tables": {
"145": {
"start-ts": 421736930647212033,
"mark-table-id": 0
}
},
"operation": null,
"admin-job-type": 0
}
}
]
}
日志:
16:31到16:50之间一条日志都没有打(16点50关闭集群)
完整日志(在测试前先清空了一下,所以完整日志只有这么大):
cdc.log (20.4 KB)
若提问为性能优化、故障排查类问题,请下载脚本运行。终端输出的打印结果,请务必全选并复制粘贴上传。