TiCDC无法同步大表数据到s3-sink

为提高效率,提问时请提供以下信息,问题描述清晰可优先响应。

  • 【TiDB 版本】:tidb v4.0.9 ticdc v4.0.9
  • 【问题描述】:ticdc同步cdclog到s3上时,如果我用sysbench插入1万行数据,那cdc能够正常同步数据到s3上;但是如果我用sysbench插入10万行数据,就只能同步DDL和logmeta,DML全都没有同步上,而且checkpointTs也一直没有更新。
    s3上只有ddl和log.meta:
    image
    tidb集群如下(只是单机上的一个测试集群):

    tidb:
    image
    changefeed状态显示正常:
[tidb@localhost bin]$ ./cdc cli changefeed query  -c s3-sink
{
  "info": {
    "sink-uri": "s3://tidb-test/cdclog/?endpoint=http://10.142.90.98:9000\u0026access-Key=xxx\u0026secret-Access-Key=xxx",
    "opts": {},
    "create-time": "2020-12-24T16:31:18.518593386+08:00",
    "start-ts": 421736920779063297,
    "target-ts": 0,
    "admin-job-type": 0,
    "sort-engine": "memory",
    "sort-dir": ".",
    "config": {
      "case-sensitive": true,
      "enable-old-value": false,
      "force-replicate": false,
      "check-gc-safe-point": true,
      "filter": {
        "rules": [
          "*.*"
        ],
        "ignore-txn-start-ts": null,
        "ddl-allow-list": null
      },
      "mounter": {
        "worker-num": 16
      },
      "sink": {
        "dispatchers": null,
        "protocol": "default"
      },
      "cyclic-replication": {
        "enable": false,
        "replica-id": 0,
        "filter-replica-ids": null,
        "id-buckets": 0,
        "sync-ddl": false
      },
      "scheduler": {
        "type": "table-number",
        "polling-time": -1
      }
    },
    "state": "normal",
    "history": null,
    "error": null,
    "sync-point-enabled": false,
    "sync-point-interval": 600000000000
  },
  "status": {
    "resolved-ts": 421736935326482436,
    "checkpoint-ts": 421736930647212033,
    "admin-job-type": 0
  },
  "count": 0,
  "task-status": [
    {
      "capture-id": "a64f9444-50a3-47fa-8cc6-9009140adba4",
      "status": {
        "tables": {
          "145": {
            "start-ts": 421736930647212033,
            "mark-table-id": 0
          }
        },
        "operation": null,
        "admin-job-type": 0
      }
    }
  ]
}

日志:
16:31到16:50之间一条日志都没有打(16点50关闭集群)


完整日志(在测试前先清空了一下,所以完整日志只有这么大):
cdc.log (20.4 KB)
若提问为性能优化、故障排查类问题,请下载脚本运行。终端输出的打印结果,请务必全选并复制粘贴上传。

顶一下

跟踪代码发现processor成功执行了EmitRowChangedEvents,但是一直没有更新sinkEmittedResolvedTs而导致一直没有运行FlushRowChangedEvent,有没有大佬知道怎么回事啊,搞了好久了:joy: