tidb 使用flink-sql-connector同步时,大表不能成功同步

Describe the bug(Please use English)
A clear and concise description of what the bug is.
tidb 使用flink sql 同步时,数据量大的表同步不成功,相同的表结构,数据量小的表能够同步成功。
Environment :

  • Flink version : 1.15.1
  • Flink CDC version: flink-sql-connector-tidb-cdc-2.2.1.jar
  • Database and version: tidb 6.1

To Reproduce
Steps to reproduce the behavior:

  1. The test data :
  2. The test code :
  3. The error :
    2022-08-18 11:19:34,325 WARN org.tikv.common.region.AbstractRegionStoreClient - no followers of region[4052] available, retry
    2022-08-18 11:19:34,326 WARN org.tikv.common.operation.RegionErrorHandler - request failed because of: UNKNOWN
    2022-08-18 11:19:34,333 INFO org.tikv.cdc.CDCClient - remove regions:
    2022-08-18 11:19:34,333 WARN org.apache.flink.runtime.taskmanager.Task - Source: cnft_item_backup[1] (1/1)ClassCastException: java.lang.Integer cannot be cast to java.lang.Long when using oracle connector #121 (c40c3c7e9651b3343fa3c8f33eadd159) switched from RUNNING to FAILED with failure cause: org.tikv.common.exception.TiClientInternalException: Error scanning data from region.
    at org.tikv.common.operation.iterator.ScanIterator.cacheLoadFails(ScanIterator.java:115)
    at org.tikv.common.operation.iterator.ConcreteScanIterator.hasNext(ConcreteScanIterator.java:105)
    at java.base/java.util.Iterator.forEachRemaining(Unknown Source)
    at org.tikv.txn.KVClient.scan(KVClient.java:117)
    at com.ververica.cdc.connectors.tidb.TiKVRichParallelSourceFunction.readSnapshotEvents(TiKVRichParallelSourceFunction.java:172)
    at com.ververica.cdc.connectors.tidb.TiKVRichParallelSourceFunction.run(TiKVRichParallelSourceFunction.java:127)
    at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110)
    at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:67)
    at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:332)
    Caused by: org.tikv.common.exception.TiClientInternalException: ScanResponse failed without a cause
    at org.tikv.common.region.RegionStoreClient.isScanSuccess(RegionStoreClient.java:315)
    at org.tikv.common.region.RegionStoreClient.scan(RegionStoreClient.java:306)
    at org.tikv.common.region.RegionStoreClient.scan(RegionStoreClient.java:346)
    at org.tikv.common.operation.iterator.ConcreteScanIterator.loadCurrentRegionToCache(ConcreteScanIterator.java:80)
    at org.tikv.common.operation.iterator.ScanIterator.cacheLoadFails(ScanIterator.java:80)
    … 8 more

Additional Description
If applicable, add screenshots to help explain your problem.

【 TiDB 使用环境】线上、测试、调研
【 TiDB 版本】
【遇到的问题】
【复现路径】做过哪些操作出现的问题
【问题现象及影响】

【附件】

请提供各个组件的 version 信息,如 cdc/tikv,可通过执行 cdc version/tikv-server --version 获取。

是不是数据格式不对导致的,

2022-08-18 11:19:34,333 WARN org.apache.flink.runtime.taskmanager.Task [] - Source: cnft_item_backup[1] (1/1)ClassCastException: java.lang.Integer cannot be cast to java.lang.Long when using oracle connector #121 (c40c3c7e9651b3343fa3c8f33eadd159) switched from RUNNING to FAILED with failure cause: org.tikv.common.exception.TiClientInternalException: Error scanning data from region.

日志上描述的类型转换错误呢

oracle connector 我使用的是tidb connector,还有一个问题,同样的表数据量少的时候为什么不会出现问题

数据类型不对的意思是tidb字段中的类型和flink中表数据类型不一致?

cnft_item_backup

你自己核对一下吧

此话题已在最后回复的 60 天后被自动关闭。不再允许新回复。