版本升级TIDB5.1CDC向TIDB7.5.3同步数据出错ErrProcessorUnknown
【 TiDB 使用环境】生产环境
【 TiDB 版本】TIDB5.1–>TIDB7.5.3同步数据
【复现路径】做过哪些操作出现的问题
使用CDC (v5.1.4) 向TIDB7.5.3同步数据，常运行一段时间,出错CDC:ErrProcessorUnknown"
【遇到的问题：问题现象及影响】
/data/cdc-8300/bin/cdc cli changefeed query -s -c atpub-upload-pic --pd=http://xx:2379
“code”: “CDC:ErrProcessorUnknown”,
“message”: “[CDC:ErrReachMaxTry]reach maximum try: 8”
====日志信息:===
日志信息：
9262 [2024/10/21 19:14:33.607 +08:00] [INFO] [statistics.go:154] [“sink replication status”] [name=mysql] [changefeed=atpub-motobike] [capture=10.x.x.x:8300] [count=0] [qp
s=0] [ddl=0]
79263 [2024/10/21 19:14:41.497 +08:00] [WARN] [mysql.go:903] [“execute DMLs with error, retry later”] [error=“[CDC:ErrMySQLTxnError]Error 1105: statement count 5001 exceeds th
e transaction limitation, transaction has been rollback, autocommit = true”] [errorVerbose=“[CDC:ErrMySQLTxnError]Error 1105: statement count 5001 exceeds the transactio
n limitation, transaction has been rollback, autocommit = true\ngithub.com/pingcap/errors.AddStack\n\tgithub.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/err
ors.go:174\ngithub.com/pingcap/errors.(*Error).GenWithStackByCause\n\tgithub.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/normalize.go:279\ngithub.com/pingca
p/tiflow/pkg/errors.WrapError\n\tgithub.com/pingcap/tiflow/pkg/errors/helper.go:30\ngithub.com/pingcap/tiflow/cdc/sink.(*mysqlSink).execDMLWithMaxRetries.func1.3\n\tgith
ub.com/pingcap/tiflow/cdc/sink/mysql.go:953\ngithub.com/pingcap/tiflow/cdc/sink.(*Statistics).RecordBatchExecution\n\tgithub.com/pingcap/tiflow/cdc/sink/statistics.go:11
2\ngithub.com/pingcap/tiflow/cdc/sink.(*mysqlSink).execDMLWithMaxRetries.func1\n\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:940\ngithub.com/pingcap/tiflow/pkg/retry.ru
n\n\tgithub.com/pingcap/tiflow/pkg/retry/retry_with_opt.go:54\ngithub.com/pingcap/tiflow/pkg/retry.Do\n\tgithub.com/pingcap/tiflow/pkg/retry/retry_with_opt.go:32\ngithub
.com/pingcap/tiflow/cdc/sink.(*mysqlSink).execDMLWithMaxRetries\n\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:932\ngithub.com/pingcap/tiflow/cdc/sink.(*mysqlSink).execD
MLs\n\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:1091\ngithub.com/pingcap/tiflow/cdc/sink.(*mysqlSinkWorker).run.func3\n\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:7
86\ngithub.com/pingcap/tiflow/cdc/sink.(*mysqlSinkWorker).run\n\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:806\ngithub.com/pingcap/tiflow/cdc/sink.(*mysqlSink).createS
inkWorkers.func1\n\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:629\nruntime.goexit\n\truntime/asm_amd64.s:1371”]
79264 [2024/10/21 19:14:41.497 +08:00] [ERROR] [mysql.go:1092] [“execute DMLs failed”] [err=“[CDC:ErrReachMaxTry]reach maximum try: 8”]
79265 [2024/10/21 19:14:41.602 +08:00] [ERROR] [processor.go:321] [“error on running processor”] [capture=10.x.x.x:8300] [changefeed=atpub-upload-pic] [error=“[CDC:ErrReach
MaxTry]reach maximum try: 8”] [errorVerbose=“[CDC:ErrReachMaxTry]reach maximum try: 8\ngithub.com/pingcap/errors.AddStack\n\tgithub.com/pingcap/errors@v0.11.5-0.20201126
102027-b0a155152ca3/errors.go:174\ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs\n\tgithub.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/normalize.go:
156\ngithub.com/pingcap/tiflow/pkg/retry.run\n\tgithub.com/pingcap/tiflow/pkg/retry/retry_with_opt.go:65\ngithub.com/pingcap/tiflow/pkg/retry.Do\n\tgithub.com/pingcap/ti
flow/pkg/retry/retry_with_opt.go:32\ngithub.com/pingcap/tiflow/cdc/sink.(*mysqlSink).execDMLWithMaxRetries\n\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:932\ngithub.com
/pingcap/tiflow/cdc/sink.(*mysqlSink).execDMLs\n\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:1091\ngithub.com/pingcap/tiflow/cdc/sink.(*mysqlSinkWorker).run.func3\n\tgi
thub.com/pingcap/tiflow/cdc/sink/mysql.go:786\ngithub.com/pingcap/tiflow/cdc/sink.(*mysqlSinkWorker).run\n\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:806\ngithub.com/p
ingcap/tiflow/cdc/sink.(*mysqlSink).createSinkWorkers.func1\n\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:629\nruntime.goexit\n\truntime/asm_amd64.s:1371”]
10.50.4.193:8300 这个节点是干嘛的？
连个拓扑都没有… 嗯，看不懂
tidb cdc batch size 配置下试试 https://docs.pingcap.com/zh/tidb/stable/ticdc-sink-to-mysql#sink-uri-配置-mysqltidb
。默认感觉不应该有这个错误。
stmt-count-limit
TiDB 单个事务允许的最大语句条数限制。
默认值：5000
在一个事务中，超过 stmt-count-limit 条语句后还没有 rollback 或者 commit，TiDB 将会返回 statement count 5001 exceeds the transaction limitation, autocommit = false 错误。该限制只在可重试的乐观事务中生效，如果使用悲观事务或者关闭了事务重试，事务中的语句数将不受此限制。
max-txn-row 同步的batch大小，默认256.
答:
193:8300 是CDC节点。
配置是这样的：
/data/cdc-8300/bin/cdc cli changefeed --config=/data/cdc_task_conf/product/xxxTo_atpub_upload_pic.toml --changefeed-id=atpub-upload-pic create --start-ts=453340585703243810 --pd=‘http://10.xx.xx:2379,http://10.xx.xx.194:2379,http://10.xx.xx.195:2379’ --sink-uri='mysql://ticdc_user:dxxx@10.xx.xx:4000/?worker-count=24&max-txn-row=10000
下游tidb有配置 stmt-count-limit参数吗？
这个看着是超过了限制报错导致的。
我看4楼小哥说的靠谱！
