ticdc 报错信息不友好,Unknown column 'flag' in 'field list',也不提示哪个表

【 TiDB 使用环境`】生产

【 TiDB 版本】5.4.2

【遇到的问题】
cdc 报错
[CDC:ErrReachMaxTry]reach maximum try: 8, error: [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list': [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list',

也不提示哪个表,不能快速定位异常

日志往上拉一下,看看有没有 table id, 通过 table id 可以很快定位的

日志中应该有提示 可能在前后文

cdc.log.gz (3.4 MB)

上下文涉及的tableid 不是报缺少字段的表。

MySQL标准报错也是这样的?我记得我以前开发应用,好像遇到过这种报错。
带上肯定是最好的。

列都不存在呢,怎么提示是哪个表的?

[2022/08/19 15:47:52.571 +08:00] [ERROR] [mysql.go:737] ["execute DMLs failed"] [err="[CDC:ErrReachMaxTry]reach maximum try: 8, error: [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list': [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list'"]
[2022/08/19 15:47:52.694 +08:00] [ERROR] [processor.go:530] ["error on running processor"] [capture=192.168.6.49:8301] [changefeed=cbd-user-behavior-analysis-task] [error="[CDC:ErrReachMaxTry]reach maximum try: 8, error: [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list': [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list'"] [errorVerbose="[CDC:ErrReachMaxTry]reach maximum try: 8, error: [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list': [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list'\ngithub.com/pingcap/errors.AddStack\n\tgithub.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/errors.go:174\ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs\n\tgithub.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/normalize.go:164\ngithub.com/pingcap/tiflow/pkg/retry.run\n\tgithub.com/pingcap/tiflow/pkg/retry/retry_with_opt.go:69\ngithub.com/pingcap/tiflow/pkg/retry.Do\n\tgithub.com/pingcap/tiflow/pkg/retry/retry_with_opt.go:34\ngithub.com/pingcap/tiflow/cdc/sink.(*mysqlSink).execDMLWithMaxRetries\n\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:563\ngithub.com/pingcap/tiflow/cdc/sink.(*mysqlSink).execDMLs\n\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:736\ngithub.com/pingcap/tiflow/cdc/sink.(*mysqlSinkWorker).run.func3\n\tgithub.com/pingcap/tiflow/cdc/sink/mysql_worker.go:114\ngithub.com/pingcap/tiflow/cdc/sink.(*mysqlSinkWorker).run\n\tgithub.com/pingcap/tiflow/cdc/sink/mysql_worker.go:150\ngithub.com/pingcap/tiflow/cdc/sink.(*mysqlSink).createSinkWorkers.func1\n\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:380\nruntime.goexit\n\truntime/asm_amd64.s:1371"]
[2022/08/19 15:47:52.694 +08:00] [ERROR] [processor.go:316] ["run processor failed"] [changefeed=cbd-user-behavior-analysis-task] [capture=192.168.6.49:8301] [error="[CDC:ErrReachMaxTry]reach maximum try: 8, error: [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list': [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list'"] [errorVerbose="[CDC:ErrReachMaxTry]reach maximum try: 8, error: [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list': [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list'\ngithub.com/pingcap/errors.AddStack\n\tgithub.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/errors.go:174\ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs\n\tgithub.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/normalize.go:164\ngithub.com/pingcap/tiflow/pkg/retry.run\n\tgithub.com/pingcap/tiflow/pkg/retry/retry_with_opt.go:69\ngithub.com/pingcap/tiflow/pkg/retry.Do\n\tgithub.com/pingcap/tiflow/pkg/retry/retry_with_opt.go:34\ngithub.com/pingcap/tiflow/cdc/sink.(*mysqlSink).execDMLWithMaxRetries\n\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:563\ngithub.com/pingcap/tiflow/cdc/sink.(*mysqlSink).execDMLs\n\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:736\ngithub.com/pingcap/tiflow/cdc/sink.(*mysqlSinkWorker).run.func3\n\tgithub.com/pingcap/tiflow/cdc/sink/mysql_worker.go:114\ngithub.com/pingcap/tiflow/cdc/sink.(*mysqlSinkWorker).run\n\tgithub.com/pingcap/tiflow/cdc/sink/mysql_worker.go:150\ngithub.com/pingcap/tiflow/cdc/sink.(*mysqlSink).createSinkWorkers.func1\n\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:380\nruntime.goexit\n\truntime/asm_amd64.s:1371"]
[2022/08/19 15:47:52.694 +08:00] [INFO] [processor.go:1061] ["processor closing ..."] [changefeed=cbd-user-behavior-analysis-task]
[capture=192.168.6.49:8301] [changefeed=cbd-user-behavior-analysis-task]

日志里头提示得很清楚拉,这个changefeed 出错了嘛,检查下范围就好了?

这个changefeed 同步了1000个表,不提示表名或者 日志打印出错的SQL,排查起来很麻烦

这个目前好像也没太好的办法,只能在设定changefeed 的时候,做好切分了…

或者,你找下上下文中相关的region,通过region 的 id 去获取 table id,这样会靠谱点,没其他的好办法了…

我是从备份SQL过滤字段排查的。希望后期cdc 能优化下 报错日志