ticdc 报错信息不友好，Unknown column 'flag' in 'field list'，也不提示哪个表

foxchan · 2022 年8 月 19 日 08:44

【 TiDB 使用环境`】生产

【 TiDB 版本】5.4.2

【遇到的问题】
cdc 报错
[CDC:ErrReachMaxTry]reach maximum try: 8, error: [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list': [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list',

也不提示哪个表，不能快速定位异常

xfworld · 2022 年8 月 19 日 09:41

日志往上拉一下，看看有没有 table id，通过 table id 可以很快定位的

xiaohetao · 2022 年8 月 20 日 11:15

日志中应该有提示可能在前后文

foxchan · 2022 年8 月 22 日 05:50

cdc.log.gz (3.4 MB)

上下文涉及的tableid 不是报缺少字段的表。

张雨齐0720 · 2022 年8 月 22 日 06:09

MySQL标准报错也是这样的？我记得我以前开发应用，好像遇到过这种报错。
带上肯定是最好的。

我是咖啡哥 · 2022 年8 月 22 日 06:13

列都不存在呢，怎么提示是哪个表的？

xfworld · 2022 年8 月 22 日 07:03

[2022/08/19 15:47:52.571 +08:00] [ERROR] [mysql.go:737] ["execute DMLs failed"] [err="[CDC:ErrReachMaxTry]reach maximum try: 8, error: [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list': [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list'"]
[2022/08/19 15:47:52.694 +08:00] [ERROR] [processor.go:530] ["error on running processor"] [capture=192.168.6.49:8301] [changefeed=cbd-user-behavior-analysis-task] [error="[CDC:ErrReachMaxTry]reach maximum try: 8, error: [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list': [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list'"] [errorVerbose="[CDC:ErrReachMaxTry]reach maximum try: 8, error: [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list': [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list'\
github.com/pingcap/errors.AddStack\
\tgithub.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/errors.go:174\
github.com/pingcap/errors.(*Error).GenWithStackByArgs\
\tgithub.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/normalize.go:164\
github.com/pingcap/tiflow/pkg/retry.run\
\tgithub.com/pingcap/tiflow/pkg/retry/retry_with_opt.go:69\
github.com/pingcap/tiflow/pkg/retry.Do\
\tgithub.com/pingcap/tiflow/pkg/retry/retry_with_opt.go:34\
github.com/pingcap/tiflow/cdc/sink.(*mysqlSink).execDMLWithMaxRetries\
\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:563\
github.com/pingcap/tiflow/cdc/sink.(*mysqlSink).execDMLs\
\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:736\
github.com/pingcap/tiflow/cdc/sink.(*mysqlSinkWorker).run.func3\
\tgithub.com/pingcap/tiflow/cdc/sink/mysql_worker.go:114\
github.com/pingcap/tiflow/cdc/sink.(*mysqlSinkWorker).run\
\tgithub.com/pingcap/tiflow/cdc/sink/mysql_worker.go:150\
github.com/pingcap/tiflow/cdc/sink.(*mysqlSink).createSinkWorkers.func1\
\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:380\
runtime.goexit\
\truntime/asm_amd64.s:1371"]
[2022/08/19 15:47:52.694 +08:00] [ERROR] [processor.go:316] ["run processor failed"] [changefeed=cbd-user-behavior-analysis-task] [capture=192.168.6.49:8301] [error="[CDC:ErrReachMaxTry]reach maximum try: 8, error: [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list': [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list'"] [errorVerbose="[CDC:ErrReachMaxTry]reach maximum try: 8, error: [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list': [CDC:ErrMySQLTxnError]MySQL txn error: Error 1054: Unknown column 'flag' in 'field list'\
github.com/pingcap/errors.AddStack\
\tgithub.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/errors.go:174\
github.com/pingcap/errors.(*Error).GenWithStackByArgs\
\tgithub.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/normalize.go:164\
github.com/pingcap/tiflow/pkg/retry.run\
\tgithub.com/pingcap/tiflow/pkg/retry/retry_with_opt.go:69\
github.com/pingcap/tiflow/pkg/retry.Do\
\tgithub.com/pingcap/tiflow/pkg/retry/retry_with_opt.go:34\
github.com/pingcap/tiflow/cdc/sink.(*mysqlSink).execDMLWithMaxRetries\
\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:563\
github.com/pingcap/tiflow/cdc/sink.(*mysqlSink).execDMLs\
\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:736\
github.com/pingcap/tiflow/cdc/sink.(*mysqlSinkWorker).run.func3\
\tgithub.com/pingcap/tiflow/cdc/sink/mysql_worker.go:114\
github.com/pingcap/tiflow/cdc/sink.(*mysqlSinkWorker).run\
\tgithub.com/pingcap/tiflow/cdc/sink/mysql_worker.go:150\
github.com/pingcap/tiflow/cdc/sink.(*mysqlSink).createSinkWorkers.func1\
\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:380\
runtime.goexit\
\truntime/asm_amd64.s:1371"]
[2022/08/19 15:47:52.694 +08:00] [INFO] [processor.go:1061] ["processor closing ..."] [changefeed=cbd-user-behavior-analysis-task]

[capture=192.168.6.49:8301] [changefeed=cbd-user-behavior-analysis-task]

日志里头提示得很清楚拉，这个changefeed 出错了嘛，检查下范围就好了？

foxchan · 2022 年8 月 22 日 07:09

这个changefeed 同步了1000个表，不提示表名或者日志打印出错的SQL，排查起来很麻烦

xfworld · 2022 年8 月 22 日 07:22

这个目前好像也没太好的办法，只能在设定changefeed 的时候，做好切分了…

或者，你找下上下文中相关的region，通过region 的 id 去获取 table id，这样会靠谱点，没其他的好办法了…

foxchan · 2022 年8 月 22 日 07:27

我是从备份SQL过滤字段排查的。希望后期cdc 能优化下报错日志