增加表的column后dm 同步异常

为提高效率,提问时请提供以下信息,问题描述清晰可优先响应。

  • 【TiDB 版本】:3.0.5
  • 【问题描述】: 我们这边又分库合表使用dm同步mysql到tidb; dm的配置中设置了:is-sharding: true 今天mysql增加了表的column;增加后发现dm同步任务中部分dm instance的同步报错了,提示如下: { “id”: 140, “name”: “sharding table futures_streamer.order_history consistency checking”, “desc”: “check consistency of sharding table structures”, “state”: “fail”, “errorMsg”: “column length mismatch (28 vs 31)\ table futures_streamer.order_history\ columns user_id bigint(20)\ order_id bigint(20)\ symbol varchar(10)\ account_id bigint(20)\ orig_client_order_id varchar(100)\ client_order_id varchar(100)\ price decimal(20,8)\ orig_qty decimal(20,8)\ executed_qty decimal(20,8)\ executed_quote_qty decimal(20,8)\ iceberg_qty decimal(20,8)\ status varchar(50)\ time_in_force varchar(10)\ type varchar(30)\ side varchar(10)\ stop_price decimal(20,8)\ target_strategy int(11)\ liquidation_cp_type tinyint(4)\ insert_time datetime\ create_update_id bigint(20)\ update_time datetime\ update_id bigint(20)\ order_list_id bigint(20)\ working_type tinyint(4)\ working_time bigint(20)\ working_indicator tinyint(1)\ db_create_time datetime(3)\ db_modify_time datetime(3)\ \ tablefutures_streamer.order_history\ columns user_id bigint(20)\ order_id bigint(20)\ symbol varchar(100)\ position_side tinyint(4)\ account_id bigint(20)\ orig_client_order_id varchar(100)\ client_order_id varchar(100)\ price decimal(20,8)\ orig_qty decimal(20,8)\ executed_qty decimal(20,8)\ executed_quote_qty decimal(20,8)\ iceberg_qty decimal(20,8)\ status varchar(50)\ time_in_force varchar(10)\ type varchar(30)\ side varchar(10)\ stop_price decimal(20,8)\ activate_price decimal(20,8)\ price_rate int(11)\ target_strategy int(11)\ liquidation_cp_type tinyint(4)\ insert_time datetime\ create_update_id bigint(20)\ update_time datetime\ update_id bigint(20)\ order_list_id bigint(20)\ working_type tinyint(4)\ working_time bigint(20)\ working_indicator tinyint(1)\ db_create_time datetime(3)\ db_modify_time datetime(3)\ ”,

登录到tidb发现mysql增加的column没有在tidb表中。 手动在tidb 执行 alter table add column后;重启dm task 发现启动不了了。报错:

                    {
                            "id": 140,
                            "name": "sharding table `futures_streamer`.`order_history` consistency checking",
                            "desc": "check consistency of sharding table structures",
                            "state": "fail",
                            "errorMsg": "column length mismatch (28 vs 31)\
 table `futures_streamer`.`order_history`\
columns user_id bigint(20)\
order_id bigint(20)\
symbol varchar(10)\
account_id bigint(20)\
orig_client_order_id varchar(100)\
client_order_id varchar(100)\
price decimal(20,8)\
orig_qty decimal(20,8)\
executed_qty decimal(20,8)\
executed_quote_qty decimal(20,8)\
iceberg_qty decimal(20,8)\
status varchar(50)\
time_in_force varchar(10)\
type varchar(30)\
side varchar(10)\
stop_price decimal(20,8)\
target_strategy int(11)\
liquidation_cp_type tinyint(4)\
insert_time datetime\
create_update_id bigint(20)\
update_time datetime\
update_id bigint(20)\
order_list_id bigint(20)\
working_type tinyint(4)\
working_time bigint(20)\
working_indicator tinyint(1)\
db_create_time datetime(3)\
db_modify_time datetime(3)\
\
table`futures_streamer`.`order_history`\
columns user_id bigint(20)\
order_id bigint(20)\
symbol varchar(100)\
position_side tinyint(4)\
account_id bigint(20)\
orig_client_order_id varchar(100)\
client_order_id varchar(100)\
price decimal(20,8)\
orig_qty decimal(20,8)\
executed_qty decimal(20,8)\
executed_quote_qty decimal(20,8)\
iceberg_qty decimal(20,8)\
status varchar(50)\
time_in_force varchar(10)\
type varchar(30)\
side varchar(10)\
stop_price decimal(20,8)\
activate_price decimal(20,8)\
price_rate int(11)\
target_strategy int(11)\
liquidation_cp_type tinyint(4)\
insert_time datetime\
create_update_id bigint(20)\
update_time datetime\
update_id bigint(20)\
order_list_id bigint(20)\
working_type tinyint(4)\
working_time bigint(20)\
working_indicator tinyint(1)\
db_create_time datetime(3)\
db_modify_time datetime(3)\
",
                            "instruction": "please set same table structure for sharding tables",
                            "extra": "instance futures-streamer6 on sharding `futures_streamer`.`order_history`"
                    },

请问目前这种问题该如何处理? 遇到类似的ddl问题,正确的处理过程应该是什么样的?万分感谢!!!

若提问为性能优化、故障排查类问题,请下载脚本运行。终端输出的打印结果,请务必全选并复制粘贴上传。

确认一下上下游的 Tables schema 是否一致,另外默认同步表会同步 DDL 操作,不清楚你的配置是否 skip DDL 操作。

您好: 根据报错看,sharding table好像不一致,你说使用了分库合表,请问上游mysql的分表同一都增加列了吗? 请检查一下,多谢。

此话题已在最后回复的 1 分钟后被自动关闭。不再允许新回复。