【 TiDB 使用环境】生产环境
【 TiDB 版本】v6.5.0
【复现路径】做过哪些操作出现的问题
【遇到的问题:问题现象及影响】
1、在原有任务中,新增忽略update语句的过滤字段表达式
[filter]
# 过滤器规则
# 过滤规则语法:https://docs.pingcap.com/zh/tidb/stable/table-filter#表库过滤语法
rules = ['maindb.es_charge_incoming_data']
# Event filter rules.
# The detailed syntax is described in <https://docs.pingcap.com/tidb/stable/ticdc-filter>
# The first event filter rule.
[[filter.event-filters]]
matcher = ["maindb.es_charge_incoming_data"] # matcher is an allow list, which means this rule only applies to the worker table in the test database.
# ignore-event = ["insert"] # Ignore insert events.
# ignore-sql = ["^drop", "add column"] # Ignore DDLs that start with "drop" or contain "add column".
# ignore-delete-value-expr = "name = 'john'" # Ignore delete DMLs that contain the condition "name = 'john'".
# ignore-insert-value-expr = "id >= 100" # Ignore insert DMLs that contain the condition "id >= 100".
# ignore-update-old-value-expr = "age < 18" # Ignore update DMLs whose old value contains "age < 18".
ignore-update-new-value-expr = "fld_is_hand_in = 1 or fld_is_hand_in = 0" # Ignore update DMLs whose new value contains "gender = 'male'".
2、更新任务
# 关闭任务
tiup cdc cli changefeed pause --pd=http://10.3.8.244:2379 --changefeed-id kafka-to-tianjin-pro-01
# 更新任务
tiup cdc cli changefeed update --pd=http://10.3.8.244:2379 --sink-uri="kafka://172.17.44.96:9092,172.17.44.97:9092,172.17.44.98:9092/charge?protocol=canal-json&kafka-version=3.0.0&replication-factor=3" --changefeed-id="kafka-to-tianjin-pro-01" --config="./kafka-to-tianjin-pro-01.toml"
# 启动任务
tiup cdc cli changefeed resume --pd=http://10.3.8.244:2379 --changefeed-id kafka-to-tianjin-pro-01
3、查看任务详情
tiup cdc cli changefeed query --pd=http://10.3.8.244:2379 --changefeed-id kafka-to-tianjin-pro-01
4、再注释过虑表达式选项
[filter]
# 过滤器规则
# 过滤规则语法:https://docs.pingcap.com/zh/tidb/stable/table-filter#表库过滤语法
rules = ['maindb.es_charge_incoming_data']
# Event filter rules.
# The detailed syntax is described in <https://docs.pingcap.com/tidb/stable/ticdc-filter>
# The first event filter rule.
#[[filter.event-filters]]
#matcher = ["maindb.es_charge_incoming_data"] # matcher is an allow list, which means this rule only applies to the worker table in the test database.
# ignore-event = ["insert"] # Ignore insert events.
# ignore-sql = ["^drop", "add column"] # Ignore DDLs that start with "drop" or contain "add column".
# ignore-delete-value-expr = "name = 'john'" # Ignore delete DMLs that contain the condition "name = 'john'".
# ignore-insert-value-expr = "id >= 100" # Ignore insert DMLs that contain the condition "id >= 100".
# ignore-update-old-value-expr = "age < 18" # Ignore update DMLs whose old value contains "age < 18".
#ignore-update-new-value-expr = "fld_is_hand_in = 1 or fld_is_hand_in = 0" # Ignore update DMLs whose new value contains "gender = 'male'".
执行第2步后,提示配置没有变化
changefeed config is the same with the old one, do nothing
真正的需求是修改后,不再过虑"fld_is_hand_in = 1 or fld_is_hand_in = 0"。
5、经过测试发现,只能remove掉这个任务,再重新创建才行。
【资源配置】进入到 TiDB Dashboard -集群信息 (Cluster Info) -主机(Hosts) 截图此页面
【附件:截图/日志/监控】