packet for query is too large. Try adjusting the 'max_allowed_packet' variable on the server"

query-status test

{
    "result": true,
    "msg": "",
    "sources": [
        {
            "result": true,
            "msg": "",
            "sourceStatus": {
                "source": "mysql-replica-01",
                "worker": "worker1",
                "result": null,
                "relayStatus": null
            },
            "subTaskStatus": [
                {
                    "name": "test",
                    "stage": "Paused",
                    "unit": "Load",
                    "result": {
                        "isCanceled": false,
                        "errors": [
                            {
                                "msg": "file abc_erp_trade.et_device_order_data.0.sql: execute statement failed: INSERT INTO `et_device_order_data` VALUES\
(HIDDENSQL...: packet for query is too large. Try adjusting the 'max_allowed_packet' variable on the server",
                                "error": null
                            }
                        ],
                        "detail": null
                    },
                    "unresolvedDDLLockID": "",
                    "load": {
                        "finishedBytes": "7607557",
                        "totalBytes": "86420927",
                        "progress": "8.80 %",
                        "metaBinlog": "(mysql-bin.000010, 716006960)"
                    }
                }
            ]
        }
    ]
}

Release Version: v2.0.0-beta.2
Git Commit Hash: e1c5376b4d3d31c353c6431d3eb9e91dc0608105
Git Branch: master
UTC Build Time: 2020-06-19 02:26:16
Go Version: go version go1.13 linux/amd64

已在 TiDB 中执行,做过相关调整

set @@global.max_allowed_packet=134217728;

task 配置

---
name: test
task-mode: all

target-database:
  host: "a.b.c.d"
  port: 4000
  user: "root"
  password: "" # 如果密码不为空,也需要配置 dmctl 加密后的密码

routes:
  route-rule-1:
    schema-pattern: "abc_erp_trade"
    target-schema: "dev_abc_erp_trade"
  route-rule-2:
    schema-pattern: "abc_system_cust"
    target-schema: "dev_abc_system_cust"

black-white-list: # 上游数据库实例匹配的表的 black & white list 过滤规则集
  bw-rule-1: # 黑白名单配置的名称
    do-dbs: ["abc_erp_trade", "abc_system_cust"] # 同步哪些库

mysql-instances:
  - source-id: "mysql-replica-01"
    black-white-list: "bw-rule-1"
    route-rules: ["route-rule-1", "route-rule-2"]
    mydumper-config-name: "global"
    mydumper-thread: 4 # mydumper 用于导出数据的线程数量,在 v1.0.2 版本引入
    loader-thread: 16 # loader 用于导入数据的线程数量,在 v1.0.2 版本引入
    syncer-thread: 16 # syncer 用于同步增量数据的线程数量,在 v1.0.2 版本引入

mydumpers:
  global:
    statement-size: 100

调整 tidb 中配置之后,是重启了 task 任务还是重做了 task 任务,还是不可行么?

https://docs.pingcap.com/zh/tidb/stable/load-misuse-handling#报错:try-adjusting-the-max_allowed_packet-variable
这个文档有参考过吗?

文档有看过。

看到相关错误报出后,我执行了以下命令。

1、修改 TiDB 参数 set @@global.max_allowed_packet=134217728
2、stop-task conf/test.yaml
3、修改 test.yaml,并将其添加到对应的 mysql-instances

mydumpers:
  global:
    statement-size: 100

4、start-task conf/test.yaml

重启 task 没有清理 checkpoint 信息吗?
如果没有清理 checkpoint 信息的话,mydumper 不会重新备份,会接着之前的断点继续

所以 statement-size 没有生效,可以清理一下断点信息,并重启 task 任务尝试下

没有清理,清理这部分是指手动清理吗?有没有相关的文档或者指令?


可以参照这个重置 task 同步任务