【问题澄清】
- 通过 DM v1.0.6 将数据从 MySQL 同步到 TiDB，其中单表存在大字段，logtext
- dump 阶段报错，其中 dm-worker log 如下：
[2020/07/13 12:24:14.134 +08:00] [ERROR] [mydumper.go:170] [“Could not read data from oms.so_attachment: Lost connection to MySQL server during query”] [task=so_attachment] [unit=dump]
[2020/07/13 12:24:14.147 +08:00] [INFO] [mydumper.go:164] [“Thread 1 shutting down”] [task=so_attachment] [unit=dump]
[2020/07/13 12:24:14.147 +08:00] [INFO] [mydumper.go:164] [“Finished dump at: 2020-07-13 12:24:14”] [task=so_attachment] [unit=dump]
[2020/07/13 12:24:14.642 +08:00] [ERROR] [mydumper.go:118] [“dump data exits with error”] [task=so_attachment] [unit=dump] [“cost time”=1h56m9.983838484s] [error=“msg:”[code=32001:class=dump-unit:scope=internal:level=high] mydumper runs with error: exit status 1.
” “]
【解决方案】
- 对于单表特别大，占用内存较多，mydumer 提供 -r --row 参数。
- 在 task 文件中 mydumpers - extra-args 添加 --row rows 解决。
【参数说明】
./bin/mydumper --help |grep ‘\-\-rows’
-r, --rows Try to split tables into chunks of this many rows. This option turns off --chunk-filesize