使用 mydumper loader 进行备份和恢复,从mysql 5.6导出 到mysql 8.0 ,有大库大表;单表有上百G;
/soft/tidb-tools/bin/mydumper --version
mydumper 0.9.5 (ba3946edc087e87bba7cb2728bc2407f6dbd0a8b), built against MySQL 5.7.24
/soft/tidb-tools/bin/loader
Release Version: v1.0.0-78-g6aea485
Git Commit Hash: 6aea4851bb0c6e599c64b5c952ce257863c21586
Git Branch: master
UTC Build Time: 2019-12-18 04:25:49
Go Version: go version go1.13 linux/amd64
导出的配置语句:
/mydumper -u root -p ‘xxx’ -h xxxx -P 13360 -t 16 -F 128 -l 72000 --skip-tz-utc -o /data/xx…all/
导入,
[info] config: {“log-level”:“info”,“log-file”:“/home/。。。/history7.log”,“status-addr”:“:8272”,“pool-size”:16,“dir”:“/data/。。。/history2/”,“db”:{“host”:“1xxxx”,“user”:“xxxx”,“port”:10,“sql-mode”:“”,“m
ax-allowed-packet”:134217728},“checkpoint-schema”:“tidb_loader”,“config-file”:“load10.toml”,“route-rules”:null,“do-table”:null,“do-db”:null,“ignore-table”:null,“ignore-db”:null,“rm-checkpoint”:false}
感觉loader的参数已经中规中矩按照官网又是大表,maxpacker都已经调128M最大了是吧,但出现的现象是运行两小时左右就仍然报invalid connections 需要重启loader 如此循环;请问是哪方面原因呢?
thread 我这边参数: innodb_buffer_pool_size = 51539607552 即48G table_open_cache 18000
±------------------±------+
| Variable_name | Value |
±------------------±------+
| Threads_cached | 7 |
| Threads_connected | 17 |
| Threads_created | 24 |
| Threads_running | 18 |
±------------------±------+
下面看运行截图,
1
2
3
请问各路大侠,我还要怎么调? mysql8.0 innodb;cpu 40核 内存一百多G;
若提问为性能优化、故障排查类问题,请下载脚本运行。终端输出的打印结果,请务必全选并复制粘贴上传。