为提高效率,请提供以下信息,问题描述清晰能够更快得到解决:
【TiDB 版本】
v4.0.9
【问题描述】
使用如下语句
"./dumpling -u qdtech -P 4000 -h 127.0.0.1 --filetype sql --threads 1 -o /mnt/tidb-data/20210502/databases -r 10000 -F 256MiB -pWRnYAc8mL3uE04Hf90x5 -T databases --snapshot “2021-05-02 08:00:00” --tidb-mem-quota-query 4294967296 & "
导出了1.9T的SQL,导出成功无报错,导出的TiDB数据库版本为v4.0.9,导入的新的TiDB集群产生的报错,新的TiDB集群为云厂商的TiDB集群,只提供TiDB的IP、端口、用户名、密码,版本为:v4.0.8-dirty
报错信息如下:
[2021/05/07 11:37:14.311 +08:00] [INFO] [restore.go:1190] [“You may also run ./tidb-lightning-ctl --checkpoint-error-destroy=all --config=...
to start from scratch”]
[2021/05/07 11:37:14.311 +08:00] [INFO] [restore.go:1191] [“For details of this failure, read the log file from the PREVIOUS run”]
[2021/05/07 11:37:14.311 +08:00] [ERROR] [restore.go:331] [“run failed”] [step=3] [error=“TiDB Lightning has failed last time; please resolve these errors first”]
[2021/05/07 11:37:14.311 +08:00] [INFO] [restore.go:955] [“everything imported, stopping periodic actions”]
[2021/05/07 11:37:14.311 +08:00] [ERROR] [restore.go:342] [“the whole procedure failed”] [takeTime=4.062372998s] [error=“TiDB Lightning has failed last time; please resolve these errors first”]
[2021/05/07 11:37:14.312 +08:00] [ERROR] [main.go:90] [“tidb lightning encountered error stack info”] [error=“TiDB Lightning has failed last time; please resolve these errors first”] [errorVerbose=“TiDB Lightning has failed last time; please resolve these errors first\ngithub.com/pingcap/br/pkg/lightning/restore.(*Controller).restoreTables\
\tgithub.com/pingcap/br@/pkg/lightning/restore/restore.go:1193\
github.com/pingcap/br/pkg/lightning/restore.(*Controller).Run\
\tgithub.com/pingcap/br@/pkg/lightning/restore/restore.go:318\
github.com/pingcap/br/pkg/lightning.(*Lightning).run\
\tgithub.com/pingcap/br@/pkg/lightning/lightning.go:315\
github.com/pingcap/br/pkg/lightning.(*Lightning).RunOnce\
\tgithub.com/pingcap/br@/pkg/lightning/lightning.go:191\
main.main.func2\
\tgithub.com/pingcap/br@/cmd/tidb-lightning/main.go:86\
main.main\
\tgithub.com/pingcap/br@/cmd/tidb-lightning/main.go:87\
runtime.main\
\truntime/proc.go:203\
runtime.goexit\
\truntime/asm_amd64.s:1357”]
[2021/05/07 11:37:14.312 +08:00] [ERROR] [main.go:91] [“tidb lightning encountered error”] [error=“TiDB Lightning has failed last time; please resolve these errors first”]
配置文件信息如下:
[lightning]
level = “info”
file = “tidb-lightning.log”
status-addr = ‘:8289’
region-concurrency = 4
check-requirements = true
[checkpoint]
enable = true
schema = “tidb_lightning_checkpoint”
[mydumper]
data-source-dir = “/mnt/tidb-data/20210502/databases/”
read-block-size = 65536
batch-import-ratio = 0.75
character-set = “auto”
strict-format = false
[tidb]
host = “192.169.171.212”
port = 6033
status-port = 10080
user = “myuser”
password = “WRnYAc8mL3uE04Hf90x5”
log-level = “error”
sql-mode = “”
max-allowed-packet = 0
[tikv-importer]
backend = “tidb”
on-duplicate = “ignore”
[post-restore]
checksum = false
level-1-compact = false
analyze = false
[cron]
switch-mode = “5m”
log-progress = “5m”
恢复的命令如下:
#!/bin/bash
nohup ./tidb-lightning -config tidb-lightning.toml > nohup.out &
编辑成为一个shell脚本,使用shell脚本恢复
备份出来了四个数据库,一个500MB,一个300MB,数据量小的已经恢复到新的TIDB集群,但是这个大的数据库一直卡在370G左右的时候报错。
恢复这个大的数据库的时候有些建表语句出现报错:
ERROR 1101 (42000): BLOB/TEXT/JSON column ‘suy_config’ can’t have a default value
ERROR 1067 (42000): Invalid default value for ‘uptime’
我都已经按照提示,去掉默认值,以及将默认的时间改为”1970-1-1“,改掉之后的建表语句成功执行无报错。
这里报错的表也不是修改建表语句后建的表。
这里的备份、恢复服务器的配置为4核8G内存3T存储的VM。
若提问为性能优化、故障排查类问题,请下载脚本运行。终端输出的打印结果,请务必全选并复制粘贴上传。