BR:5.0.1
tidb:5.0.1
Error: backup lock exists, may be some backup files in the path already: [BR:Common:ErrInvalidArgument]invalid argument
测试锁文件在backup目录下backup.lock,线上环境没有这个文件
就是在报这个错，新装一下还是一样，这个lock文件在哪里，什么情况下报这个错
cmd:/deploy/tidb/tidb-toolkit-v5.0.1-linux-amd64/bin/br backup full --pd “10.8.52.53:2379” --filter ‘TPCH*.*’ --storage “local:///data01/tidb/backup/” --log-file “/data01/tidb/log/backupdb.log”
脚本白写了半天
10.8.52.62 delete /data01/tidb/backup/*
10.8.52.63 delete /data01/tidb/backup/*
10.8.52.64 delete /data01/tidb/backup/*
10.8.52.65 delete /data01/tidb/backup/*
10.8.52.66 delete /data01/tidb/backup/*
Detail BR log in /data01/tidb/log/backupdb.log
Error: backup lock exists, may be some backup files in the path already: [BR:Common:ErrInvalidArgument]invalid argument
scp 10.8.52.62
scp: /data01/tidb/backup/: No such file or directory
scp 10.8.52.63
scp: /data01/tidb/backup/: No such file or directory
scp 10.8.52.64
scp: /data01/tidb/backup/: No such file or directory
scp 10.8.52.65
scp: /data01/tidb/backup/: No such file or directory
scp 10.8.52.66
scp: /data01/tidb/backup/: No such file or directory
game over
没有问件报的错证明文件夹是空的，锁文件在哪呢
#backup tidb data
vDate=date +%Y%m%d
#集群名称
tidbName=TPCH_001
#删除远端之前的备份
echo "110.8.52.62 delete /data01/tidb/backup/"
ssh root@10.8.52.62 “rm -rf /data01/tidb/backup/" “exit”
echo "110.8.52.63 delete /data01/tidb/backup/”
ssh root@10.8.52.63 “rm -rf /data01/tidb/backup/" “exit”
echo "110.8.52.64 delete /data01/tidb/backup/”
ssh root@10.8.52.64 “rm -rf /data01/tidb/backup/" “exit”
echo "110.8.52.65 delete /data01/tidb/backup/”
ssh root@10.8.52.65 “rm -rf /data01/tidb/backup/" “exit”
echo "110.8.52.66 delete /data01/tidb/backup/”
ssh root@10.8.52.66 “rm -rf /data01/tidb/backup/*” “exit”
#backp
/deploy/tidb/tidb-toolkit-v5.0.2-linux-amd64/bin/br backup full --pd “10.8.52.54:2379” --filter "TPCH*." --storage “local:///dtata01/tidb/backup” --log-file “/data01/tidb/log/backupdb.log”
#copy data
fileName=$tidbName+‘-’+$vDate
rm -rf /tidb//data/allbackup/$fileName
mkdir -p /tidb/data/allbackup/$fileName/
echo “scp 10.8.52.62”
scp root@10.8.52.62:/data01/tidb/backup/ /data01/tidb/allbackup/$fileName/
echo “scp 10.8.52.63”
scp root@10.8.52.63:/data01/tidb/backup/* /data01/tidb/allbackup/$fileName/
echo “scp 10.8.52.64”
scp root@10.8.52.64:/data01/tidb/backup/* /data01/tidb/allbackup/$fileName/
echo “scp 10.8.52.65”
scp root@10.8.52.65:/data01/tidb/backup/* /data01/tidb/allbackup/$fileName/
echo “scp 10.8.52.66”
scp root@10.8.52.66:/data01/tidb/backup/* /data01/tidb/allbackup/$fileName/
在pd节点找到了backup.lock删了
新的错误：Error: [BR:KV:ErrKVStorage]tikv storage occur I/O error
[2021/07/01 20:32:14.706 +08:00] [ERROR] [backup.go:41] [“failed to backup”] [error=“[BR:KV:ErrKVStorage]tikv storage occur I/O error”] [errorVerbose=“[BR:KV:ErrKVStorage]tikv storage occur I/O error
github.com/pingcap/errors.AddStack
\tgithub.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/errors.go:174
github.com/pingcap/errors.Trace
\tgithub.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/juju_adaptor.go:15
github.com/pingcap/br/pkg/backup.(*Client).BackupRange
\tgithub.com/pingcap/br/pkg/backup/client.go:473
github.com/pingcap/br/pkg/backup.(*Client).BackupRanges.func1
\tgithub.com/pingcap/br/pkg/backup/client.go:424
github.com/pingcap/br/pkg/utils.(*WorkerPool).ApplyOnErrorGroup.func1
\tgithub.com/pingcap/br/pkg/utils/worker.go:63
golang.org/x/sync/errgroup.(*Group).Go.func1
\tgolang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57
runtime.goexit
\truntime/asm_amd64.s:1371”] [stack=“main.runBackupCommand
\tcommand-line-arguments/backup.go:41
main.newFullBackupCommand.func1
\tcommand-line-arguments/backup.go:109
github.com/spf13/cobra.(*Command).execute
\tgithub.com/spf13/cobra@v1.0.0/command.go:842
github.com/spf13/cobra.(*Command).ExecuteC
\tgithub.com/spf13/cobra@v1.0.0/command.go:950
github.com/spf13/cobra.(*Command).Execute
\tgithub.com/spf13/cobra@v1.0.0/command.go:887
main.main
\tcommand-line-arguments/main.go:56
runtime.main
\truntime/proc.go:225”]
[2021/07/01 20:32:14.706 +08:00] [ERROR] [main.go:58] [“br failed”] [error=“[BR:KV:ErrKVStorage]tikv storage occur I/O error”] [errorVerbose=“[BR:KV:ErrKVStorage]tikv storage occur I/O error
github.com/pingcap/errors.AddStack
\tgithub.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/errors.go:174
github.com/pingcap/errors.Trace
\tgithub.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/juju_adaptor.go:15
github.com/pingcap/br/pkg/backup.(*Client).BackupRange
\tgithub.com/pingcap/br/pkg/backup/client.go:473
github.com/pingcap/br/pkg/backup.(*Client).BackupRanges.func1
\tgithub.com/pingcap/br/pkg/backup/client.go:424
github.com/pingcap/br/pkg/utils.(*WorkerPool).ApplyOnErrorGroup.func1
\tgithub.com/pingcap/br/pkg/utils/worker.go:63
golang.org/x/sync/errgroup.(*Group).Go.func1
\tgolang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57
runtime.goexit
\truntime/asm_amd64.s:1371”] [stack=“main.main
\tcommand-line-arguments/main.go:58
runtime.main
\truntime/proc.go:225”]
当发生lock exists时，查询不到back.lock文件，但是它不知道什么时候又有了，很难捕捉
脚本也都是在测试环境搞好的，到了线上环境就问题频出
所有的backup文件都删掉了，依然是backup lock exists，这个错报的坑爹
