【 TiDB 使用环境】测试
【 TiDB 版本】 7.1.0
【复现路径】通过创建bks对象对tidb进行备份
【遇到的问题:问题现象及影响】
问题:
1、bks调用br备份时备份失败,手动调用br可以备份成功
2、容器中已经存在libresolv.so.2这个库文件了,为什么bks在备份的时候还报找不到这个文件
br备份报错日志:
[root@vm10-0-2-6 ~]# kubectl logs backup-4494cccb-9cda-4a85-805f-9f62b78dec62-vzdvg -n $ns
Create rclone.conf file.
/tidb-backup-manager backup --namespace=b0b996d0-4c67-4c13-be35-c712c7162c05 --backupName=4494cccb-9cda-4a85-805f-9f62b78dec62 --tikvVersion=v7.1.0
I0705 10:58:26.075002 9 backup.go:72] start to process backup b0b996d0-4c67-4c13-be35-c712c7162c05/4494cccb-9cda-4a85-805f-9f62b78dec62
I0705 10:58:27.651814 9 backup_status_updater.go:86] Backup: [b0b996d0-4c67-4c13-be35-c712c7162c05/4494cccb-9cda-4a85-805f-9f62b78dec62] updated successfully
E0705 10:58:27.654504 9 backup_status_updater.go:89] Failed to update backup [b0b996d0-4c67-4c13-be35-c712c7162c05/4494cccb-9cda-4a85-805f-9f62b78dec62], error: Operation cannot be fulfilled on backups.pingcap.com “4494cccb-9cda-4a85-805f-9f62b78dec62”: the object has been modified; please apply your changes to the latest version and try again
I0705 10:58:27.686458 9 backup_status_updater.go:86] Backup: [b0b996d0-4c67-4c13-be35-c712c7162c05/4494cccb-9cda-4a85-805f-9f62b78dec62] updated successfully
I0705 10:58:27.686491 9 backup.go:69] Running br command with args: [backup full --pd=basic-pd.b0b996d0-4c67-4c13-be35-c712c7162c05:2379 --send-credentials-to-tikv=true --storage=s3://tidb-ks3-bucket/2000104981/b0b996d0-4c67-4c13-be35-c712c7162c05/manual/4494cccb-9cda-4a85-805f-9f62b78dec62 --s3.region=BEIJING --s3.provider=other --s3.endpoint=http://ks3-cn-beijing-internal.ksyun.com]
I0705 10:58:27.687946 9 backup.go:93]
I0705 10:58:27.687971 9 backup.go:100] Error loading shared library libresolv.so.2: No such file or directory (needed by /var/lib/br-bin/br)
E0705 10:58:27.688042 9 manager.go:293] backup cluster b0b996d0-4c67-4c13-be35-c712c7162c05/4494cccb-9cda-4a85-805f-9f62b78dec62 data failed, err: cluster b0b996d0-4c67-4c13-be35-c712c7162c05/4494cccb-9cda-4a85-805f-9f62b78dec62, wait pipe message failed, errMsg Error loading shared library libresolv.so.2: No such file or directory (needed by /var/lib/br-bin/br)
, err: exit status 127
E0705 10:58:27.697283 9 backup_status_updater.go:89] Failed to update backup [b0b996d0-4c67-4c13-be35-c712c7162c05/4494cccb-9cda-4a85-805f-9f62b78dec62], error: Operation cannot be fulfilled on backups.pingcap.com “4494cccb-9cda-4a85-805f-9f62b78dec62”: the object has been modified; please apply your changes to the latest version and try again
I0705 10:58:27.713958 9 backup_status_updater.go:86] Backup: [b0b996d0-4c67-4c13-be35-c712c7162c05/4494cccb-9cda-4a85-805f-9f62b78dec62] updated successfully
error: cluster b0b996d0-4c67-4c13-be35-c712c7162c05/4494cccb-9cda-4a85-805f-9f62b78dec62, wait pipe message failed, errMsg Error loading shared library libresolv.so.2: No such file or directory (needed by /var/lib/br-bin/br)
, err: exit status 127
【资源配置】
tidb-operator版本为1.3.3
tidb、pd、tikv配置8核、20G
【附件:截图/日志/监控】
bks配置:
apiVersion: v1
items:
- apiVersion: pingcap.com/v1alpha1
kind: BackupSchedule
metadata:
creationTimestamp: “2023-07-03T02:39:23Z”
generation: 19
name: basic-backup-schedule
namespace: b0b996d0-4c67-4c13-be35-c712c7162c05
resourceVersion: “527849164”
selfLink: /apis/pingcap.com/v1alpha1/namespaces/b0b996d0-4c67-4c13-be35-c712c7162c05/backupschedules/basic-backup-schedule
uid: d0446128-18aa-4f58-8e99-ef5e0b1326b1
spec:
backupTemplate:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: failure-domain.beta.kubernetes.io/zone
operator: In
values:
- cn-qingyangtest-1a
backupType: full
br:
cluster: basic
clusterNamespace: b0b996d0-4c67-4c13-be35-c712c7162c05
sendCredToTikv: true
cleanPolicy: Retain
resources: {}
s3:
bucket: tidb-ks3
endpoint: http://ks3-cn.com
prefix: 2000104981/b0b996d0-4c67-
provider: other
region: BEIJING
secretName: ks3-secret
imagePullSecrets:- name: nosql-image
maxBackups: 0
schedule: 58 10 * * *
storageClassName: openebs-hostpath
status:
lastBackup: 4494cccb-9cda-4a85-805f-9f62b78dec62
lastBackupTime: “2023-07-05T02:58:00Z”
kind: List
metadata:
resourceVersion: “”
selfLink: “”
- name: nosql-image