BR如何备份到阿里云OSS

Hacker_BCi747nd · 2021 年11 月 18 日 09:38

背景：准备通过BR工具，将数据备份到阿里云OSS上(兼容S3接口)，按照官方文档的配置，发现还是访问的AWS S3接口，没办法备份成功。执行命令如下：

export AWS_ACCESS_KEY_ID="" 
export AWS_SECRET_ACCESS_KEY="" 

tiup br backup full \
	--send-credentials-to-tikv=true \
    --pd "172.18.243.13:2379" \
    --storage "s3://mysql-data-archive/asset-tidb-backup-20211118" \
    --s3.provider alibaba \
    --s3.endpoint "http://oss-cn-shenzhen-internal.aliyuncs.com" \
    --s3.region "cn-shenzhen" \
    --ratelimit 32 \
    --log-file backupfull.log

打印日志为：

[2021/11/18 17:30:04.747 +08:00] [INFO] [info.go:40] ["Welcome to Backup & Restore (BR)"] [release-version=v5.1.0] [git-hash=f2fcbfa5a9783e33ff44b696920d8bbe6a19eaa6] [git-branch=heads/refs/tags/v5.1.0] [go-version=go1.16.4] [utc-build-time="2021-06-24 07:09:26"] [race-enabled=false]
[2021/11/18 17:30:04.747 +08:00] [INFO] [common.go:502] [arguments] [__command="br backup full"] [log-file=backupfull.log] [pd="[172.18.243.13:2379]"] [ratelimit=32] [s3.endpoint=http://oss-cn-shenzhen-internal.aliyuncs.com] [s3.provider=alibaba] [s3.region=cn-shenzhen] [send-credentials-to-tikv=true] [storage=s3://mysql-data-archive/asset-tidb-backup-20211118]
[2021/11/18 17:30:04.747 +08:00] [INFO] [client.go:214] ["[pd] create pd client with endpoints"] [pd-address="[172.18.243.13:2379]"]
[2021/11/18 17:30:04.754 +08:00] [INFO] [base_client.go:346] ["[pd] update member urls"] [old-urls="[http://172.18.243.13:2379]"] [new-urls="[http://172.18.243.13:2379,http://172.18.244.16:2379,http://172.18.244.17:2379]"]
[2021/11/18 17:30:04.754 +08:00] [INFO] [base_client.go:365] ["[pd] switch leader"] [new-leader=http://172.18.243.13:2379] [old-leader=]
[2021/11/18 17:30:04.754 +08:00] [INFO] [base_client.go:126] ["[pd] init cluster id"] [cluster-id=6984216311785899551]
[2021/11/18 17:30:04.754 +08:00] [INFO] [client.go:238] ["[pd] create tso dispatcher"] [dc-location=global]
[2021/11/18 17:30:04.759 +08:00] [INFO] [client.go:214] ["[pd] create pd client with endpoints"] [pd-address="[172.18.243.13:2379]"]
[2021/11/18 17:30:04.766 +08:00] [INFO] [base_client.go:346] ["[pd] update member urls"] [old-urls="[http://172.18.243.13:2379]"] [new-urls="[http://172.18.243.13:2379,http://172.18.244.16:2379,http://172.18.244.17:2379]"]
[2021/11/18 17:30:04.766 +08:00] [INFO] [base_client.go:365] ["[pd] switch leader"] [new-leader=http://172.18.243.13:2379] [old-leader=]
[2021/11/18 17:30:04.766 +08:00] [INFO] [base_client.go:126] ["[pd] init cluster id"] [cluster-id=6984216311785899551]
[2021/11/18 17:30:04.766 +08:00] [INFO] [client.go:238] ["[pd] create tso dispatcher"] [dc-location=global]
[2021/11/18 17:30:04.768 +08:00] [INFO] [conn.go:205] ["new mgr"] [pdAddrs=172.18.243.13:2379]
[2021/11/18 17:30:04.769 +08:00] [INFO] [client.go:93] ["new backup client"]
[2021/11/18 17:30:05.770 +08:00] [WARN] [s3.go:733] ["failed to request s3, retrying"] [error="RequestError: send request failed\
caused by: Put \"http://169.254.169.254/latest/api/token\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"] [backoff=1.158468992s]
[2021/11/18 17:30:07.929 +08:00] [WARN] [s3.go:733] ["failed to request s3, retrying"] [error="RequestError: send request failed\
caused by: Put \"http://169.254.169.254/latest/api/token\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"] [backoff=2.242699196s]
[2021/11/18 17:30:11.173 +08:00] [WARN] [s3.go:733] ["failed to request s3, retrying"] [error="RequestError: send request failed\
caused by: Put \"http://169.254.169.254/latest/api/token\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"] [backoff=7.005792392s]
[2021/11/18 17:30:17.048 +08:00] [WARN] [main.go:30] ["received signal to exit"] [signal=interrupt]
[2021/11/18 17:30:17.048 +08:00] [INFO] [client.go:510] ["[pd] exit tso dispatcher"] [dc-location=global]

Hacker_BCi747nd · 2021 年11 月 18 日 09:40

网上搜了下，这个IP是AWS的地址，说明根本没有访问到阿里云的endpoint

Billmay表妹 · 2021 年11 月 18 日 10:34

参考下~

hey-hoho · 2021 年11 月 18 日 10:38

还可以参考下这个：
https://asktug.com/t/topic/153129

Hacker_BCi747nd · 2021 年11 月 19 日 02:08

感谢两位的推荐，不过这三篇我都看了，和我遇到的问题还不一样。
上边的两个案例都是能访问到对应的远端存储，不过我按照文档的配置，报错的IP根本不是阿里云的IP，搜了一下是AWS的一个END_POINT的IP，所以感觉还是配置问题。
可以帮忙看下配置是否有问题么？

hey-hoho · 2021 年11 月 19 日 03:33

你把–s3.provider 的值加上引号试试

Hacker_BCi747nd · 2021 年11 月 19 日 06:27

还真的是，全部加上引号，就可以了

tiup br backup full \
	--send-credentials-to-tikv=true \
    --pd "172.18.243.13:2379" \
    --storage "s3://mysql-data-archive/asset-tidb-backup-20211118" \
    --s3.provider "alibaba" \
    --s3.endpoint "https://oss-cn-shenzhen-internal.aliyuncs.com" \
    --ratelimit 32 \
    --log-file backupfull.log

hey-hoho · 2021 年11 月 19 日 06:32

Billmay表妹 · 2021 年11 月 19 日 07:20

赞~~

system · 2022 年10 月 31 日 19:06

此话题已在最后回复的 1 分钟后被自动关闭。不再允许新回复。