TiDB 6.4后通过S3访问阿里云OSS失败，同样配置在6.1, 5.1都可上传成功

🌞 建议反馈 产品缺陷
1 
--storage s3://test-bucket/tidb-backup-20221201?access-key=********&secret-access-key=******** --s3.provider alibaba --s3.endpoint https://oss-cn-shenzhen-internal.aliyuncs.com

之前只要这么设置，就可以在BR、DM、dumping、linghtning等工具中，用OSS做临时存储，而不用扩容本地磁盘，不过换到6.4后，同样的参数，上报时候一直报错，报错内容是：

Forbidden: Forbidden
	status code: 403, request id: 6389988AAE2EFC36368E99E8, host id:
failed to get region of bucket mysql-data-archive
github.com/pingcap/tidb/br/pkg/storage.newS3Storage
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/storage/s3.go:368
github.com/pingcap/tidb/br/pkg/storage.New
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/storage/storage.go:176
github.com/pingcap/tidb/br/pkg/task.GetStorage
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/task/common.go:631
github.com/pingcap/tidb/br/pkg/task.ReadBackupMeta
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/task/common.go:651
github.com/pingcap/tidb/br/pkg/task.RunRestore
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/task/restore.go:524
main.runRestoreCommand
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/cmd/br/restore.go:58
main.newFullRestoreCommand.func1
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/cmd/br/restore.go:143
github.com/spf13/cobra.(*Command).execute
	/go/pkg/mod/github.com/spf13/cobra@v1.5.0/command.go:872
github.com/spf13/cobra.(*Command).ExecuteC
	/go/pkg/mod/github.com/spf13/cobra@v1.5.0/command.go:990
github.com/spf13/cobra.(*Command).Execute
	/go/pkg/mod/github.com/spf13/cobra@v1.5.0/command.go:918
main.main
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/cmd/br/main.go:57
runtime.main
	/usr/local/go/src/runtime/proc.go:250
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1594
create storage failed] [stack=github.com/pingcap/log.Error
	/go/pkg/mod/github.com/pingcap/log@v1.1.1-0.20221015072633-39906604fb81/global.go:46
main.runRestoreCommand
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/cmd/br/restore.go:59
main.newFullRestoreCommand.func1
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/cmd/br/restore.go:143
github.com/spf13/cobra.(*Command).execute
	/go/pkg/mod/github.com/spf13/cobra@v1.5.0/command.go:872
github.com/spf13/cobra.(*Command).ExecuteC
	/go/pkg/mod/github.com/spf13/cobra@v1.5.0/command.go:990
github.com/spf13/cobra.(*Command).Execute
	/go/pkg/mod/github.com/spf13/cobra@v1.5.0/command.go:918
main.main
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/cmd/br/main.go:57
runtime.main
	/usr/local/go/src/runtime/proc.go:250```
2 
	// s3manager.GetBucketRegionWithClient will set credential anonymous, which works with s3.
	// we need reassign credential to be compatible with minio authentication.
	confCred := ses.Config.Credentials
	setCredOpt := func(req *request.Request) {
		if confCred != nil {
			req.Config.Credentials = confCred
		}
	}
	region, err := s3manager.GetBucketRegionWithClient(context.Background(), c, qs.Bucket, setCredOpt)
	if err != nil {
		return nil, errors.Annotatef(err, "failed to get region of bucket %s", qs.Bucket)
	}

看起来 s3manager.GetBucketRegionWithClient will set credential anonymous这个是关键，看了下6.1的版本没有使用s3manger获取region的逻辑。

1 个赞
3

额，没有人回复么。。。
翻到merge信息了：https://github.com/pingcap/tidb/pull/34419 ，看这里其实为了减少一个region参数的输入而已，如果因为这个原因， 导致不兼容国内阿里云的OSS了，感觉不太划算。官方看下能不能考虑回滚一下？

4

请稍等，我们会做一下产品层面确认，进展会及时同步。

1 个赞
5

这个问题是因为在 6.2 我们引入了自动获取 s3 bucket region 功能，aws 的 sdk 在这个功能的实现上将 S3ForcePathStyle 默认设置为 true，导致在请求阿里云 OSS 接口时生成的 URL 是 path style，进而导致请求失败。已经提了 PR，会在 6.5 上修复。

1 个赞
6

此话题已在最后回复的 60 天后被自动关闭。不再允许新回复。