【TiDB 使用环境】测试环境
【TiDB 版本】8.5.0
【操作系统】Rocky 9.1
【部署方式】k8s
【集群数据量】10G
【集群节点数】1
【遇到的问题:问题现象及影响】
准备从6.5.5升级到8.5.0。我在本地使用kind安装了一个tidb集群。在使用br从s3进行数据恢复时,报错。数据来自于6.5.5的快照备份,我做的也是快照恢复。恢复的yanl文件如下:
【复制黏贴 ERROR 报错的日志】
[2025/03/11 06:41:45.051 +00:00] [ERROR] [main.go:38] [“br failed”] [error=“[kv:8004]Transaction is too large, size: 105426498”] [errorVerbose=“[kv:8004]Transaction is too large, size: 105426498\ngithub.com/pingcap/errors.AddStack\n\t/root/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20240318064555-6bd07397691f/errors.go:178\ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs\n\t/root/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20240318064555-6bd07397691f/normalize.go:175\ngithub.com/pingcap/tidb/pkg/store/driver/error.ToTiDBErr\n\t/workspace/source/tidb/pkg/store/driver/error/error.go:92\ngithub.com/pingcap/tidb/pkg/store/driver/txn.(*memBuffer).SetWithFlags\n\t/workspace/source/tidb/pkg/store/driver/txn/unionstore_driver.go:96\ngithub.com/pingcap/tidb/pkg/table/tblctx.(*EncodeRowBuffer).WriteMemBufferEncoded\n\t/workspace/source/tidb/pkg/table/tblctx/buffers.go:82\ngithub.com/pingcap/tidb/pkg/table/tables.(*TableCommon).addRecord\n\t/workspace/source/tidb/pkg/table/tables/tables.go:868\ngithub.com/pingcap/tidb/pkg/table/tables.(*TableCommon).AddRecord\n\t/workspace/source/tidb/pkg/table/tables/tables.go:687\ngithub.com/pingcap/tidb/pkg/executor.(*InsertValues).addRecordWithAutoIDHint\n\t/workspace/source/tidb/pkg/executor/insert_common.go:1421\ngithub.com/pingcap/tidb/pkg/executor.(*InsertValues).addRecord\n\t/workspace/source/tidb/pkg/executor/insert_common.go:1406\ngithub.com/pingcap/tidb/pkg/executor.(*InsertExec).exec\n\t/workspace/source/tidb/pkg/executor/insert.go:113\ngithub.com/pingcap/tidb/pkg/executor.insertRows\n\t/workspace/source/tidb/pkg/executor/insert_common.go:254\ngithub.com/pingcap/tidb/pkg/executor.(*InsertExec).Next\n\t/workspace/source/tidb/pkg/executor/insert.go:359\ngithub.com/pingcap/tidb/pkg/executor/internal/exec.Next\n\t/workspace/source/tidb/pkg/executor/internal/exec/executor.go:456\ngithub.com/pingcap/tidb/pkg/executor.(*ExecStmt).next\n\t/workspace/source/tidb/pkg/executor/adapter.go:1266\ngithub.com/pingcap/tidb/pkg/executor.(*ExecStmt).handleNoDelayExecutor\n\t/workspace/source/tidb/pkg/executor/adapter.go:1015\ngithub.com/pingcap/tidb/pkg/executor.(*ExecStmt).handleNoDelay\n\t/workspace/source/tidb/pkg/executor/adapter.go:848\ngithub.com/pingcap/tidb/pkg/executor.(*ExecStmt).Exec\n\t/workspace/source/tidb/pkg/executor/adapter.go:611\ngithub.com/pingcap/tidb/pkg/session.runStmt\n\t/workspace/source/tidb/pkg/session/session.go:2288\ngithub.com/pingcap/tidb/pkg/session.(*session).ExecuteStmt\n\t/workspace/source/tidb/pkg/session/session.go:2150\ngithub.com/pingcap/tidb/pkg/session.(*session).ExecuteInternal\n\t/workspace/source/tidb/pkg/session/session.go:1523\ngithub.com/pingcap/tidb/pkg/ddl/session.(*Session).Execute\n\t/workspace/source/tidb/pkg/ddl/session/session.go:85\ngithub.com/pingcap/tidb/pkg/ddl.insertDDLJobs2Table\n\t/workspace/source/tidb/pkg/ddl/job_submitter.go:699\ngithub.com/pingcap/tidb/pkg/ddl.(*JobSubmitter).GenGIDAndInsertJobsWithRetry.func1\n\t/workspace/source/tidb/pkg/ddl/job_submitter.go:444\ngithub.com/pingcap/tidb/pkg/ddl.genGIDAndCallWithRetry.func1\n\t/workspace/source/tidb/pkg/ddl/job_submitter.go:614\ngithub.com/pingcap/tidb/pkg/ddl.genGIDAndCallWithRetry\n\t/workspace/source/tidb/pkg/ddl/job_submitter.go:618\ngithub.com/pingcap/tidb/pkg/ddl.(*JobSubmitter).GenGIDAndInsertJobsWithRetry\n\t/workspace/source/tidb/pkg/ddl/job_submitter.go:424\ngithub.com/pingcap/tidb/pkg/ddl.(*JobSubmitter).addBatchDDLJobs2Table\n\t/workspace/source/tidb/pkg/ddl/job_submitter.go:351\ngithub.com/pingcap/tidb/pkg/ddl.(*JobSubmitter).addBatchDDLJobs\n\t/workspace/source/tidb/pkg/ddl/job_submitter.go:119\ngithub.com/pingcap/tidb/pkg/ddl.(*JobSubmitter).submitLoop\n\t/workspace/source/tidb/pkg/ddl/job_submitter.go:93\ngithub.com/pingcap/tidb/pkg/ddl.(*ddl).Start.func1\n\t/workspace/source/tidb/pkg/ddl/ddl.go:790\ngithub.com/pingcap/tidb/pkg/util.(*WaitGroupWrapper).Run.func1\n\t/workspace/source/tidb/pkg/util/wait_group_wrapper.go:157\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1700”] [stack=“main.main\n\t/workspace/source/tidb/br/cmd/br/main.go:38\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:272”]
Error: [kv:8004]Transaction is too large, size: 105426498
, err: exit status 1
Sleeping for 10 seconds before exit…
【其他附件:截图/日志/监控】
按照文档,我修改了环境变量 tidb_mem_quota_query 为 2个G
但依旧报错,我看报错的大小正好是100M,我就看了下配置txn-total-size-limit,这个默认值正好是100M。我就又修改了这个参数,重装了集群。但还原时还是报上面同样的错。我怀疑是不是我的参数配置的有问题,我用如下sql查询,确实没查到:
但是我的集群配置文件确实是配置了:
而且安装文档所说,这个配置项应该是不用再配置了才对啊。默认走tidb_mem_quota_query 。