tikv:9006

【 TiDB 使用环境】生产环境 /测试/ Poc
【 TiDB 版本】
【复现路径】做过哪些操作出现的问题
【遇到的问题:问题现象及影响】
【资源配置】进入到 TiDB Dashboard -集群信息 (Cluster Info) -主机(Hosts) 截图此页面
【附件:截图/日志/监控】

Tidb 报错

2024-09-13 13:30:02 (UTC+08:00)

TiDB 10.223.8.239:4000

[conn.go:1200] [“command dispatched failed”] [conn=7497020033992005059] [connInfo=“id:7497020033992005059, addr:10.223.117.100:53792 status:1, collation:utf8_general_ci, user:TEetcdConsole_tidb_rw”] [command=Query] [status=“inTxn:1, autocommit:0”] [sql=“SELECT id,instance,idc_name,env,clock,deleted,available,create_time,update_time,del_unique_key FROM etcd_monitor_tab \n \n WHERE (deleted = 0 AND idc_name = ‘officeidc_hd2’ AND env = ‘qa’ AND available = 1 AND clock > 1726205312184) ORDER BY create_time ASC”] [txn_mode=PESSIMISTIC] [timestamp=452489861479268355] [err=“[tikv:9006]GC life time is shorter than transaction duration, transaction starts at 2024-09-12 11:30:37.308 +0800 CST, GC safe point is 2024-09-13 13:13:43.958 +0800 CST\ngithub.com/pingcap/errors.AddStack\n\t/root/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20231212100244-799fae176cfb/errors.go:174\ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs\n\t/root/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20231212100244-799fae176cfb/normalize.go:164\ngithub.com/pingcap/tidb/store/driver/error.ToTiDBErr\n\t/workspace/source/tidb/store/driver/error/error.go:134\ngithub.com/pingcap/tidb/store/copr.(*kvStore).CheckVisibility\n\t/workspace/source/tidb/store/copr/store.go:47\ngithub.com/pingcap/tidb/store/copr.(*copIterator).Next\n\t/workspace/source/tidb/store/copr/coprocessor.go:1076\ngithub.com/pingcap/tidb/distsql.(*selectResult).fetchResp\n\t/workspace/source/tidb/distsql/select_result.go:370\ngithub.com/pingcap/tidb/distsql.(*selectResult).Next\n\t/workspace/source/tidb/distsql/select_result.go:438\ngithub.com/pingcap/tidb/executor.(*tableResultHandler).nextChunk\n\t/workspace/source/tidb/executor/table_reader.go:545\ngithub.com/pingcap/tidb/executor.(*TableReaderExecutor).Next\n\t/workspace/source/tidb/executor/table_reader.go:261\ngithub.com/pingcap/tidb/executor.Next\n\t/workspace/source/tidb/executor/executor.go:326\ngithub.com/pingcap/tidb/executor.(*UnionScanExec).getSnapshotRow\n\t/workspace/source/tidb/executor/union_scan.go:243\ngithub.com/pingcap/tidb/executor.(*UnionScanExec).getOneRow\n\t/workspace/source/tidb/executor/union_scan.go:195\ngithub.com/pingcap/tidb/executor.(*UnionScanExec).Next\n\t/workspace/source/tidb/executor/union_scan.go:145\ngithub.com/pingcap/tidb/executor.Next\n\t/workspace/source/tidb/executor/executor.go:326\ngithub.com/pingcap/tidb/executor.(*ProjectionExec).unParallelExecute\n\t/workspace/source/tidb/executor/projection.go:196\ngithub.com/pingcap/tidb/executor.(*ProjectionExec).Next\n\t/workspace/source/tidb/executor/projection.go:183\ngithub.com/pingcap/tidb/executor.Next\n\t/workspace/source/tidb/executor/executor.go:326\ngithub.com/pingcap/tidb/executor.(*SortExec).fetchRowChunks\n\t/workspace/source/tidb/executor/sort.go:198\ngithub.com/pingcap/tidb/executor.(*SortExec).Next\n\t/workspace/source/tidb/executor/sort.go:116\ngithub.com/pingcap/tidb/executor.Next\n\t/workspace/source/tidb/executor/executor.go:326\ngithub.com/pingcap/tidb/executor.(*ExecStmt).next\n\t/workspace/source/tidb/executor/adapter.go:1202\ngithub.com/pingcap/tidb/executor.(*recordSet).Next\n\t/workspace/source/tidb/executor/adapter.go:151\ngithub.com/pingcap/tidb/server.(*tidbResultSet).Next\n\t/workspace/source/tidb/server/driver_tidb.go:469\ngithub.com/pingcap/tidb/server.(*clientConn).writeChunks\n\t/workspace/source/tidb/server/conn.go:2352\ngithub.com/pingcap/tidb/server.(*clientConn).writeResultSet\n\t/workspace/source/tidb/server/conn.go:2295\ngithub.com/pingcap/tidb/server.(*clientConn).handleStmt\n\t/workspace/source/tidb/server/conn.go:2163\ngithub.com/pingcap/tidb/server.(*clientConn).handleQuery\n\t/workspace/source/tidb/server/conn.go:1901\ngithub.com/pingcap/tidb/server.(*clientConn).dispatch\n\t/workspace/source/tidb/server/conn.go:1388\ngithub.com/pingcap/tidb/server.(*clientConn).Run\n\t/workspace/source/tidb/server/conn.go:1169\ngithub.com/pingcap/tidb/server.(*Server).onConn\n\t/workspace/source/tidb/server/server.go:718\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1598”]

看这个错误,是有长事务?持续一天?没关?

GC生命周期比事务持续时间短,看一下是什么长事务在跑,需要这么久吗,如果需要的话,加大GC时间

怀疑是业务框架问题,让业务自己排查吧~

https://github.com/pingcap/tidb/issues/24640

活跃事务阻塞 gc 时间最长由 tidb_gc_max_wait_time 参数控制,你这个事务时间太长了
https://docs.pingcap.com/zh/tidb/stable/system-variables#tidb_gc_max_wait_time-从-v610-版本开始引入

1 个赞

事务时间长,要么调gc要么改sql

show processlist看下当前还有哪些长事务在运行,如果当前没有,那看下慢日志,找出这个SQL语句

长事务会阻塞GC,提交事务默认24h阻塞GC的推进就超时了。