tiproxy节点启动失败 /sys/memory.stat 不存在

【TiDB 使用环境】测试环境
【TiDB 版本】v8.5.1
【操作系统】
【部署方式】k8s部署
Client Version: v1.29.0
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.0
[admin@ps-sale-tecbs01 sys]$
【集群数据量】
【集群节点数】只有一个work节点
【问题复现路径】
【遇到的问题:问题现象及影响】第一次启动成功了,但是使用的时候突然不行了。后面再就起不来了。tiproxy启动失败:pod/advanced-tidb-tiproxy-0 CrashLoopBackOff。tiproxy镜像用的latest版本。 报错中提到的 /sys/memory.stat 文件
【复制黏贴 ERROR 报错的日志】
[FATAL] [terror.go:300] [“unexpected error”] [error=“can’t read file memory.stat from cgroup v1: open /sys/memory.stat: no such file or directory”] [errorVerbose="open /sys/memory.stat: no such file or directory\ncan’t read file memory.stat from cgroup
【其他附件:截图/日志/监控】
完整日志:
[FATAL] [terror.go:300] [“unexpected error”] [error=“can’t read file memory.stat from cgroup v1: open /sys/memory.stat: no such file or directory”] [errorVerbose=“open /sys/memory.stat: no such file or directory\ncan’t read file memory.stat from cgroup v1\ngithub.com/pingcap/tidb/util/cgroup.detectMemStatValue\n\t/root/go/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20230103132820-3ccff46aa3bc/util/cgroup/cgroup_memory.go:162\ngithub.com/pingcap/tidb/util/cgroup.detectMemLimitInV1\n\t/root/go/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20230103132820-3ccff46aa3bc/util/cgroup/cgroup_memory.go:139\ngithub.com/pingcap/tidb/util/cgroup.getCgroupMemLimit\n\t/root/go/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20230103132820-3ccff46aa3bc/util/cgroup/cgroup_memory.go:127\ngithub.com/pingcap/tidb/util/cgroup.GetMemoryLimit\n\t/root/go/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20230103132820-3ccff46aa3bc/util/cgroup/cgroup_memory.go:32\ngithub.com/pingcap/tidb/util/memory.MemTotalCGroup\n\t/root/go/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20230103132820-3ccff46aa3bc/util/memory/meminfo.go:110\ngithub.com/pingcap/tidb/util/memory.init.0\n\t/root/go/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20230103132820-3ccff46aa3bc/util/memory/meminfo.go:159\nruntime.doInit1\n\t/usr/lib/go/src/runtime/proc.go:6740\nruntime.doInit\n\t/usr/lib/go/src/runtime/proc.go:6707\nruntime.main\n\t/usr/lib/go/src/runtime/proc.go:249\nruntime.goexit\n\t/usr/lib/go/src/runtime/asm_amd64.s:1650”] [stack=“github.com/pingcap/tidb/parser/terror.MustNil\n\t/root/go/pkg/mod/github.com/pingcap/tidb/parser@v0.0.0-20230103132820-3ccff46aa3bc/terror/terror.go:300\ngithub.com/pingcap/tidb/util/memory.init.0\n\t/root/go/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20230103132820-3ccff46aa3bc/util/memory/meminfo.go:160\nruntime.doInit1\n\t/usr/lib/go/src/runtime/proc.go:6740\nruntime.doInit\n\t/usr/lib/go/src/runtime/proc.go:6707\nruntime.main\n\t/usr/lib/go/src/runtime/proc.go:249”] [stack=“github.com/pingcap/tidb/parser/terror.MustNil\n\t/root/go/pkg/mod/github.com/pingcap/tidb/parser@v0.0.0-20230103132820-3ccff46aa3bc/terror/terror.go:300\ngithub.com/pingcap/tidb/util/memory.init.0\n\t/root/go/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20230103132820-3ccff46aa3bc/util/memory/meminfo.go:160\nruntime.doInit1\n\t/usr/lib/go/src/runtime/proc.go:6740\nruntime.doInit\n\t/usr/lib/go/src/runtime/proc.go:6707\nruntime.main\n\t/usr/lib/go/src/runtime/proc.go:249”]

tiproxy:latest 没有更新了,请使用 tiproxy:v1.3.0
报错是因为引用的 tidb 库的一个 bug(panic at cgroup when to startup · Issue #42659 · pingcap/tidb · GitHub),已被修复

2 个赞

感谢大佬 :pray:

此话题已在最后回复的 7 天后被自动关闭。不再允许新回复。