tidb-server cpu飙高

TiDBer_ahdDbR5B · 2022 年4 月 28 日 11:06

show processlist里面都是autocommit，这个应该没有影响吧

h5n1 · 2022 年4 月 28 日 11:08

6.0版本才支持，auto analyze 用show processlist也是看不到的

h5n1 · 2022 年4 月 28 日 11:13

把这个tidb实例在突增前后的tidb.log日志拿下

TiDBer_ahdDbR5B · 2022 年4 月 28 日 11:26

好的，正在传

TiDBer_ahdDbR5B · 2022 年4 月 28 日 11:49

上传中：tidb-2022-04-27T16-59-18.011.log…

TiDBer_ahdDbR5B · 2022 年4 月 28 日 11:54

归档.zip

TiDBer_ahdDbR5B · 2022 年4 月 28 日 11:55

有看到文件吗，好像上传不上去

TiDBer_ahdDbR5B · 2022 年4 月 28 日 12:19

归档.zip (38.7 MB)

h5n1 · 2022 年4 月 29 日 01:21

把tikv的监控也上传一下，日志没看出啥来，看监控图25日17:30左右开始出问题，整理了几个报错，感觉像是错误积累到一定程度后爆发的，还得请研发大佬们看看
另外把cpu的Profile弄下，也可以使用dashboard的高级调试 - 实例性能调试功能

[2022/04/25 16:59:54.598 +08:00] [ERROR] [terror.go:307] [“encountered error”] [error=EOF] [stack=“github.com/pingcap/tidb/parser/terror.Log\ \t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/parser/terror/terror.go:307\ github.com/pingcap/tidb/server.(*Server).onConn\ \t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/server.go:516”]
[2022/04/25 16:59:55.291 +08:00] [WARN] [client_batch.go:794] [“wait response is cancelled”] [to=10.8.52.174:20160] [cause=“context canceled”]

[2022/04/25 17:00:01.900 +08:00] [WARN] [session.go:1721] [“run statement failed”] [conn=755329] [schemaVersion=2933] [error="[variable:1238]Variable ‘optimizer_switch’ is a read only variable"]
[session="{\ “currDBName”: “”,\ “id”: 755329,\ “status”: 2,\ “strictMode”: false,\ “user”: {\ “Username”: “root”,\ “Hostname”: “10.8.52.239”,\ “CurrentUser”: false,\
“AuthUsername”: “root”,\ “AuthHostname”: “%”\ }\ }"]

[2022/04/25 17:30:01.695 +08:00] [INFO] [conn.go:1115] [“command dispatched failed”] [conn=756981] [connInfo=“id:756981, addr:10.8.52.239:55009 status:10, collation:utf8_general_ci, user:root”] [command=Query] [status=“inTxn:0, autocommit:1”] [sql=“set optimizer_switch=‘semijoin=off’”] [txn_mode=PESSIMISTIC] [err="[variable:1238]Variable ‘optimizer_switch’ is a read only variable"]
[2022/04/25 17:30:01.695 +08:00] [WARN] [session.go:1721] [“run statement failed”] [conn=756979] [schemaVersion=2933] [error="[variable:1238]Variable ‘optimizer_switch’ is a read only variable"] [session="{\ “currDBName”: “”,\ “id”: 756979,\ “status”: 2,\ “strictMode”: false,\ “user”: {\ “Username”: “root”,\ “Hostname”: “10.8.52.239”,\ “CurrentUser”: false,\ “AuthUsername”: “root”,\ “AuthHostname”: “%”\ }\ }"]
[2022/04/25 17:30:01.695 +08:00] [INFO] [conn.go:1115] [“command dispatched failed”] [conn=756979] [connInfo=“id:756979, addr:10.8.52.239:55008 status:10, collation:utf8_general_ci, user:root”] [command=Query] [status=“inTxn:0, autocommit:1”] [sql=“set optimizer_switch=‘semijoin=off’”] [txn_mode=PESSIMISTIC] [err="[variable:1238]Variable ‘optimizer_switch’ is a read only variable"]
[2022/04/25 17:30:01.739 +08:00] [INFO] [tidb.go:260] [“rollbackTxn called due to ddl/autocommit failure”]

[2022/04/25 17:33:31.393 +08:00] [WARN] [pd.go:152] [“get timestamp too slow”] [“cost time”=55.221256ms]