执行analyze table报错 ：invalid memory address or nil pointer dereference

🪐 TiDB 技术问题
1

【概述】执行analyze table 报错
【现象】报错无法执行
【TiDB 版本】 V4.0.8
已经执行过重启组件
tiup cluster restart xxxxxxxx-R tikv
tiup cluster restart xxxxxxxx-R tidb
报错依旧

QQ%E6%88%AA%E5%9B%BE20211028115431

[2021/10/28 11:45:54.247 +08:00] [ERROR] [conn.go:717] ["connection running loop panic"] [conn=319] [lastSQL="ANALYZE table xxxxxx"] [err="runtime error: invalid memory address or nil pointer dereference"] [stack="goroutine 331391 [running]:\
github.com/pingcap/tidb/server.(*clientConn).Run.func1(0x3778a60, 0xc0217571d0, 0xc01af791d0)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/server/conn.go:715 +0xee\
panic(0x2e82400, 0x4d78fb0)\
\t/usr/local/go/src/runtime/panic.go:679 +0x1b2\
github.com/pingcap/tidb/executor.(*ExecStmt).Exec.func1(0xc013b84840, 0xc027b5a7a8, 0xc027b5a788)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/executor/adapter.go:288 +0x54b\
panic(0x2e82400, 0x4d78fb0)\
\t/usr/local/go/src/runtime/panic.go:679 +0x1b2\
github.com/pingcap/tidb/statistics.(*HistColl).getEqualCondSelectivity(0xc027adc500, 0xc002e465a0, 0xc027aa42a0, 0x1b, 0x1b, 0x3, 0x3)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/statistics/table.go:434 +0x1e3\
github.com/pingcap/tidb/statistics.(*HistColl).getIndexRowCount(0xc027adc500, 0xc02726d340, 0x1, 0xc020e9a230, 0x1, 0x1, 0x4, 0x3, 0xc05658eae0)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/statistics/table.go:472 +0xa82\
github.com/pingcap/tidb/statistics.(*HistColl).GetRowCountByIndexRanges(0xc027adc500, 0xc02726d340, 0x1, 0xc020e9a230, 0x1, 0x1, 0x4, 0x4, 0xc027aa4260)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/statistics/table.go:279 +0x2b5\
github.com/pingcap/tidb/planner/core.(*DataSource).fillIndexPath(0xc01807fc00, 0xc026587d40, 0xc027adc4c0, 0x3, 0x4, 0x37ec840, 0xc027a30fa0)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/planner/core/logical_plans.go:762 +0x631\
github.com/pingcap/tidb/planner/core.(*DataSource).DeriveStats(0xc01807fc00, 0x52c28c8, 0x0, 0x0, 0xc027a1b9b0, 0x52c28c8, 0x0, 0x0, 0x7fabda555b28, 0x0, ...)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/planner/core/stats.go:189 +0xce\
github.com/pingcap/tidb/planner/core.(*baseLogicalPlan).recursiveDeriveStats(0xc01807fc20, 0x0, 0x0, 0x0)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/planner/core/stats.go:104 +0x230\
github.com/pingcap/tidb/planner/core.(*baseLogicalPlan).recursiveDeriveStats(0xc027a85200, 0x0, 0x0, 0x0)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/planner/core/stats.go:97 +0x10e\
github.com/pingcap/tidb/planner/core.physicalOptimize(0x37bb8a0, 0xc027a851e0, 0x2082, 0x37bb8a0, 0xc027a851e0, 0x37bb8a0, 0xc027a851e0)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/planner/core/optimizer.go:173 +0x38\
github.com/pingcap/tidb/planner/core.DoOptimize(0x37789e0, 0xc000054078, 0x37bcaa0, 0xc002213e60, 0x82, 0x37bb8a0, 0xc027a851e0, 0x4, 0x37bcaa0, 0xc002213e60, ...)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/planner/core/optimizer.go:135 +0xc0\
github.com/pingcap/tidb/planner/core.(*PlanBuilder).buildDelete(0xc026587b00, 0x37789e0, 0xc000054078, 0xc0273a1b90, 0x0, 0x0, 0x0, 0x0)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/planner/core/logical_plan_builder.go:3695 +0x614\
github.com/pingcap/tidb/planner/core.(*PlanBuilder).Build(0xc026587b00, 0x37789e0, 0xc000054078, 0x3779660, 0xc0273a1b90, 0xc026587b00, 0xc0244c0600, 0xc0267e1512, 0xd)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/planner/core/planbuilder.go:564 +0x7b4\
github.com/pingcap/tidb/planner.optimize(0x37789e0, 0xc000054078, 0x37bcaa0, 0xc002213e60, 0x3779660, 0xc0273a1b90, 0x37b21c0, 0xc001bf49c0, 0x0, 0x0, ...)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/planner/optimize.go:208 +0x1e9\
github.com/pingcap/tidb/planner.Optimize(0x37789e0, 0xc000054078, 0x37bcaa0, 0xc002213e60, 0x3779660, 0xc0273a1b90, 0x37b21c0, 0xc001bf49c0, 0x0, 0x0, ...)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/planner/optimize.go:107 +0x352\
github.com/pingcap/tidb/executor.(*Compiler).Compile(0xc027b594f0, 0x37789e0, 0xc000054078, 0x3781220, 0xc0273a1b90, 0x0, 0x0, 0x0)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/executor/compiler.go:62 +0x294\
githu"]
3 个赞
2

表有多少数据和索引？

2 个赞
3

参考一下这里

2 个赞
4

我又不是版本升级引发的。
我所有节点 所有组件版本一致。

2 个赞
5

空表执行analyze 报错依旧。

1 个赞
6

辛苦贴下，执行之后tikv,tidb的日志

1 个赞
7

tidb的日志主贴里面贴了，tikv的怎么贴？ 17个tikv实例。

1 个赞
8

Hello～ 麻烦提供一下 tidb-stderr.log 和 tidb.log 的上线文本 log 日志，我们再确认一下哈。感谢

2 个赞
9

附件已私发

1 个赞
10

执行时候，能否再帮忙抓一下 go pprof 文件 ？我们看一下哈 https://studygolang.com/articles/23909

1 个赞
11

tidb_debug.zip (1.2 MB)

1 个赞
12

故障现象依旧

1 个赞
13

请问是特定的表上执行 analyze 会发生 panic 还是全部表的都有问题？如果是特定的表，麻烦提供下表结构信息，谢谢

1 个赞
14

所有表吧，我尝试过不同的库，不同大小的表，甚至是空表、小表（几十行那种）也尝试过。

1 个赞
15

analyze 报错时 tidb.log 中完整的报错信息提供下吧，另外在 v4.0 中有两个 bug 在 analyze 时会导致 tidb 发生 panic ，参考：反复出现 analyze worker panicked 报错

1 个赞
16

就输出了这两条，其他都是不相关的。

[2021/11/07 10:29:41.976 +08:00] [ERROR] [conn.go:717] ["connection running loop panic"] [conn=6063815] [lastSQL="analyze table xxxxx.xxxxxx"] [err="runtime error: invalid memory address or nil pointer dereference"] [stack="goroutine 3858801802 [running]:\
github.com/pingcap/tidb/server.(*clientConn).Run.func1(0x3778a60, 0xc028a9b8f0, 0xc045c661e0)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/server/conn.go:715 +0xee\
panic(0x2e82400, 0x4d78fb0)\
\t/usr/local/go/src/runtime/panic.go:679 +0x1b2\
github.com/pingcap/tidb/executor.(*ExecStmt).Exec.func1(0xc047ab4000, 0xc047aca7a8, 0xc047aca788)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/executor/adapter.go:288 +0x54b\
panic(0x2e82400, 0x4d78fb0)\
\t/usr/local/go/src/runtime/panic.go:679 +0x1b2\
github.com/pingcap/tidb/statistics.(*HistColl).getEqualCondSelectivity(0xc02f590980, 0xc00bed65a0, 0xc041861520, 0x1b, 0x1b, 0x3, 0x3)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/statistics/table.go:434 +0x1e3\
github.com/pingcap/tidb/statistics.(*HistColl).getIndexRowCount(0xc02f590980, 0xc0349f8dc0, 0x1, 0xc039ae1d78, 0x1, 0x1, 0x3, 0x2, 0xc02b2355c0)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/statistics/table.go:472 +0xa82\
github.com/pingcap/tidb/statistics.(*HistColl).GetRowCountByIndexRanges(0xc02f590980, 0xc0349f8dc0, 0x1, 0xc039ae1d78, 0x1, 0x1, 0x3, 0x4, 0xc0418614e0)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/statistics/table.go:279 +0x2b5\
github.com/pingcap/tidb/planner/core.(*DataSource).fillIndexPath(0xc034295a00, 0xc050bdbd40, 0xc02f590940, 0x3, 0x4, 0x37ec840, 0xc047ef5b30)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/planner/core/logical_plans.go:762 +0x631\
github.com/pingcap/tidb/planner/core.(*DataSource).DeriveStats(0xc034295a00, 0x52c28c8, 0x0, 0x0, 0xc037177710, 0x52c28c8, 0x0, 0x0, 0x7f31285c2528, 0x0, ...)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/planner/core/stats.go:189 +0xce\
github.com/pingcap/tidb/planner/core.(*baseLogicalPlan).recursiveDeriveStats(0xc034295a20, 0x0, 0x0, 0x0)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/planner/core/stats.go:104 +0x230\
github.com/pingcap/tidb/planner/core.(*baseLogicalPlan).recursiveDeriveStats(0xc01fd68260, 0x0, 0x0, 0x0)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/planner/core/stats.go:97 +0x10e\
github.com/pingcap/tidb/planner/core.(*baseLogicalPlan).recursiveDeriveStats(0xc04a914e90, 0x0, 0x0, 0x0)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/planner/core/stats.go:97 +0x10e\
github.com/pingcap/tidb/planner/core.physicalOptimize(0x37bb8a0, 0xc04a914e70, 0x2cf6, 0x37bb8a0, 0xc04a914e70, 0x37bb8a0, 0xc04a914e70)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/planner/core/optimizer.go:173 +0x38\
github.com/pingcap/tidb/planner/core.DoOptimize(0x37789e0, 0xc000054080, 0x37bcaa0, 0xc002899b00, 0xcf6, 0x37bb8a0, 0xc04a914e70, 0x0, 0x0, 0xc047ac9b20, ...)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/planner/core/optimizer.go:135 +0xc0\
github.com/pingcap/tidb/planner.optimize(0x37789e0, 0xc000054080, 0x37bcaa0, 0xc002899b00, 0x377a6a0, 0xc04a4bec30, 0x37b21c0, 0xc0f808bd10, 0x0, 0x0, ...)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/planner/optimize.go:248 +0x676\
github.com/pingcap/tidb/planner.Optimize(0x37789e0, 0xc000054080, 0x37bcaa0, 0xc002899b00, 0x377a6a0, 0xc04a4bec30, 0x37b21c0, 0xc0f808bd10, 0x0, 0x0, ...)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/planner/optimize.go:107 +0x352\
github.com/pingcap/tidb/executor.(*Compiler).Compile(0xc047ac92e0, 0x37789e0, 0xc000054080, 0x3781960, 0xc04a4bec30, 0x0, 0x0, 0x0)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/executor/compiler.go:62 +0x294\
github.com/pingcap/tidb/session.(*session).execute(0xc002899b00, 0x37789e0, 0xc000054080, 0xc050d14230, 0x63, 0x1205840, 0x1e, 0xc047ac95c8, 0x5, 0x12089d0)\
\t/home/jenkins/agent/workspace/tidb_v4.0.8/go/src/github.com/pingcap/tidb/session/session.go:1154 +0x5b9\
github.com/pingcap/tidb/session.(*session).Execute(0xc002899b00, 0x37789e0"]
[2021/11/07 10:29:43.221 +08:00] [INFO] [tidb.go:218] ["rollbackTxn for ddl/autocommit failed"]
1 个赞
17

麻烦将集群的配置文件也提供下，需要核实下是否为已知的问题 :pray:

18

meta.yaml (9.2 KB)

19

所有表analyze 都会panic 吗？看你上面描述的是好像是能稳定复习对吧。（你上面的 tidb—stderr日志，最好能发一下）另外，咱们 analyze 相关的参数最好能提供一下（show variables like ’%analyze%‘；）

20

不管大表、小表、空表都是报错。

-rw-r–r-- 1 tidb tidb 0 Oct 28 23:05 tidb_stderr.log

stderr日志为空，每个tidb node我都检查过了。