memory high

为提高效率,提问时请提供以下信息,问题描述清晰可优先响应。

  • 【TiDB 版本】:v4.0.7
  • 【问题描述】:memory high, and no high load, most mem spent on bytes.makeSlice, why?how to release these part mem?

Type: inuse_space
Time: Oct 27, 2020 at 10:54pm (CST)
Entering interactive mode (type “help” for commands, “o” for options)
(pprof) top
Showing nodes accounting for 53461.73MB, 98.40% of 54329.14MB total
Dropped 390 nodes (cum <= 271.65MB)
Showing top 10 nodes out of 17
flat flat% sum% cum cum%
53172.84MB 97.87% 97.87% 53172.84MB 97.87% bytes.makeSlice
287.89MB 0.53% 98.40% 287.89MB 0.53% google.golang.org/grpc.(*parser).recvMsg
1MB 0.0018% 98.40% 291.89MB 0.54% github.com/pingcap/kvproto/pkg/tikvpb.(*tikvCoprocessorStreamClient).Recv
0 0% 98.40% 53172.84MB 97.87% bytes.(*Buffer).Write
0 0% 98.40% 53172.84MB 97.87% bytes.(*Buffer).grow
0 0% 98.40% 306.89MB 0.56% github.com/pingcap/tidb/store/tikv.(*copIteratorWorker).handleTask
0 0% 98.40% 306.39MB 0.56% github.com/pingcap/tidb/store/tikv.(*copIteratorWorker).handleTaskOnce
0 0% 98.40% 306.89MB 0.56% github.com/pingcap/tidb/store/tikv.(*copIteratorWorker).run
0 0% 98.40% 291.89MB 0.54% github.com/pingcap/tidb/store/tikv/tikvrpc.(*CopStreamResponse).Recv
0 0% 98.40% 299.39MB 0.55% google.golang.org/grpc.(*clientStream).RecvMsg

  1. 请问, enable-streaming 有设置为 true 吗? 如果有可以改为 false,重启 tidb-server 试试
  2. 如果没有,尝试调小 tidb_distsql_scan_concurrency ,

设置为true了

可以按照上面的方法试试

我看对应的执行计划,是indexscan和tablescan,调整tidb_distsql_scan_concurrency和 tidb_index_lookup_concurrency吗?


当一个大的select(ap)过去,process内存远大于堆内存,比如图中239的60G,堆18G,其他32G内存基本都在makeSlice中,应该是读写buffer中,这部分内存怎么快速释放啊

另外in-heap内存是否可以释放以及是否可以快速释放?
——————
看了下tidb的session变量,以及tidb-server的参数,没找到这类cache控制参数
——————
谢谢🙏

tidb 应该没有参数,可以试试配置go的参数 GOGC=30 试试; 上面调低参数不行吗?