tidb server内存使用过高,如何限制

【 TiDB 使用环境】生产环境
【 TiDB 版本】
【复现路径】执行较大的sql
【遇到的问题:问题现象及影响】 机器内存使用90%
【资源配置】机器内存128G
【附件:截图/日志/监控】
SQL语句:
“SELECT 自定义 SQL 查询.activation_user_num AS activation_user_num,\n 自定义 SQL 查询.amount AS amount,\n 自定义 SQL 查询.campaign_id AS campaign_id,\n 自定义 SQL 查询.groupid AS groupid,\n 自定义 SQL 查询.guiyin_source AS guiyin_source,\n 自定义 SQL 查询.hour_time AS hour_time,\n 自定义 SQL 查询.is_dxz AS is_dxz,\n 自定义 SQL 查询.keyword_id AS keyword_id,\n 自定义 SQL 查询.ocpc_user_num AS ocpc_user_num,\n 自定义 SQL 查询.order_num AS order_num,\n 自定义 SQL 查询.rong_adid AS rong_adid,\n 自定义 SQL 查询.source AS source,\n 自定义 SQL 查询.subTaskId AS subTaskId,\n 自定义 SQL 查询.tjy_amount AS tjy_amount,\n 自定义 SQL 查询.tjy_order_num AS tjy_order_num\nFROM (\n select \r\n hour_time as hour_time\r\n ,lower(source) as source\r\n ,lower(guiyin_source) as guiyin_source\r\n ,trim(campaign_id) as campaign_id\r\n ,trim(rong_adid) as rong_adid\r\n ,trim(keyword_id )as keyword_id\r\n ,trim(groupid) as groupid\r\n ,trim(subTaskId) as subTaskId\r\n ,is_dxz as is_dxz\r\n ,sum(activation_user_num) as activation_user_num\r\n ,sum(order_num) as order_num\r\n ,sum(amount) as amount\r\n ,sum(tjy_order_num) as tjy_order_num\r\n ,sum(tjy_amount) as tjy_amount\r\n ,sum(ocpc_user_num) as ocpc_user_num\r\n from \r\n (select \r\n t1.hour_time as hour_time\r\n ,t1.source as source\r\n ,t1.source as guiyin_source\r\n ,t1.campaign_id as campaign_id\r\n ,case when t1.rong_adid is null then 0 else t1.rong_adid end as rong_adid\r\n ,t1.keyword_id as keyword_id\r\n ,case when t1.groupid is null then 0 else t1.groupid end as groupid\r\n ,t1.is_dxz as is_dxz\r\n ,case when t1.subTaskId is null then 0 else t1.subTaskId end as subTaskId\r\n ,count(distinct t1.rid) as activation_user_num\r\n ,0 as order_num\r\n ,0 as amount\r\n ,0 as tjy_order_num\r\n ,0 as tjy_amount\r\n ,0 as ocpc_user_num\r\n from \r\n app_active_campaign_hour t1\r\n where date(t1.hour_time) >= ‘2023-11-01’\r\n group by t1.hour_time\r\n ,t1.source \r\n ,t1.source \r\n ,t1.campaign_id \r\n ,case when t1.rong_adid is null then 0 else t1.rong_adid end \r\n ,t1.keyw”(len:7768);

执行计划:
slow.sql (108.1 KB)

低版本的tidb-server内存限制是不太好,升级之后总内存限制和单个内存限制的参数都比较有用

您可以通过设置系统变量 tidb_server_memory_limit 来限制 TiDB Server 实例的内存使用阈值。

的确,可以通过 tidb_server_memory_limit 进行限制。

5.4版本限制内存有问题,经常失效,建议升级到高版本

tidb_server_memory_limit

使用系统变量tidb_mem_quota_query来配置单条SQL执行过程中的内存使用阈值,单位为字节。

tidb_mem_quota_query这个参数,也可以限制内容使用的配额。

tidb_server_memory_limit参数可以限制总内存

升级到新版本 内存回收更快 ,算子下推更多

通过参数限制,或者升级tidb的版本。

高版本内存会有很高的优化(升级下来感觉很好),其次 TiDB 内存控制文档 | PingCAP 文档中心 参考下内存参数可以控得住,最后租户可以针对个别用户进行限制

tidb_server_memory_limit

升级新版本也可以

可以试一下用tidb_server_memory_limit进行限制。

通过 tidb_server_memory_limit 来设置 tidb-server 实例的内存使用阈值。还有一种可能是某个SQL使用了大量的线程,也可以考虑限制下单条SQL的使用内存:tidb_mem_quota_query

建议升级tidb 版本到 7.5.x ,低版本内存管理上有一些缺陷。

tidb_server_memory_limit

tidb_server_memory_limit

用新版本 老的gidb 单独部署