生产环境tidb 版本v7.5.4
Your MySQL connection id is 3683000214 Server version: 8.0.11-TiDB-v7.5.4 TiDB Server
生产配置
4c 16g tidb+pd，12c 96g tikv存储，业务低峰期，整体QPS不高
tidb计算节点内存上涨和服务器监控内存上涨一致
tidb计算发生OOM
内存上涨近3倍持续几分钟时间后，两个节点节点先后发生OOM
/tmp/1101_tidb目录下也没有记录相关慢sql，推断也可能是/tmp目录文件刚好超过10天被系统自动清理了
慢sql日志分析
统计该tidb这2分钟内慢sql日志
sed -n ‘/^# Time: 2024-11-26T08:00./,/^# Time: 2024-11-26T08:02./p’ tidb_slow_query-2024-11-26T08-04-37.802.log >/rebate_tidb_slow_log
2分钟内慢sql日志总数量110个
grep “Mem_max” /rebate_tidb_slow_log | cut -d":" -f2 | sort -nr | wc -l
110
计算出慢sql日志总内存消耗为1063 MB ，远小于tidb计算节点内存上涨的6G内存
grep “Mem_max” /rebate_tidb_slow_log | cut -d":" -f2 | awk ‘{ sum += $1 } END { print sum/1024/1024 }’
1069.3
grep “Mem_max” /rebate_tidb_slow_log | cut -d":" -f2 | sort -nr
92573710
92573710
92572982
92572982
92572982
84158051
84158051
84158051
84157323
84157323
84157323
84157323
1757760
1691988
1681956
1681030
1667926
1663216
…
2分钟内2个计算节点 select慢查询如下，无明显很耗时、内存占用高情况
10分钟内2个计算节点所有慢查询如下，无明显很耗时、内存占用高情况，且慢查询类型都为select，无内部sql任务
相关select sql
SELECT
cs.*
FROM
xxxx cs
WHERE
cs.mart_code = ‘wm’
AND cs.goods_code = ?
AND cs.shop_code IN (
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
?,
? (len :97669);
楼上回答的非常全面，楼主可以参考一下。