升级到7.5之后,新增了两个参数,不过当前还必须手动改配置来,没办法通过TiUP配置。
修改方法是:
- 复制ngmonitoring的配置到新文件中。(原始的配置文件每次reload prometheus时候都会被覆盖)
cd /data/tidb-deploy/prometheus-9090
cp conf/ngmonitoring.toml conf/ngmonitoring-new.toml
- 在
conf/ngmonitoring-new.toml
中增加tsdb的两个配置项。我这里里缩短了一下tsdb的保留时间,以及调大了search-max-unique-timeseries的值
[tsdb]
# Data with timestamps outside the retentionPeriod is automatically deleted
# The following optional suffixes are supported: h (hour), d (day), w (week), y (year).
# If suffix isn't set, then the duration is counted in months.
retention-period = "7d"
# `search-max-unique-timeseries` limits the number of unique time series a single query can find and process.
# VictoriaMetrics(tsdb) keeps in memory some metainformation about the time series located by each query
# and spends some CPU time for processing the found time series. This means that the maximum memory usage
# and CPU usage a single query can use is proportional to `search-max-unique-timeseries`.
search-max-unique-timeseries = 9000000
- 修改
scripts/ng-wrapper.sh
,让其使用新的配置文件。(scripts里的文件,只有在tiup cluster upgrade
时候才会被覆盖,常规的的scale-out/scale-in/reload
操作并不会被覆盖)
#!/bin/bash
# WARNING: This file was auto-generated to restart ng-monitoring when fail.
# Do not edit! All your edit might be overwritten!
while true
do
bin/ng-monitoring-server --config /home/tidb/tidb-deploy/prometheus-9090/conf/ngmonitoring-new.toml >/dev/null 2>&1
sleep 15s
done