tidb监控数据占用大量磁盘空间，如何清理

🪐 TiDB 技术问题
21

这段文字很熟悉呀，哈哈

22

我觉得这个是正解，要考虑是因为什么生成的数据量大，然后再做缩短保留日期或者提高监控级别日志来减缓数据生成的频率何大小

23

storage_retention是prometheus的配置参数，对应prometheus命令行启动参数是–storage.tsdb.retention。连到tidb-server必然查不到

24

理解了，谢谢。

25

大佬写的好啊，借鉴下哈哈

26

这个参数具体如何配置？我上面截图的配置方法对不对？

27

先tiup cluster edit-config xxx 修改storage_retention，然后tiup cluster restart xxx --node xxx重启prometheus节点

2 个赞
28

:smile: :smile: :smile:

29

学习了

1 个赞
30

暴力的来个Rm -rf

31

赞同！！！！！

32

还有一个方面：采集项的数量

33

过来学习一下

34

统计过tikv的指标，其中有些指标返回的metrics很多，如果不需要可以drop掉，能节省很多空间。第一列是行数，第二列是指标名。

cat tikv|grep -v "#"|awk -F'{' '{print $1}'|sort|uniq -c|sort

    105 tikv_engine_compaction_duration_seconds_bucket
    129 tikv_thread_cpu_seconds_total
    129 tikv_thread_nonvoluntary_context_switches
    129 tikv_thread_voluntary_context_switches
    144 tikv_config_rocksdb
    144 tikv_pd_request_duration_seconds_bucket
    147 tikv_raftstore_event_duration_bucket
    152 tikv_engine_bloom_efficiency
    152 tikv_engine_cache_efficiency
    152 tikv_engine_compaction_flow_bytes
    152 tikv_engine_compaction_key_drop
    152 tikv_engine_file_status
    152 tikv_engine_flow_bytes
    152 tikv_engine_get_served
    152 tikv_engine_locate
    152 tikv_engine_memtable_efficiency
    152 tikv_engine_read_amp_flow_bytes
    152 tikv_engine_write_served
    168 tikv_coprocessor_request_duration_seconds_bucket
    168 tikv_coprocessor_request_handle_seconds_bucket
    168 tikv_coprocessor_request_handler_build_seconds_bucket
    168 tikv_raftstore_apply_perf_context_time_duration_secs_bucket
    168 tikv_raftstore_store_perf_context_time_duration_secs_bucket
    258 tikv_threads_io_bytes_total
    288 tikv_coprocessor_scan_details
    336 tikv_coprocessor_scan_keys_bucket
    384 tikv_coprocessor_rocksdb_perf
    504 tikv_coprocessor_request_wait_seconds_bucket
    612 tikv_scheduler_stage_total
    714 tikv_scheduler_command_duration_seconds_bucket
    714 tikv_scheduler_latch_wait_duration_seconds_bucket
    714 tikv_scheduler_processing_read_duration_seconds_bucket
    748 tikv_scheduler_kv_command_key_read_bucket
    748 tikv_scheduler_kv_command_key_write_bucket
    748 tikv_storage_check_mem_lock_duration_seconds_bucket
    945 tikv_grpc_msg_duration_seconds_bucket
   1224 tikv_scheduler_kv_scan_details
   1632 tikv_storage_rocksdb_perf

cat tidb|grep -v "#"|awk -F'{' '{print $1}'|sort|uniq -c|sort
tidb:
    120 tidb_tikvclient_cop_duration_seconds_bucket
    140 pd_client_cmd_handle_failed_cmds_duration_seconds_bucket
    150 tidb_tikvclient_txn_cmd_duration_seconds_bucket
    161 tidb_owner_new_session_duration_seconds_bucket
    174 tidb_ddl_worker_operation_duration_seconds_bucket
    174 tidb_session_transaction_duration_seconds_bucket
    200 tidb_topsql_report_duration_seconds_bucket
    208 tidb_tikvclient_txn_regions_num_bucket
    210 tidb_tikvclient_rawkv_cmd_seconds_bucket
    224 pd_client_cmd_handle_cmds_duration_seconds_bucket
    330 tidb_tikvclient_backoff_seconds_bucket
    480 tidb_server_handle_query_duration_seconds_bucket
   1050 tidb_tikvclient_request_seconds_bucket

cat tiflash|grep -v "#"|awk -F'{' '{print $1}'|sort|uniq -c|sort
    189 tiflash_storage_subtask_duration_seconds_bucket
    217 tiflash_coprocessor_request_duration_seconds_bucket
    217 tiflash_coprocessor_request_handle_seconds_bucket

cat pd.metrics |grep -v "#"|awk -F'{' '{print $1}'|sort|uniq -c|sort
    516 grpc_server_handling_seconds_bucket
    699 grpc_server_handled_total
35

此话题已在最后回复的 60 天后被自动关闭。不再允许新回复。