TIKV 读写延迟优化咨询

为提高效率,提问时请提供以下信息,问题描述清晰可优先响应。

  • 【TiDB 版本】:
    TiKV
    Release Version: 4.0.2
    Edition: Community
    Git Commit Hash: 98ee08c587ab47d9573628aba6da741433d8855c
    Git Commit Branch: heads/refs/tags/v4.0.2
    UTC Build Time: 2020-07-01 09:34:18
    Rust Version: rustc 1.42.0-nightly (0de96d37f 2019-12-19)
    Enable Features: jemalloc portable sse protobuf-codec
    Profile: dist_release
  • 【问题描述】:
    环境:3PD 32U 64G
    3TIKVNODE 48U 384G 4*2T NVME 25GE网口 12TIKV实例
    问题:使用go-ycsb压测发现读写延迟较大(超过业务预期,期望99%写入在5us以内),求各位支招优化方案。
    测试数据:
    #四节点 两节点64线程 两节点96线程 事务单写入
    Run finished, takes 10m25.353182752s
    INSERT - Takes(s): 625.3, Count: 9997977, OPS: 15989.3, Avg(us): 5931, Min(us): 1257, Max(us): 360724, 99th(us): 18000, 99.9th(us): 38000, 99.99th(us): 142000
    Run finished, takes 10m29.663384968s
    INSERT - Takes(s): 629.6, Count: 9999345, OPS: 15881.7, Avg(us): 5993, Min(us): 1240, Max(us): 496947, 99th(us): 18000, 99.9th(us): 38000, 99.99th(us): 206000
    Run finished, takes 12m56.595600247s
    INSERT - Takes(s): 776.5, Count: 9999854, OPS: 12877.6, Avg(us): 4914, Min(us): 1246, Max(us): 497273, 99th(us): 14000, 99.9th(us): 32000, 99.99th(us): 170000
    Run finished, takes 13m2.456423591s
    INSERT - Takes(s): 782.4, Count: 9999995, OPS: 12781.4, Avg(us): 4949, Min(us): 1261, Max(us): 505101, 99th(us): 14000, 99.9th(us): 29000, 99.99th(us): 125000

可先参考下官方文档 tikv 调优相关的内容,再看下测试效果:
1.线程调优
https://docs.pingcap.com/zh/tidb/stable/tune-tikv-thread-performance
2.内存调优
https://docs.pingcap.com/zh/tidb/stable/tune-tikv-memory-performance