tidb server中存在大量`send request failed, err: context canceled` 的INFO日志

🪐 TiDB 技术问题
1

【 TiDB 使用环境】生产环境
【 TiDB 版本】7.5.3

从TIDB server日志中，存在大量如下日志

[2024/10/16 17:16:45.760 +08:00] [INFO] [region_request.go:1531] ["send request failed, err: context canceled"] [conn=1774233796] [session_alias=] [req-ts=453265380011409445] [req-type=Cop] [region="{ region id: 174377, ver: 9458, confVer: 1811 }"] [replica-read-type=leader] [stale-read=false] [request-sender="{rpcError:context canceled,replicaSelector: replicaSelector{selectorStateStr: accessKnownLeader, cacheRegionIsValid: true, replicaStatus: [peer: 174378, store: 98030, isEpochStale: false, attempts: 0, replica-epoch: 2, store-epoch: 2, store-state: resolved, store-liveness-state: reachable peer: 174379, store: 1, isEpochStale: false, attempts: 1, replica-epoch: 1, store-epoch: 1, store-state: resolved, store-liveness-state: reachable peer: 174380, store: 98031, isEpochStale: false, attempts: 0, replica-epoch: 0, store-epoch: 0, store-state: resolved, store-liveness-state: reachable]}}"] [retry-times=0] [total-backoff-ms=0] [total-backoff-times=0] [max-exec-timeout-ms=60000] [total-region-errors=]
[2024/10/16 17:16:45.760 +08:00] [INFO] [region_request.go:1531] ["send request failed, err: context canceled"] [conn=1774233796] [session_alias=] [req-ts=453265380011409445] [req-type=Cop] [region="{ region id: 161767, ver: 9458, confVer: 1835 }"] [replica-read-type=leader] [stale-read=false] [request-sender="{rpcError:context canceled,replicaSelector: replicaSelector{selectorStateStr: accessKnownLeader, cacheRegionIsValid: true, replicaStatus: [peer: 161769, store: 98030, isEpochStale: false, attempts: 0, replica-epoch: 2, store-epoch: 2, store-state: resolved, store-liveness-state: reachable peer: 18183206, store: 8, isEpochStale: false, attempts: 0, replica-epoch: 0, store-epoch: 0, store-state: resolved, store-liveness-state: reachable peer: 18399917, store: 98032, isEpochStale: false, attempts: 1, replica-epoch: 0, store-epoch: 0, store-state: resolved, store-liveness-state: reachable]}}"] [retry-times=0] [total-backoff-ms=0] [total-backoff-times=0] [max-exec-timeout-ms=60000] [total-region-errors=]
[2024/10/16 17:16:45.760 +08:00] [INFO] [region_request.go:1531] ["send request failed, err: context canceled"] [conn=1774233796] [session_alias=] [req-ts=453265380011409445] [req-type=Cop] [region="{ region id: 173975, ver: 9457, confVer: 1823 }"] [replica-read-type=leader] [stale-read=false] [request-sender="{rpcError:context canceled,replicaSelector: replicaSelector{selectorStateStr: accessKnownLeader, cacheRegionIsValid: true, replicaStatus: [peer: 173978, store: 1, isEpochStale: false, attempts: 1, replica-epoch: 1, store-epoch: 1, store-state: resolved, store-liveness-state: reachable peer: 349498, store: 98033, isEpochStale: false, attempts: 0, replica-epoch: 0, store-epoch: 0, store-state: resolved, store-liveness-state: reachable peer: 17675950, store: 98030, isEpochStale: false, attempts: 0, replica-epoch: 2, store-epoch: 2, store-state: resolved, store-liveness-state: reachable]}}"] [retry-times=0] [total-backoff-ms=0] [total-backoff-times=0] [max-exec-timeout-ms=60000] [total-region-errors=]

之前没有关注这个，不知道这个日志对tidb 性能有什么影响

2

没啥影响

3

[INFO] 级别日志没有看的必要

4

嗯，就是该类型日志量级有点，可能和同步数据有关系吧。先忽略了

5

其实info级别的日志并没有必要开，tidb的warn级别也没多大影响，error都未必有问题

6

可能的原因和解决方案：

  1. 上下文取消：
  • 如果请求执行时间过长，可能会导致上下文被取消。检查请求是否超时，或者代码中是否有手动取消的情况。
  1. 网络问题：
  • 检查是否存在网络中断，或者相关节点是否可访问。
  1. 领导选举：
  • 如果区域的领导者发生变化（由于故障或其他原因），可能会导致上下文取消。监控是否有领导选举事件。
  1. 数据库性能：
  • 分析数据库的性能，确保它没有过载。缓慢的查询或重负载可能导致响应时间延迟。
  1. 日志和监控：
  • 持续监控日志以查找任何模式或重复出现的错误。此外，考虑在涉及网络请求的区域实现更健壮的错误处理或日志记录。
  1. 配置检查：
  • 检查数据库系统的超时和请求处理配置设置，看看是否需要进行调整。
1 个赞
7

也可以考虑检查下tidb的服务器的负载是不是过高，是否有慢日志等消耗资源高的SQL等导致的

8

应该没啥影响