为提高效率,请提供以下信息,问题描述清晰能够更快得到解决:
【 TiDB 使用环境】
v5.1.1
【概述】场景+问题概述
dashboard报错error.pd.client_request_failed: Request
【背景】做过哪些操作
【现象】业务和数据库现象
【业务影响】
【TiDB 版本】
【附件】
- 相关日志 和 监控
- TiUP Cluster Display 信息
-
TiUP Cluster Edit Config 信息
-
TiDB- Overview 监控
- 对应模块日志(包含问题前后1小时日志)
[2021/09/10 17:30:09.682 +08:00] [INFO] [kvstore_compaction.go:55] [“finished scheduled compaction”] [compact-revision=20463580] [took=258.158432ms]
[2021/09/10 17:34:44.291 +08:00] [INFO] [raft.go:1530] [“f406256caca6d847 switched to configuration voters=(13091707281300229008 16558456657676540723 17583782943797991495)”]
[2021/09/10 17:34:44.292 +08:00] [INFO] [cluster.go:422] [“removed member”] [cluster-id=5af7ae9b61fa2fcd] [local-member-id=f406256caca6d847] [removed-remote-peer-id=631d9ddf34107f42] [removed-remote-peer-urls=“[http://172.31.72.2:2380]”][2021/09/10 17:34:44.292 +08:00] [INFO] [peer.go:333] [“stopping remote peer”] [remote-peer-id=631d9ddf34107f42]
[2021/09/10 17:34:44.294 +08:00] [WARN] [stream.go:291] [“closed TCP streaming connection with remote peer”] [stream-writer-type=“stream MsgApp v2”] [remote-peer-id=631d9ddf34107f42][2021/09/10 17:34:44.294 +08:00] [WARN] [stream.go:301] [“stopped TCP streaming connection with remote peer”] [stream-writer-type=“stream MsgApp v2”] [remote-peer-id=631d9ddf34107f42]
[2021/09/10 17:34:44.294 +08:00] [WARN] [http.go:421] [“rejected stream from remote peer because it was removed”] [local-member-id=f406256caca6d847] [remote-peer-id-stream-handler=f406256caca6d847] [remote-peer-id-from=631d9ddf34107f42]
[2021/09/10 17:34:44.295 +08:00] [WARN] [stream.go:291] [“closed TCP streaming connection with remote peer”] [stream-writer-type=“stream Message”] [remote-peer-id=631d9ddf34107f42][2021/09/10 17:34:44.295 +08:00] [WARN] [stream.go:301] [“stopped TCP streaming connection with remote peer”] [stream-writer-type=“stream Message”] [remote-peer-id=631d9ddf34107f42]
[2021/09/10 17:34:44.295 +08:00] [INFO] [pipeline.go:86] [“stopped HTTP pipelining with remote peer”] [local-member-id=f406256caca6d847] [remote-peer-id=631d9ddf34107f42][2021/09/10 17:34:44.295 +08:00] [WARN] [http.go:421] [“rejected stream from remote peer because it was removed”] [local-member-id=f406256caca6d847] [remote-peer-id-stream-handler=f406256caca6d847] [remote-peer-id-from=631d9ddf34107f42]
[2021/09/10 17:34:44.296 +08:00] [WARN] [stream.go:436] [“lost TCP streaming connection with remote peer”] [stream-reader-type=“stream MsgApp v2”] [local-member-id=f406256caca6d847] [remote-peer-id=631d9ddf34107f42] [error=“context canceled”]
[2021/09/10 17:34:44.296 +08:00] [WARN] [peer_status.go:68] [“peer became inactive (message send to peer failed)”] [peer-id=631d9ddf34107f42] [error=“failed to read 631d9ddf34107f42 on stream MsgApp v2 (context canceled)”][2021/09/10 17:34:44.296 +08:00] [INFO] [stream.go:459] [“stopped stream reader with remote peer”] [stream-reader-type=“stream MsgApp v2”] [local-member-id=f406256caca6d847] [remote-peer-id=631d9ddf34107f42]
[2021/09/10 17:34:44.296 +08:00] [WARN] [stream.go:436] [“lost TCP streaming connection with remote peer”] [stream-reader-type=“stream Message”] [local-member-id=f406256caca6d847] [remote-peer-id=631d9ddf34107f42] [error=“context canceled”]
[2021/09/10 17:34:44.296 +08:00] [INFO] [stream.go:459] [“stopped stream reader with remote peer”] [stream-reader-type=“stream Message”] [local-member-id=f406256caca6d847] [remote-peer-id=631d9ddf34107f42]
[2021/09/10 17:34:44.296 +08:00] [INFO] [peer.go:340] [“stopped remote peer”] [remote-peer-id=631d9ddf34107f42][2021/09/10 17:34:44.296 +08:00] [INFO] [transport.go:369] [“removed remote peer”] [local-member-id=f406256caca6d847] [removed-remote-peer-id=631d9ddf34107f42]
[2021/09/10 18:26:47.417 +08:00] [INFO] [server.go:1380] [“triggering snapshot”] [local-member-id=f406256caca6d847] [local-member-applied-index=27500279] [local-member-snapshot-index=27400278] [local-member-snapshot-count=100000]
[2021/09/10 18:26:47.418 +08:00] [INFO] [server.go:2371] [“saved snapshot”] [snapshot-index=27500279]
[2021/09/10 18:26:47.419 +08:00] [INFO] [server.go:2413] [“compacted Raft logs”] [compact-index=27495279]
[2021/09/10 18:27:06.278 +08:00] [INFO] [purge.go:69] [purged] [path=/data/vols/tidb/tidb-data/pd-2379/member/snap/0000000000000002-00000000019bfdd2.snap]
[2021/09/10 18:30:09.428 +08:00] [INFO] [index.go:189] [“compact tree index”] [revision=20479110]
[2021/09/10 18:30:09.652 +08:00] [INFO] [kvstore_compaction.go:55] [“finished scheduled compaction”] [compact-revision=20479110] [took=223.775426ms]
[2021/09/10 19:30:09.431 +08:00] [INFO] [index.go:189] [“compact tree index”] [revision=20503608]
[2021/09/10 19:30:09.846 +08:00] [INFO] [kvstore_compaction.go:55] [“finished scheduled compaction”] [compact-revision=20503608] [took=353.822844ms]
[2021/09/10 20:30:09.435 +08:00] [INFO] [index.go:189] [“compact tree index”] [revision=20529724]
[2021/09/10 20:30:09.796 +08:00] [INFO] [kvstore_compaction.go:55] [“finished scheduled compaction”] [compact-revision=20529724] [took=360.437296ms]
[2021/09/10 21:26:57.232 +08:00] [INFO] [server.go:1380] [“triggering snapshot”] [local-member-id=f406256caca6d847] [local-member-applied-index=27600280] [local-member-snapshot-index=27500279] [local-member-snapshot-count=100000]
[2021/09/10 21:26:57.233 +08:00] [INFO] [server.go:2371] [“saved snapshot”] [snapshot-index=27600280]
[2021/09/10 21:26:57.234 +08:00] [INFO] [server.go:2413] [“compacted Raft logs”] [compact-index=27595280]
[2021/09/10 21:27:06.509 +08:00] [INFO] [purge.go:69] [purged] [path=/data/vols/tidb/tidb-data/pd-2379/member/snap/0000000000000002-00000000019d8473.snap]
[2021/09/10 21:30:09.437 +08:00] [INFO] [index.go:189] [“compact tree index”] [revision=20555786]
[2021/09/10 21:30:09.805 +08:00] [INFO] [kvstore_compaction.go:55] [“finished scheduled compaction”] [compact-revision=20555786] [took=366.873024ms]
[2021/09/10 22:30:09.438 +08:00] [INFO] [index.go:189] [“compact tree index”] [revision=20581796]
[2021/09/10 22:30:09.800 +08:00] [INFO] [kvstore_compaction.go:55] [“finished scheduled compaction”] [compact-revision=20581796] [took=361.41854ms]
[2021/09/10 23:22:05.745 +08:00] [WARN] [grpclog.go:60] [“transport: http2Server.HandleStreams failed to read frame: read tcp 10.96.217.123:2379->10.96.217.123:24688: read: connection reset by peer”]