为提高效率,提问时请提供以下信息,问题描述清晰可优先响应。
- 【TiDB 版本】:v4.0.7
- 【问题描述】:
uncategorized SQLException for SQL []; SQL state [HY000]; error code [8027]; Information schema is out of date: schema failed to update in 1 lease, please make sure TiDB can connect to TiKV;
拓扑:
3个tidb,3个pd,3个tikv,2个tiflash
tiflash是2副本
操作:
业务正常增删改查,有olap的sql请求到tiflash
同时load data
现象:
tikv2 cpu被打满
tidb无法接受新连接
其中一条错误日志:
[2020/12/02 11:11:41.496 +00:00] [INFO] [conn.go:780] [“command dispatched failed”] [conn=967] [connInfo=“id:967, addr:10.211.41.105:52666 status:10, collation:utf8_general_ci, user:jcloud_billing_rw”] [command=Query] [status=“inTxn:0, autocommit:1”] [sql=“insert into tidb_accounting_sub_bill\
( bill_id,\
\
\
bill_time,\
\
\
data_source,\
\
\
source_id,\
\
\
pin,\
\
\
start_time,\
\
\
end_time,\
\
\
site,\
\
\
region,\
\
\
app_code,\
\
\
service_code,\
\
\
resource_id,\
\
\
billing_type,\
\
\
formula,\
\
\
bill_fee,\
\
\
bill_fee2,\
\
\
discount_fee,\
\
\
coupon_id,\
\
\
coupon_fee,\
\
\
actual_fee,\
\
\
cash_coupon_fee,\
\
\
\
\
balance_pay_fee,\
\
\
cash_pay_fee,\
\
\
\
transaction_no,\
\
\
\
org,\
\
\
pay_state,\
\
\
pay_time,\
\
\
trade_type,\
\
\
formula_desc,\
\
\
is_deleted,\
\
\
create_time,\
\
\
update_time ) \
values ( 1226649735,\
\
\
‘2020-12-02 19:11:36’,\
\
\
1,\
\
\
‘1226649719’,\
\
\
‘iaas_jke_admin’,\
\
\
‘2020-12-02 18:00:00’,\
\
\
‘2020-12-02 18:59:59’,\
\
\
0,\
\
\
‘cn-north-1’,\
\
\
‘jcloud’,\
\
\
‘ip’,\
\
\
‘fip-e5b1fp3vhr’,\
\
\
1,\
\
\
'[{“key”:“bw”,“unit”:”",“value”:10.0}]’,\
\
\
0.9200,\
\
\
0.92,\
\
\
0.92,\
\
\
‘’,\
\
\
0.00,\
\
\
0.00,\
\
\
0,\
\
\
\
\
0,\
\
\
0,\
\
\
\
‘12096942324452304’,\
\
\
\
‘950’,\
\
\
1,\
\
\
‘2020-12-02 19:11:36’,\
\
\
1,\
\
\
‘带宽:10M’,\
\
\
0,\
\
\
‘2020-12-02 19:11:36’,\
\
\
‘2020-12-02 19:11:36’ )"] [txn_mode=PESSIMISTIC] [err="[domain:8027]Information schema is out of date: schema failed to update in 1 lease, please make sure TiDB can connect to TiKV\ngithub.com/pingcap/errors.AddStack\
\t/home/jenkins/agent/workspace/tidb_v4.0.7/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20200917111840-a15ef68f753d/errors.go:174\
github.com/pingcap/errors.Trace\
\t/home/jenkins/agent/workspace/tidb_v4.0.7/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20200917111840-a15ef68f753d/juju_adaptor.go:15\
github.com/pingcap/tidb/store/tikv.(*twoPhaseCommitter).checkSchemaValid\
\t/home/jenkins/agent/workspace/tidb_v4.0.7/go/src/github.com/pingcap/tidb/store/tikv/2pc.go:1531\ngithub.com/pingcap/tidb/store/tikv.(*twoPhaseCommitter).execute\
\t/home/jenkins/agent/workspace/tidb_v4.0.7/go/src/github.com/pingcap/tidb/store/tikv/2pc.go:1349\ngithub.com/pingcap/tidb/store/tikv.(*tikvTxn).Commit\
\t/home/jenkins/agent/workspace/tidb_v4.0.7/go/src/github.com/pingcap/tidb/store/tikv/txn.go:307\ngithub.com/pingcap/tidb/session.(*TxnState).Commit\
\t/home/jenkins/agent/workspace/tidb_v4.0.7/go/src/github.com/pingcap/tidb/session/txn.go:279\ngithub.com/pingcap/tidb/session.(*session).doCommit\
\t/home/jenkins/agent/workspace/tidb_v4.0.7/go/src/github.com/pingcap/tidb/session/session.go:432\ngithub.com/pingcap/tidb/session.(*session).doCommitWithRetry\
\t/home/jenkins/agent/workspace/tidb_v4.0.7/go/src/github.com/pingcap/tidb/session/session.go:452\ngithub.com/pingcap/tidb/session.(*session).CommitTxn\
\t/home/jenkins/agent/workspace/tidb_v4.0.7/go/src/github.com/pingcap/tidb/session/session.go:510\ngithub.com/pingcap/tidb/session.autoCommitAfterStmt\
\t/home/jenkins/agent/workspace/tidb_v4.0.7/go/src/github.com/pingcap/tidb/session/tidb.go:230\
github.com/pingcap/tidb/session.finishStmt\
\t/home/jenkins/agent/workspace/tidb_v4.0.7/go/src/github.com/pingcap/tidb/session/tidb.go:196\
github.com/pingcap/tidb/session.runStmt\
\t/home/jenkins/agent/workspace/tidb_v4.0.7/go/src/github.com/pingcap/tidb/session/tidb.go:316\
github.com/pingcap/tidb/session.(*session).executeStatement\
\t/home/jenkins/agent/workspace/tidb_v4.0.7/go/src/github.com/pingcap/tidb/session/session.go:1061\ngithub.com/pingcap/tidb/session.(*session).execute\
\t/home/jenkins/agent/workspace/tidb_v4.0.7/go/src/github.com/pingcap/tidb/session/session.go:1173\ngithub.com/pingcap/tidb/session.(*session).Execute\
\t/home/jenkins/agent/workspace/tidb_v4.0.7/go/src/github.com/pingcap/tidb/session/session.go:1104\ngithub.com/pingcap/tidb/server.(*TiDBContext).Execute\
\t/home/jenkins/agent/workspace/tidb_v4.0.7/go/src/github.com/pingcap/tidb/server/driver_tidb.go:248\ngithub.com/pingcap/tidb/server.(*clientConn).handleQuery\
\t/home/jenkins/agent/workspace/tidb_v4.0.7/go/src/github.com/pingcap/tidb/server/conn.go:1335\ngithub.com/pingcap/tidb/server.(*clientConn).dispatch\
\t/home/jenkins/agent/workspace/tidb_v4.0.7/go/src/github.com/pingcap/tidb/server/conn.go:966\ngithub.com/pingcap/tidb/server.(*clientConn).Run\
\t/home/jenkins/agent/workspace/tidb_v4.0.7/go/src/github.com/pingcap/tidb/server/conn.go:765\ngithub.com/pingcap/tidb/server.(*Server).onConn\
\t/home/jenkins/agent/workspace/tidb_v4.0.7/go/src/github.com/pingcap/tidb/server/server.go:421\
runtime.goexit\
\t/usr/local/go/src/runtime/asm_amd64.s:1357"]
其他日志可以继续补充。谢谢。