tiup升级连接丢失

  • 【TiDB 版本】:v4.0.0-rc
  • 【问题描述】: 连接丢失 在tiup从4.0.0-beta1升级到4.0.0-rc的过程中,发现了连接丢失的情况。导致了一些任务失败。完整日志如下

2020/04/09 16:31:01 - TiDB - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : org.pentaho.di.core.exception.KettleDatabaseException: 2020/04/09 16:31:01 - TiDB - Error comitting connection 2020/04/09 16:31:01 - TiDB - Communications link failure during commit(). Transaction resolution unknown. 2020/04/09 16:31:01 - TiDB - 2020/04/09 16:31:01 - TiDB - at org.pentaho.di.core.database.Database.commit(Database.java:819) 2020/04/09 16:31:01 - TiDB - at org.pentaho.di.core.database.Database.commit(Database.java:791) 2020/04/09 16:31:01 - TiDB - at org.pentaho.di.core.database.Database.disconnect(Database.java:667) 2020/04/09 16:31:01 - TiDB - at org.pentaho.di.trans.steps.insertupdate.InsertUpdate.dispose(InsertUpdate.java:516) 2020/04/09 16:31:01 - TiDB - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:97) 2020/04/09 16:31:01 - TiDB - at java.lang.Thread.run(Thread.java:745) 2020/04/09 16:31:01 - TiDB - Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Communications link failure during commit(). Transaction resolution unknown. 2020/04/09 16:31:01 - TiDB - at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 2020/04/09 16:31:01 - TiDB - at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 2020/04/09 16:31:01 - TiDB - at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 2020/04/09 16:31:01 - TiDB - at java.lang.reflect.Constructor.newInstance(Constructor.java:408) 2020/04/09 16:31:01 - TiDB - at com.mysql.jdbc.Util.handleNewInstance(Util.java:425) 2020/04/09 16:31:01 - TiDB - at com.mysql.jdbc.Util.getInstance(Util.java:408) 2020/04/09 16:31:01 - TiDB - at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:919) 2020/04/09 16:31:01 - TiDB - at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:898) 2020/04/09 16:31:01 - TiDB - at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:887) 2020/04/09 16:31:01 - TiDB - at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:861) 2020/04/09 16:31:01 - TiDB - at com.mysql.jdbc.ConnectionImpl.commit(ConnectionImpl.java:1559) 2020/04/09 16:31:01 - TiDB - at org.pentaho.di.core.database.Database.commit(Database.java:810) 2020/04/09 16:31:01 - TiDB - … 5 more 2020/04/09 16:31:01 - 插入 / 更新.6 - Finished processing (I=44249, O=44249, R=44249, W=44248, U=0, E=0) 2020/04/09 16:31:01 - 插入 / 更新.2 - Finished processing (I=42891, O=42891, R=42891, W=42890, U=0, E=0) 2020/04/09 16:31:01 - 插入 / 更新.8 - Finished processing (I=42713, O=42713, R=42713, W=42712, U=0, E=0) 2020/04/09 16:31:01 - 插入 / 更新.7 - Finished processing (I=42884, O=42884, R=42884, W=42883, U=0, E=0) 2020/04/09 16:31:01 - 插入 / 更新.9 - Finished processing (I=42607, O=42607, R=42607, W=42606, U=0, E=0) 2020/04/09 16:31:01 - 插入 / 更新.1 - Finished processing (I=42812, O=42812, R=42812, W=42811, U=0, E=0) 2020/04/09 16:31:01 - 插入 / 更新.5 - Finished processing (I=42810, O=42810, R=42810, W=42809, U=0, E=0) 2020/04/09 16:31:01 - ods_hzsmz_rn_project_worker_entry_exit_history_df - Transformation detected one or more steps with errors. 2020/04/09 16:31:01 - ods_hzsmz_rn_project_worker_entry_exit_history_df - Transformation is killing the other steps! 2020/04/09 16:31:05 - 表输入.0 - Finished processing (I=526081, O=0, R=0, W=526080, U=0, E=0) 2020/04/09 16:31:05 - ods_hzsmz_rn_project_worker_entry_exit_history_df - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : Errors detected!

升级的是会对 TiDB 进行重启的。所以是有可能对该 TIDB 正在执行 SQL 造成影响。因此,一般建议在 TiDB 前面添加 LB 以及避开业务高峰期进行升级。

是连接LB的,但是LB最终还是把请求给某个TIDB。升级那个tidb节点的时候,SQL就挂了。希望后续能加个自动failover:smiley:

目前无缝升级需要 LB 和应用配合实现,在滚动升级前将要操作的 TiDB 节点从 LB 上摘除,等会话执行完成后再对该 TiDB 做升级操作,升级完成后将 TiDB 节点添加回 LB,并确认有流量进来。

1赞