“Communications link failure”错误

为提高效率,提问时请提供以下信息,问题描述清晰可优先响应。

  • 【TiDB 版本】:3.0.3
  • 【问题描述】:间隔一段时间(大约5分钟)访问tidb时就会抛以下错误

Cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet successfully received from the server was 948,688 milliseconds ago. The last packet sent successfully to the server was 948,688 milliseconds ago.
; SQL []; Communications link failure

The last packet successfully received from the server was 948,688 milliseconds ago. The last packet sent successfully to the server was 948,688 milliseconds ago.; nested exception is com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet successfully received from the server was 948,688 milliseconds ago. The last packet sent successfully to the server was 948,688 milliseconds ago.
at org.springframework.jdbc.support.SQLExceptionSubclassTranslator.doTranslate(SQLExceptionSubclassTranslator.java:98)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:74)
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:421)
at com.sun.proxy.$Proxy17.selectOne(Unknown Source)
at org.mybatis.spring.SqlSessionTemplate.selectOne(SqlSessionTemplate.java:166)
at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:63)
at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:43)
at com.sun.proxy.$Proxy18.selectSegmentRange(Unknown Source)
at com.taimeitetch.flinketl.service.base.impl.CommonServiceImpl.findSegmentRange(CommonServiceImpl.java:122)
at com.taimeitetch.flinketl.facade.engines.process.impl.OfflineJobProcessImpl.process(OfflineJobProcessImpl.java:95)
at com.taimeitetch.flinketl.facade.engines.impl.FlinkEngineImpl.executeEngin(FlinkEngineImpl.java:37)
at com.taimeitetch.flinketl.facade.impl.ExecuteEngineFacadeImpl.process(ExecuteEngineFacadeImpl.java:83)
at com.taimeitetch.flinketl.process.FlinkETLProcess.main(FlinkETLProcess.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:423)
at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:813)
at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:287)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)
at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1050)
at org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1126)
at org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1126)
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet successfully received from the server was 948,688 milliseconds ago. The last packet sent successfully to the server was 948,688 milliseconds ago.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:981)
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3652)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2460)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2625)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2551)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861)

以下是我连接池参数:
jdbc.maxConnectionsPerPartition=4
jdbc.minConnectionsPerPartition=3
jdbc.partitionCount=3
jdbc.acquireIncrement=5
jdbc.poolAvailabilityThreshold=20

jdbc.idleMaxAgeInMinutes=600
jdbc.connectionTimeoutInMs=600000
jdbc.idleConnectionTestPeriodInMinutes=100
jdbc.maxConnectionAgeInSeconds=36000

但我觉得应该跟这个关系不大,因为在用jdbc连接tidb时也会报同样的问题,该怎么解决?
另外,我在实时任务(时间间隔很小)中,也会出现这个问题

你提到直接用 JDBC 不过连接池也会报同样的错误吗?是指连一个 TIDB SERVER ? 正常服务问题会报 connection refused 类似的错误。

948,688 milliseconds ago. 你的 java 连接基本都是经过这么长时间后,再次重新连接数据库就报连接不上了。这个时间是固定的,也就是还是连接池的参数影响到了

是的,直接使用jdbc连接tidb也会出现这个问题。 我只有一个tidb节点

经过了 948 秒以后,连接就失效了

但是我一直有"select 1"这个尝试连接的啊

我感觉应该不会,不然正报错时间间隔不会固定的,那次可能是其他问题。

正常情况下,如果探测的话,不会出现这个错误的“The last packet successfully received from the server was 948,688 milliseconds ago. ”

我看下连接池配置,这块不是很懂,稍等

是的,所以我也觉得奇怪

我看了下配置文件,select 1 上面的两个参数应该是没有设置默认值吧?

[2019/12/09 20:40:14.074 +08:00] [INFO] [server.go:416] [“connection closed”] [conn=17027]

上面那句是tidb的日志,它何时会关闭连接?

这个是正常的日志,业务关闭连接后就会断开

image

这个连接池的配置似乎对tidb不起作用

这个 idleconnectionTest 设置短一点,刚刚看了下是 15 分钟后报的错误,那你设置 13 分钟试试看

select 1 探活间隔设置小一点


是指红线框中的参数吗?

这个和这个 image 是一个参数吗?

好的,我试试