读取大量数据UNKNOWN: tso count should be positive

为提高效率,提问时请提供以下信息,问题描述清晰可优先响应。

  • 【TiDB 版本】:3.0.5
  • 【问题描述】:使用scala编写代码,读取TiDB中一个千万级别的表报错。

20/01/16 15:57:41 WARN ConcreteBackOffer: backoffer.maxSleep 5000ms is exceeded, errors: 1.shade.io.grpc.StatusRuntimeException: UNKNOWN: tso count should be positive 2.shade.io.grpc.StatusRuntimeException: UNKNOWN: tso count should be positive 3.shade.io.grpc.StatusRuntimeException: UNKNOWN: tso count should be positive Exception in thread “main” java.lang.Exception: com.pingcap.tikv.exception.GrpcException: retry is exhausted. at io.github.interestinglab.waterdrop.Waterdrop$.main(Waterdrop.scala:42) at io.github.interestinglab.waterdrop.Waterdrop.main(Waterdrop.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:906) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:197) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:227) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:136) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: com.pingcap.tikv.exception.GrpcException: retry is exhausted. at com.pingcap.tikv.util.ConcreteBackOffer.doBackOff(ConcreteBackOffer.java:124) at com.pingcap.tikv.operation.PDErrorHandler.handleRequestError(PDErrorHandler.java:54) at com.pingcap.tikv.policy.RetryPolicy.callWithRetry(RetryPolicy.java:59) at com.pingcap.tikv.AbstractGRPCClient.callWithRetry(AbstractGRPCClient.java:65) at com.pingcap.tikv.PDClient.getTimestamp(PDClient.java:75) at com.pingcap.tikv.TiSession.getTimestamp(TiSession.java:56) at com.pingcap.tikv.TiSession.createSnapshot(TiSession.java:60) at com.pingcap.tikv.TiSession.lambda$getCatalog$0(TiSession.java:85) at com.pingcap.tikv.catalog.Catalog.(Catalog.java:135) at com.pingcap.tikv.TiSession.getCatalog(TiSession.java:89) at org.apache.spark.sql.TiContext.(TiContext.scala:38) at io.github.interestinglab.waterdrop.input.batch.Tidb.getDataset(Tidb.scala:28) at io.github.interestinglab.waterdrop.Waterdrop$$anonfun$registerInputTempView$1.apply(Waterdrop.scala:294) at io.github.interestinglab.waterdrop.Waterdrop$$anonfun$registerInputTempView$1.apply(Waterdrop.scala:292) at scala.collection.immutable.List.foreach(List.scala:381) at io.github.interestinglab.waterdrop.Waterdrop$.registerInputTempView(Waterdrop.scala:292) at io.github.interestinglab.waterdrop.Waterdrop$.batchProcessing(Waterdrop.scala:197) at io.github.interestinglab.waterdrop.Waterdrop$.io$github$interestinglab$waterdrop$Waterdrop$$entrypoint(Waterdrop.scala:119) at io.github.interestinglab.waterdrop.Waterdrop$$anonfun$1.apply$mcV$sp(Waterdrop.scala:37) at io.github.interestinglab.waterdrop.Waterdrop$$anonfun$1.apply(Waterdrop.scala:37) at io.github.interestinglab.waterdrop.Waterdrop$$anonfun$1.apply(Waterdrop.scala:37) at scala.util.Try$.apply(Try.scala:192) at io.github.interestinglab.waterdrop.Waterdrop$.main(Waterdrop.scala:37) … 11 more Caused by: shade.io.grpc.StatusRuntimeException: UNKNOWN: tso count should be positive at shade.io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:210) at shade.io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:191) at shade.io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:124) at com.pingcap.tikv.AbstractGRPCClient.lambda$callWithRetry$0(AbstractGRPCClient.java:68) at com.pingcap.tikv.policy.RetryPolicy.callWithRetry(RetryPolicy.java:55) … 31 more

脚本一直报错,找不到。 TiDB 集群信息 Traceback (most recent call last): File “./info_collecting.py”, line 262, in table = execute(item, pro_url, tbl) File “./info_collecting.py”, line 253, in execute raise e urllib2.URLError: <urlopen error [Errno -2] Name or service not known>

可以看下这个 issue 能否帮助到你

谢谢,我一直在看官网的文档,没去看github,严重大失误呀。

嗯,github 上的 issue 也有很多内容的