读取大量数据UNKNOWN: tso count should be positive

为提高效率,提问时请提供以下信息,问题描述清晰可优先响应。

  • 【TiDB 版本】:3.0.5
  • 【问题描述】:使用scala编写代码,读取TiDB中一个千万级别的表报错。

20/01/16 15:57:41 WARN ConcreteBackOffer: backoffer.maxSleep 5000ms is exceeded, errors:
1.shade.io.grpc.StatusRuntimeException: UNKNOWN: tso count should be positive
2.shade.io.grpc.StatusRuntimeException: UNKNOWN: tso count should be positive
3.shade.io.grpc.StatusRuntimeException: UNKNOWN: tso count should be positive
Exception in thread “main” java.lang.Exception: com.pingcap.tikv.exception.GrpcException: retry is exhausted.
at io.github.interestinglab.waterdrop.Waterdrop$.main(Waterdrop.scala:42)
at io.github.interestinglab.waterdrop.Waterdrop.main(Waterdrop.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:906)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:197)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:227)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:136)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: com.pingcap.tikv.exception.GrpcException: retry is exhausted.
at com.pingcap.tikv.util.ConcreteBackOffer.doBackOff(ConcreteBackOffer.java:124)
at com.pingcap.tikv.operation.PDErrorHandler.handleRequestError(PDErrorHandler.java:54)
at com.pingcap.tikv.policy.RetryPolicy.callWithRetry(RetryPolicy.java:59)
at com.pingcap.tikv.AbstractGRPCClient.callWithRetry(AbstractGRPCClient.java:65)
at com.pingcap.tikv.PDClient.getTimestamp(PDClient.java:75)
at com.pingcap.tikv.TiSession.getTimestamp(TiSession.java:56)
at com.pingcap.tikv.TiSession.createSnapshot(TiSession.java:60)
at com.pingcap.tikv.TiSession.lambda$getCatalog$0(TiSession.java:85)
at com.pingcap.tikv.catalog.Catalog.(Catalog.java:135)
at com.pingcap.tikv.TiSession.getCatalog(TiSession.java:89)
at org.apache.spark.sql.TiContext.(TiContext.scala:38)
at io.github.interestinglab.waterdrop.input.batch.Tidb.getDataset(Tidb.scala:28)
at io.github.interestinglab.waterdrop.Waterdrop$$anonfun$registerInputTempView$1.apply(Waterdrop.scala:294)
at io.github.interestinglab.waterdrop.Waterdrop$$anonfun$registerInputTempView$1.apply(Waterdrop.scala:292)
at scala.collection.immutable.List.foreach(List.scala:381)
at io.github.interestinglab.waterdrop.Waterdrop$.registerInputTempView(Waterdrop.scala:292)
at io.github.interestinglab.waterdrop.Waterdrop$.batchProcessing(Waterdrop.scala:197)
at io.github.interestinglab.waterdrop.Waterdrop$.io$github$interestinglab$waterdrop$Waterdrop$$entrypoint(Waterdrop.scala:119)
at io.github.interestinglab.waterdrop.Waterdrop$$anonfun$1.apply$mcV$sp(Waterdrop.scala:37)
at io.github.interestinglab.waterdrop.Waterdrop$$anonfun$1.apply(Waterdrop.scala:37)
at io.github.interestinglab.waterdrop.Waterdrop$$anonfun$1.apply(Waterdrop.scala:37)
at scala.util.Try$.apply(Try.scala:192)
at io.github.interestinglab.waterdrop.Waterdrop$.main(Waterdrop.scala:37)
… 11 more
Caused by: shade.io.grpc.StatusRuntimeException: UNKNOWN: tso count should be positive
at shade.io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:210)
at shade.io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:191)
at shade.io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:124)
at com.pingcap.tikv.AbstractGRPCClient.lambda$callWithRetry$0(AbstractGRPCClient.java:68)
at com.pingcap.tikv.policy.RetryPolicy.callWithRetry(RetryPolicy.java:55)
… 31 more

脚本一直报错,找不到。
TiDB 集群信息
Traceback (most recent call last):
File “./info_collecting.py”, line 262, in
table = execute(item, pro_url, tbl)
File “./info_collecting.py”, line 253, in execute
raise e
urllib2.URLError: <urlopen error [Errno -2] Name or service not known>

1 个赞

可以看下这个 issue 能否帮助到你

https://github.com/pingcap/tispark/issues/558

谢谢,我一直在看官网的文档,没去看github,严重大失误呀。

嗯,github 上的 issue 也有很多内容的

此话题已在最后回复的 1 分钟后被自动关闭。不再允许新回复。