Tispark使用overwrite写入数据后通过tispark sql读取存在延迟

写入方式:

dataset
    .write
    .mode(saveMode = "overwrite")
    .format("jdbc")
    .option("driver", "com.mysql.jdbc.Driver")
        .option("url", "jdbc:mysql://192.168.10.201:4000/test?&autoReconnect=true&failOverReadOnly=false")
    .option("useSSL", "false")
    .option("dbtable", "t_order_pay_subtotal")
    .option("isolationLevel", "NONE")
    .option("truncate", "true")
    .option("user", "test")
    .option("password", "test@123")

tisparksql查询:

select * from `hydee`.t_order_pay_subtotal

直接使用mysql客户端数据能实时查询更新后数据,使用tispark查询延迟10-20s才能查到。
尝试saveMode = "append"模式,使用tisparksql也能实时查到更新后的数据

1赞

overwrite 是先 drop table 然后再 create table,当时 catalog 是每10s刷新一次,所以需要有一定延迟才能查到,是已知问题。

1赞

@zhengyunpeng

首先感谢你使用 TiSpark 并且尝试我们的 batch write 功能。

请问是否是 drop and create 同名 table 呢? 如果是,这是预期行为。 我们正准备重新设计这一模块,优化这一行为。

1赞

overwrite的时候使用option(“truncate”, “true”)参数,不知道tidb使用什么方式处理truncate语句的,按mysql的truncate是drop and create

直接使用mysql客户端查询数据的时候没有延迟的

@zhengyunpeng 您好,这个问题我提了一个PR,不知道您有没有时间帮忙做一下测试?看看是否已经解决了您的问题 https://github.com/pingcap/tispark/pull/1108

可以的

jar包可以在这里下载,请使用spark-2.3.x或者spark-2.4.x,感谢

链接:https://pan.baidu.com/s/11J8iQrQy2xuFVYH3LNi1gQ 密码:ctn9

可以了,测试结果如下: tispark-assembly-2.3.0-SNAPSHOT-PR-1108.jar版:

scala> var dataset = spark.sql("select 1,'a'")
dataset: org.apache.spark.sql.DataFrame = [1: int, a: string]

scala> spark.sql("select * from test").show
2019-09-18 16:20:59 WARN  ObjectStore:568 - Failed to get database test, returning NoSuchObjectException
2019-09-18 16:20:59 WARN  ObjectStore:568 - Failed to get database test, returning NoSuchObjectException
2019-09-18 16:20:59 WARN  KVErrorHandler:222 - Stale Epoch encountered for region [{Region[2] ConfVer[1] Version[21] Store[1] KeyRange[t\200\000\000\000\000\000\000+]:[]}]
2019-09-18 16:20:59 WARN  KVErrorHandler:120 - Failed to send notification back to driver since CacheInvalidateCallBack is null in executor node.
2019-09-18 16:20:59 WARN  RegionStoreClient:464 - Re-splitting region task due to region error:EpochNotMatch current epoch of region 2 is conf_ver: 1 version: 22, but you sent conf_ver: 1 version: 21
+---+---+
|  1|  a|
+---+---+
|  1|  a|
+---+---+


scala> 

scala> 

scala> 

scala> var dataset = spark.sql("select 2,'b'")
dataset: org.apache.spark.sql.DataFrame = [2: int, b: string]

scala> dataset.write.mode(saveMode = "overwrite").format("jdbc").option("driver", "com.mysql.jdbc.Driver").option("url", "jdbc:mysql://192.168.10.111:4000/test?&autoReconnect=true&failOverReadOnly=false").option("useSSL", "false").option("dbtable", "test").option("isolationLevel", "NONE").option("user", "root").option("password", "hydee@123").save()

scala> spark.sql("select * from test").show
2019-09-18 16:21:31 WARN  ObjectStore:568 - Failed to get database test, returning NoSuchObjectException
2019-09-18 16:21:31 WARN  ObjectStore:568 - Failed to get database test, returning NoSuchObjectException
2019-09-18 16:21:32 WARN  KVErrorHandler:222 - Stale Epoch encountered for region [{Region[2] ConfVer[1] Version[22] Store[1] KeyRange[t\200\000\000\000\000\000\000.]:[]}]
2019-09-18 16:21:32 WARN  KVErrorHandler:120 - Failed to send notification back to driver since CacheInvalidateCallBack is null in executor node.
2019-09-18 16:21:32 WARN  RegionStoreClient:464 - Re-splitting region task due to region error:EpochNotMatch current epoch of region 2 is conf_ver: 1 version: 23, but you sent conf_ver: 1 version: 22
+---+---+
|  2|  b|
+---+---+
|  2|  b|
+---+---+

将jar包切换到tispark-core-2.1.2-spark_2.3-jar-with-dependencies.jar,重启spark-shell:

scala> spark.sql("select * from test").show
2019-09-18 16:23:22 WARN  ObjectStore:568 - Failed to get database test, returning NoSuchObjectException
2019-09-18 16:23:22 WARN  ObjectStore:568 - Failed to get database test, returning NoSuchObjectException
+---+---+
|  2|  b|
+---+---+
|  2|  b|
+---+---+


scala> var dataset = spark.sql("select 1,'a'")
dataset: org.apache.spark.sql.DataFrame = [1: int, a: string]

scala> dataset.write.mode(saveMode = "overwrite").format("jdbc").option("driver", "com.mysql.jdbc.Driver").option("url", "jdbc:mysql://192.168.10.111:4000/test?&autoReconnect=true&failOverReadOnly=false").option("useSSL", "false").option("dbtable", "test").option("isolationLevel", "NONE").option("user", "root").option("password", "hydee@123").save()
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.

scala> spark.sql("select * from test").show
2019-09-18 16:23:47 WARN  ObjectStore:568 - Failed to get database test, returning NoSuchObjectException
2019-09-18 16:23:47 WARN  ObjectStore:568 - Failed to get database test, returning NoSuchObjectException
2019-09-18 16:23:47 WARN  KVErrorHandler:193 - Stale Epoch encountered for region [{Region[2] ConfVer[1] Version[23] Store[1] KeyRange[t\200\000\000\000\000\000\0001]:[]}]
2019-09-18 16:23:47 WARN  KVErrorHandler:104 - Failed to send notification back to driver since CacheInvalidateCallBack is null in executor node.
2019-09-18 16:23:47 WARN  RegionStoreClient:564 - Re-splitting region task due to region error:EpochNotMatch current epoch of region 2 is conf_ver: 1 version: 24, but you sent conf_ver: 1 version: 23
+---+---+
|  2|  b|
+---+---+
|  2|  b|
+---+---+


scala> spark.sql("select * from test").show
2019-09-18 16:25:48 WARN  ObjectStore:568 - Failed to get database test, returning NoSuchObjectException
2019-09-18 16:25:48 WARN  ObjectStore:568 - Failed to get database test, returning NoSuchObjectException
2019-09-18 16:25:49 WARN  KVErrorHandler:193 - Stale Epoch encountered for region [{Region[2] ConfVer[1] Version[23] Store[1] KeyRange[t\200\000\000\000\000\000\0001]:[]}]
2019-09-18 16:25:49 WARN  KVErrorHandler:104 - Failed to send notification back to driver since CacheInvalidateCallBack is null in executor node.
2019-09-18 16:25:49 WARN  RegionStoreClient:564 - Re-splitting region task due to region error:EpochNotMatch current epoch of region 2 is conf_ver: 1 version: 24, but you sent conf_ver: 1 version: 23
+---+---+
|  1|  a|
+---+---+
|  1|  a|
+---+---+

目前这两个tispark-assembly-2.3.0-SNAPSHOT的jar只支持读取tidb版本3.0及以上的数据吗?我们环境tidb 2.1.15的,使用这两个jar都报错,把spark升级了也不行。我都是单机环境测试,TiDB-v3.0.0-rc.1

这个jar包应该可以读取tidb-2.1.15,请问具体报什么错?建议可以用local模式运行一下

好的,我试一下。
tidb-2.1.15 spark 2.3.3 报错信息:

16-09-2019 18:19:50 CST StructuredStream INFO - 2019-09-16 18:19:50 WARN  TaskSetManager:66 - Lost task 0.0 in stage 0.0 (TID 0, 192.168.10.204, executor 2): java.io.InvalidClassException: org.apache.spark.sql.tispark.TiRDD; local class incompatible: stream classdesc serialVersionUID = -6650751149050193034, local class serialVersionUID = 1833683067262546845
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:616)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1630)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.lang.reflect.Method.invoke(Method.java:498)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.lang.reflect.Method.invoke(Method.java:498)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.lang.reflect.Method.invoke(Method.java:498)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.lang.reflect.Method.invoke(Method.java:498)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.lang.reflect.Method.invoke(Method.java:498)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:75)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:114)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:80)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at org.apache.spark.scheduler.Task.run(Task.scala:109)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
16-09-2019 18:19:50 CST StructuredStream INFO - 	at java.lang.Thread.run(Thread.java:745)
16-09-2019 18:19:50 CST StructuredStream INFO -

请问是local模式运行的吗?

您是说spark运行模式吗?不是local,是standalone模式

建议先用local运行试试 (standalone模式的话需要先升级集群上的tispark,并重启)

使用local模式可以的,谢谢

非常感谢,等我们发布下个fix版本,可以考虑升级一下集群

好的,谢谢