可以了,测试结果如下:
tispark-assembly-2.3.0-SNAPSHOT-PR-1108.jar版:
scala> var dataset = spark.sql("select 1,'a'")
dataset: org.apache.spark.sql.DataFrame = [1: int, a: string]
scala> spark.sql("select * from test").show
2019-09-18 16:20:59 WARN ObjectStore:568 - Failed to get database test, returning NoSuchObjectException
2019-09-18 16:20:59 WARN ObjectStore:568 - Failed to get database test, returning NoSuchObjectException
2019-09-18 16:20:59 WARN KVErrorHandler:222 - Stale Epoch encountered for region [{Region[2] ConfVer[1] Version[21] Store[1] KeyRange[t200 00 00 00 00 00 00+]:[]}]
2019-09-18 16:20:59 WARN KVErrorHandler:120 - Failed to send notification back to driver since CacheInvalidateCallBack is null in executor node.
2019-09-18 16:20:59 WARN RegionStoreClient:464 - Re-splitting region task due to region error:EpochNotMatch current epoch of region 2 is conf_ver: 1 version: 22, but you sent conf_ver: 1 version: 21
+---+---+
| 1| a|
+---+---+
| 1| a|
+---+---+
scala>
scala>
scala>
scala> var dataset = spark.sql("select 2,'b'")
dataset: org.apache.spark.sql.DataFrame = [2: int, b: string]
scala> dataset.write.mode(saveMode = "overwrite").format("jdbc").option("driver", "com.mysql.jdbc.Driver").option("url", "jdbc:mysql://192.168.10.111:4000/test?&autoReconnect=true&failOverReadOnly=false").option("useSSL", "false").option("dbtable", "test").option("isolationLevel", "NONE").option("user", "root").option("password", "hydee@123").save()
scala> spark.sql("select * from test").show
2019-09-18 16:21:31 WARN ObjectStore:568 - Failed to get database test, returning NoSuchObjectException
2019-09-18 16:21:31 WARN ObjectStore:568 - Failed to get database test, returning NoSuchObjectException
2019-09-18 16:21:32 WARN KVErrorHandler:222 - Stale Epoch encountered for region [{Region[2] ConfVer[1] Version[22] Store[1] KeyRange[t200 00 00 00 00 00 00.]:[]}]
2019-09-18 16:21:32 WARN KVErrorHandler:120 - Failed to send notification back to driver since CacheInvalidateCallBack is null in executor node.
2019-09-18 16:21:32 WARN RegionStoreClient:464 - Re-splitting region task due to region error:EpochNotMatch current epoch of region 2 is conf_ver: 1 version: 23, but you sent conf_ver: 1 version: 22
+---+---+
| 2| b|
+---+---+
| 2| b|
+---+---+
将jar包切换到tispark-core-2.1.2-spark_2.3-jar-with-dependencies.jar,重启spark-shell:
scala> spark.sql("select * from test").show
2019-09-18 16:23:22 WARN ObjectStore:568 - Failed to get database test, returning NoSuchObjectException
2019-09-18 16:23:22 WARN ObjectStore:568 - Failed to get database test, returning NoSuchObjectException
+---+---+
| 2| b|
+---+---+
| 2| b|
+---+---+
scala> var dataset = spark.sql("select 1,'a'")
dataset: org.apache.spark.sql.DataFrame = [1: int, a: string]
scala> dataset.write.mode(saveMode = "overwrite").format("jdbc").option("driver", "com.mysql.jdbc.Driver").option("url", "jdbc:mysql://192.168.10.111:4000/test?&autoReconnect=true&failOverReadOnly=false").option("useSSL", "false").option("dbtable", "test").option("isolationLevel", "NONE").option("user", "root").option("password", "hydee@123").save()
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
scala> spark.sql("select * from test").show
2019-09-18 16:23:47 WARN ObjectStore:568 - Failed to get database test, returning NoSuchObjectException
2019-09-18 16:23:47 WARN ObjectStore:568 - Failed to get database test, returning NoSuchObjectException
2019-09-18 16:23:47 WARN KVErrorHandler:193 - Stale Epoch encountered for region [{Region[2] ConfVer[1] Version[23] Store[1] KeyRange[t200 00 00 00 00 00 001]:[]}]
2019-09-18 16:23:47 WARN KVErrorHandler:104 - Failed to send notification back to driver since CacheInvalidateCallBack is null in executor node.
2019-09-18 16:23:47 WARN RegionStoreClient:564 - Re-splitting region task due to region error:EpochNotMatch current epoch of region 2 is conf_ver: 1 version: 24, but you sent conf_ver: 1 version: 23
+---+---+
| 2| b|
+---+---+
| 2| b|
+---+---+
scala> spark.sql("select * from test").show
2019-09-18 16:25:48 WARN ObjectStore:568 - Failed to get database test, returning NoSuchObjectException
2019-09-18 16:25:48 WARN ObjectStore:568 - Failed to get database test, returning NoSuchObjectException
2019-09-18 16:25:49 WARN KVErrorHandler:193 - Stale Epoch encountered for region [{Region[2] ConfVer[1] Version[23] Store[1] KeyRange[t200 00 00 00 00 00 001]:[]}]
2019-09-18 16:25:49 WARN KVErrorHandler:104 - Failed to send notification back to driver since CacheInvalidateCallBack is null in executor node.
2019-09-18 16:25:49 WARN RegionStoreClient:564 - Re-splitting region task due to region error:EpochNotMatch current epoch of region 2 is conf_ver: 1 version: 24, but you sent conf_ver: 1 version: 23
+---+---+
| 1| a|
+---+---+
| 1| a|
+---+---+