【 TiDB 使用环境】测试
【 TiDB 版本】6.5.2
【复现路径】在本地创建了一个tidb集群(6.5.2),以及一个spark集群3.1.2,在用spark-shell的时候无法看到tidb集群的数据库,启动spark-shell的语句为 /bin/spark-shell --jars …/jars/mysql-connector-java-8.0.19.jar …/jars/tispark-assembly-3.1_2.12-3.2.1.jar --master local[2]
配置文件为
spark.eventLog.enabled true
spark.eventLog.dir hdfs://node02:8020/spark_log
spark.eventLog.compress true
spark.master 192.168.0.106:7077
spark.sql.extensions org.apache.spark.sql.TiExtensions
spark.tispark.pd.addresses 192.168.0.106:2379
spark.sql.catalog.tidb_catalog org.apache.spark.sql.catalyst.catalog.TiCatalog
spark.sql.catalog.tidb_catalog.pd.addresses 192.168.0.106:2379
spark.tispark.grpc.framesize 1564087970
spark.tispark.grpc.timeout_in_sec 100
spark.tispark.meta.reload_period_in_sec 60
spark.tispark.plan.allowaggpushdown true
spark.tispark.index.scan_batch_size 500000
spark.tispark.index.scan_concurrency 2
spark.tispark.table.scan_concurrency 256
spark.tispark.tidb.addr 192.168.0.106
spark.tispark.tidb.port 4000
spark.sql.tidb.user root
spark.sql.tidb.password 123456
spark.tispark.isolation_read_engines tikv
spark.tispark.tikv.addr 192.168.0.106
spark.tispark.tikv.port 20160
spark.sql.auth.enable true
spark.sql.tidb.addr 192.168.0.106
spark.sql.tidb.port 4000
【遇到的问题:问题现象及影响】会报一个 WARN TiConfiguration: Unable to find tikv.properties,并且看不到tidb集群里的数据库,spark-shell运行记录如下
scala> spark.sql(“show databases”).show()
Loading class com.mysql.jdbc.Driver'. This is deprecated. The new driver class is
com.mysql.cj.jdbc.Driver’. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
23/05/17 11:35:53 WARN TiConfiguration: Unable to find tikv.properties
23/05/17 11:35:57 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 2.3.0
23/05/17 11:35:57 WARN ObjectStore: setMetaStoreSchemaVersion called but recording version is disabled: version = 2.3.0, comment = Set by MetaStore parallels@192.168.0.106
±--------+
|namespace|
±--------+
| default|
±--------+
【资源配置】进入到 TiDB Dashboard -集群信息 (Cluster Info) -主机(Hosts) 截图此页面
【附件:截图/日志/监控】
请各位大神指点