lightning部署问题error: config file contained unknown configuration options: mydumper.filter

🪐 TiDB 技术问题 备份&数据迁移
1

部署完lightning和importer ，importer正常启动，lightning启动脚本启动报错：tidb lightning encountered error: config file contained unknown configuration options: mydumper.filter

补充日志报错：
[2020/07/14 16:52:02.216 +08:00] [ERROR] [main.go:78] [“tidb lightning encountered error stack info”] [error=“config file contained unknown configuration options: mydumper.filter”] [errorVerbose=“config file contained unknown configuration options: mydumper.filter\ngithub.com/pingcap/tidb-lightning/lightning/config.(*Config).LoadFromTOML\ \t/home/jenkins/agent/workspace/build_lightning_master/go/src/github.com/pingcap/tidb-lightning/lightning/config/config.go:340\ngithub.com/pingcap/tidb-lightning/lightning/config.(*Config).LoadFromGlobal\ \t/home/jenkins/agent/workspace/build_lightning_master/go/src/github.com/pingcap/tidb-lightning/lightning/config/config.go:269\ngithub.com/pingcap/tidb-lightning/lightning.(*Lightning).RunOnce\ \t/home/jenkins/agent/workspace/build_lightning_master/go/src/github.com/pingcap/tidb-lightning/lightning/lightning.go:136\ main.main\ \t/home/jenkins/agent/workspace/build_lightning_master/go/src/github.com/pingcap/tidb-lightning/cmd/tidb-lightning/main.go:75\ runtime.main\ \t/usr/local/go/src/runtime/proc.go:203\ runtime.goexit\ \t/usr/local/go/src/runtime/asm_amd64.s:1357”]
[2020/07/14 16:52:02.216 +08:00] [ERROR] [main.go:79] [“tidb lightning encountered error”] [error=“config file contained unknown configuration options: mydumper.filter”]

再次补充：tidb集群时v4.0 另外我将lightning和tidb部署在了同一台机器。

2

lightning 配置文件错误，可以上传下 config file 这边看下或者自己排查下
mydumper.filter

3

[lightning]
pprof-port = 8289
index-concurrency = 2
table-concurrency = 6
io-concurrency = 5
[checkpoint]
enable = true
schema = “tidb_lightning_checkpoint”
driver = “file”
[tikv-importer]
addr = “10.23.190.145:8287”
[mydumper]
read-block-size = 65536 # Byte (默认为 64 KB)
data-source-dir = “/data/csv/”
no-schema = false
character-set = “auto”
filter = [’.’]
[mydumper.csv]
separator = ‘,’
delimiter = ‘"’
header = false
not-null = false
null = ‘\N’
backslash-escape = true
trim-last-separator = false
[tidb]
host = “10.23.190.145”
port = 4000
user = “root”
password = “Centos123#”
status-port = 10080
pd-addr = “10.23.190.145:2379”
log-level = “error”
build-stats-concurrency = 20
distsql-scan-concurrency = 100
index-serial-scan-concurrency = 20
checksum-table-concurrency = 16
sql-mode = “STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION”
max-allowed-packet = 67_108_864
[post-restore]
checksum = true
level-1-compact = false
compact = false
analyze = true
[cron]
switch-mode = “5m”
log-progress = “5m”

以上为配置文件，并没有这个配置项。

4

经过修改配置，注销 filter = [’ . ’]行，现在出现新的报错 ：tidb lightning encountered error: missing {schema}-schema-create.sql

5

mydumper 目录下是否有类似形式的文件呢？没有可以手动创建以下