为提高效率,请提供以下信息,问题描述清晰能够更快得到解决:
【 TiDB 使用环境】
按照官网的 TiDB 数据库快速上手指南 部署的单节点Linux集群做测试
【概述】 场景 + 问题概述
【备份和数据迁移策略逻辑】
csv文件是从google cloud 的bigquery导出来的,格式如下:
[root@iZwz9bg08mzvex3rn5k06wZ ~]# more csv/test.tokens.csv
address,symbol,name,decimals,total_supply,block_timestamp,block_number,block_hash
xxxxxxxxxxxxxxxx,LOKI,Loki,18,100000000000000000000000000,2021-07-13 02:40:18 UTC,12816224,xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx,"","",9,0,2021-07-13 03:12:25 UTC,12816355,xxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxx,MET.KEY,MET.KEY,18,1000000000000000000000000000,2021-07-13 07:50:51 UTC,12817550,xxxxxxxxxxxxxxxxxxxx
【背景】 做过哪些操作
有试过用tidb-lightning.toml配置文件的方式,还是提示一样的错误
配置如下:
[lightning]
# 日志
level = "info"
file = "tidb-lightning.log"
[tikv-importer]
# 选择使用的 local 后端
backend = "local"
# 设置排序的键值对的临时存放地址,目标路径需要是一个空目录
sorted-kv-dir = "/tidb-data/sorted-kv-dir/"
[mydumper]
no-schema = true
data-source-dir = /nfs_storage/tmp/csv/
[mydumper.csv]
# 字段分隔符,必须为 ASCII 字符。
separator = ','
# 引用定界符,可以为 ASCII 字符或空字符。
delimiter = '"'
# CSV 文件是否包含表头。
# 如果为 true,首行将会被跳过。
header = true
# CSV 是否包含 NULL。
# 如果为 true,CSV 文件的任何列都不能解析为 NULL。
not-null = false
# 如果 `not-null` 为 false(即 CSV 可以包含 NULL),
# 为以下值的字段将会被解析为 NULL。
null = '\N'
# 是否解析字段内的反斜线转义符。
backslash-escape = true
# 是否移除以分隔符结束的行。
trim-last-separator = false
[tidb]
# 目标集群的信息
host = "192.168.11.105"
port = 4000
user = "root"
password = ""
# 表架构信息在从 TiDB 的“状态端口”获取。
status-port = 10080
# 集群 pd 的地址
pd-addr = "192.168.11.105:2379"
【现象】 业务和数据库现象
暂无
【问题】 当前遇到的问题
- 使用tidb-lightning命令行执行:
tidb-lightning -d /root/csv/ --backend local --pd-urls "192.168.11.105:2379" --tidb-host "192.168.11.105" --tidb-port 4000 --tidb-status 10080 --tidb-user root --no-schema --sorted-kv-dir "/tidb-data/sorted-kv-dir/"
Verbose debug logs will be written to /tmp/lightning.log.2021-07-15T12.14.46+0800
Error: read /root/csv: is a directory
tidb lightning encountered error: read /root/csv: is a directory
[root@iZwz9bg08mzvex3rn5k06wZ tmp]# more /tmp/lightning.log.2021-07-15T14.43.34+0800
[2021/07/15 14:43:34.138 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.1.0] [git-hash=f2fcbfa5a9783e33ff44b696920d8bbe6a19eaa6] [git-branch=heads/refs/tags/v5.1.0] [go-version=go1.16.4] [utc-build-time="2021-06-24 07:09:26"] [race-en
abled=false]
[2021/07/15 14:43:34.139 +08:00] [INFO] [lightning.go:219] [cfg] [cfg="{\"id\":1626331414138919458,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":4,\"io-concurrency\":5,\"check-requirements\":true,\"meta-schema-name\":\"lightn
ing_metadata\"},\"tidb\":{\"host\":\"192.168.11.105\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"192.168.11.105:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"
\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpo
int\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file:///root/csv\",\"character-set\":\"auto\",\"csv\":{\"separator\":\",\",\"d
elimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRI
CS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":true,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true,\"ignore-data-columns\":null},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"local\",\"on-duplicate\":\"replace\",
\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"/tidb-data/sorted-kv-dir/\",\"disk-quota\":74968948736,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-r
estore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-pa
th\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/07/15 14:43:34.141 +08:00] [INFO] [lightning.go:282] ["load data source start"]
[2021/07/15 14:43:34.141 +08:00] [INFO] [lightning.go:285] ["load data source completed"] [takeTime=190.771µs] []
[2021/07/15 14:43:34.141 +08:00] [INFO] [checkpoints.go:979] ["open checkpoint file failed, going to create a new one"] [path=/tmp/tidb_lightning_checkpoint.pb] [error="open /tmp/tidb_lightning_checkpoint.pb: no such file or directory"]
[2021/07/15 14:43:34.145 +08:00] [INFO] [local.go:949] ["multi ingest support"]
[2021/07/15 14:43:34.146 +08:00] [INFO] [restore.go:369] ["the whole procedure start"]
[2021/07/15 14:43:34.149 +08:00] [INFO] [restore.go:660] ["restore all schema start"]
[2021/07/15 14:43:34.150 +08:00] [ERROR] [loader.go:59] ["failed to extract table schema"] [Path=] [error="read /root/csv: is a directory"]
[2021/07/15 14:43:34.150 +08:00] [ERROR] [restore.go:679] ["restore all schema failed"] [takeTime=826.725µs] [error="read /root/csv: is a directory"]
[2021/07/15 14:43:34.150 +08:00] [ERROR] [restore.go:388] ["run failed"] [step=2] [error="read /root/csv: is a directory"]
[2021/07/15 14:43:34.150 +08:00] [ERROR] [restore.go:399] ["the whole procedure failed"] [takeTime=3.688991ms] [error="read /root/csv: is a directory"]
[2021/07/15 14:43:34.150 +08:00] [ERROR] [main.go:90] ["tidb lightning encountered error stack info"] [error="read /root/csv: is a directory"] [errorVerbose="read /root/csv: is a directory\
github.com/pingcap/errors.AddStack\
\tgithub.com/pingcap/errors@v0.11.5-0.2
0201126102027-b0a155152ca3/errors.go:174\
github.com/pingcap/errors.Trace\
\tgithub.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/juju_adaptor.go:15\
github.com/pingcap/br/pkg/lightning/mydump.ExportStatement\
\tgithub.com/pingcap/br/pkg/lightning/mydump
/reader.go:88\
github.com/pingcap/br/pkg/lightning/mydump.(*MDTableMeta).GetSchema\
\tgithub.com/pingcap/br/pkg/lightning/mydump/loader.go:57\
github.com/pingcap/br/pkg/lightning/restore.(*restoreSchemaWorker).makeJobs\
\tgithub.com/pingcap/br/pkg/lightning/restore
/restore.go:488\
github.com/pingcap/br/pkg/lightning/restore.(*Controller).restoreSchema\
\tgithub.com/pingcap/br/pkg/lightning/restore/restore.go:678\
github.com/pingcap/br/pkg/lightning/restore.(*Controller).Run\
\tgithub.com/pingcap/br/pkg/lightning/restore/rest
ore.go:375\
github.com/pingcap/br/pkg/lightning.(*Lightning).run\
\tgithub.com/pingcap/br/pkg/lightning/lightning.go:312\
github.com/pingcap/br/pkg/lightning.(*Lightning).RunOnce\
\tgithub.com/pingcap/br/pkg/lightning/lightning.go:192\
main.main.func2\
\tcommand-li
ne-arguments/main.go:86\
main.main\
\tcommand-line-arguments/main.go:87\
runtime.main\
\truntime/proc.go:225\
runtime.goexit\
\truntime/asm_amd64.s:1371"]
[2021/07/15 14:43:34.150 +08:00] [ERROR] [main.go:91] ["tidb lightning encountered error"] [error="read /root/csv: is a directory"]
- 使用配置文件的方式执行:
[root@iZwz9bg08mzvex3rn5k06wZ ~]# tidb-lightning -config tidb-lightning.toml
Verbose debug logs will be written to tidb-lightning.log
Error: read /nfs_storage/tmp/csv: is a directory
tidb lightning encountered error: read /nfs_storage/tmp/csv: is a directory
报的错跟用命令行一样。
csv文件路径
[root@iZwz9bg08mzvex3rn5k06wZ ~]# tree /root/csv/
/root/csv/
└── test.tokens.csv
【业务影响】
暂无
【TiDB 版本】
v5.1.0
【附件】
- 相关日志、配置文件、Grafana 监控(https://metricstool.pingcap.com/)
- TiUP Cluster Display 信息
- TiUP CLuster Edit config 信息
- TiDB-Overview 监控
- 对应模块的 Grafana 监控(如有 BR、TiDB-binlog、TiCDC 等)
- 对应模块日志(包含问题前后 1 小时日志)
若提问为性能优化、故障排查类问题,请下载脚本运行。终端输出的打印结果,请务必全选并复制粘贴上传。