lightning导入数据报错

🪐 TiDB 技术问题 备份&数据迁移
1

为提高效率，提问时请提供以下信息，问题描述清晰可优先响应。

  • 【TiDB 版本】：4.0.2
  • 【问题描述】：报错日志如下，数据源是 mydumper 导出的一个数据表

[2020/07/20 14:52:26.350 +08:00] [INFO] [version.go:48] [“Welcome to lightning”] [“Release Version”=v3.0.5] [“Git Commit Hash”=7dbd0c6ca304e0836ddeaf44639c530c04f7da15] [“Git Branch”=HEAD] [“UTC Build Time”=“2019-10-25 03:24:52”] [“Go Version”=“go version go1.12 linux/amd64”]
[2020/07/20 14:52:26.350 +08:00] [INFO] [lightning.go:165] [cfg] [cfg="{“id”:1595227946350165773,“lightning”:{“table-concurrency”:6,“index-concurrency”:2,“region-concurrency”:16,“io-concurrency”:5,“check-requirements”:true},“tidb”:{“host”:“192.168.0.123”,“port”:4000,“user”:“root”,“status-port”:10080,“pd-addr”:“192.168.0.123:2379”,“sql-mode”:“ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION”,“distsql-scan-concurrency”:100,“build-stats-concurrency”:20,“index-serial-scan-concurrency”:20,“checksum-table-concurrency”:16},“checkpoint”:{“enable”:false,“schema”:“tidb_lightning_checkpoint”,“driver”:“file”,“keep-after-success”:false},“mydumper”:{“read-block-size”:65536,“batch-size”:107374182400,“batch-import-ratio”:0,“data-source-dir”:"/root/database",“no-schema”:true,“character-set”:“auto”,“csv”:{“separator”:",",“delimiter”:"\"",“header”:false,“trim-last-separator”:false,“not-null”:false,“null”:"",“backslash-escape”:false},“case-sensitive”:false},“black-white-list”:{“do-tables”:null,“do-dbs”:null,“ignore-tables”:null,“ignore-dbs”:[“mysql”,“information_schema”,“performance_schema”,“sys”]},“tikv-importer”:{“addr”:“127.0.0.1:8287”,“backend”:“importer”,“on-duplicate”:“replace”},“post-restore”:{“level-1-compact”:false,“compact”:false,“checksum”:true,“analyze”:false},“cron”:{“switch-mode”:“5m0s”,“log-progress”:“5m0s”},“routes”:null}"]
[2020/07/20 14:52:26.350 +08:00] [INFO] [lightning.go:194] [“load data source start”]
[2020/07/20 14:52:26.351 +08:00] [INFO] [lightning.go:197] [“load data source completed”] [takeTime=635.792µs] []
[2020/07/20 14:52:26.360 +08:00] [INFO] [restore.go:245] [“the whole procedure start”]
[2020/07/20 14:52:26.366 +08:00] [ERROR] [restore.go:260] [“run failed”] [step=0] [error=“TiKV (at 192.168.0.123:3930): strconv.ParseInt: parsing “v4”: invalid syntax”]
[2020/07/20 14:52:26.366 +08:00] [ERROR] [restore.go:266] [“the whole procedure failed”] [takeTime=6.307682ms] [error=“TiKV (at 192.168.0.123:3930): strconv.ParseInt: parsing “v4”: invalid syntax”]
[2020/07/20 14:52:26.366 +08:00] [ERROR] [main.go:59] [“tidb lightning encountered error”] [error=“TiKV (at 192.168.0.123:3930): strconv.ParseInt: parsing “v4”: invalid syntax”] [errorVerbose=“strconv.ParseInt: parsing “v4”: invalid syntax\ TiKV (at 192.168.0.123:3930)\ngithub.com/pingcap/tidb-lightning/lightning/restore.(*RestoreController).checkTiKVVersion.func1\ \t/home/jenkins/agent/workspace/release_tidb_3.0/go/src/github.com/pingcap/tidb-lightning/lightning/restore/restore.go:1137\ngithub.com/pingcap/tidb-lightning/lightning/backend.ForAllStores.func1\ \t/home/jenkins/agent/workspace/release_tidb_3.0/go/src/github.com/pingcap/tidb-lightning/lightning/backend/tikv.go:127\ golang.org/x/sync/errgroup.(*Group).Go.func1\ \t/home/jenkins/agent/workspace/release_tidb_3.0/go/pkg/mod/golang.org/x/sync@v0.0.0-20190911185100-cd5d95a43a6e/errgroup/errgroup.go:57\ runtime.goexit\ \t/usr/local/go/src/runtime/asm_amd64.s:1337”]

2

你好， invalid syntax 报错语法不支持，可以看下 tidb.log 中是否有报错？

3

这是执行导入时间段的日志

[2020/07/20 14:48:34.386 +08:00] [INFO] [server.go:391] [“connection closed”] [conn=14]
[2020/07/20 14:52:26.352 +08:00] [INFO] [server.go:388] [“new connection”] [conn=15] [remoteAddr=192.168.0.123:44564]
[2020/07/20 14:52:26.359 +08:00] [INFO] [set.go:207] [“set session var”] [conn=15] [name=sql_mode] [val=ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION]
[2020/07/20 14:52:26.366 +08:00] [INFO] [server.go:391] [“connection closed”] [conn=15]
[2020/07/20 14:56:07.880 +08:00] [INFO] [gc_worker.go:262] ["[gc worker] starts the whole job"] [uuid=5cdab7251fc0002] [safePoint=418179335529693184] [concurrency=3]
[2020/07/20 14:56:07.882 +08:00] [INFO] [gc_worker.go:949] ["[gc worker] start resolve locks"] [uuid=5cdab7251fc0002] [safePoint=418179335529693184] [concurrency=3]
[2020/07/20 14:56:07.882 +08:00] [INFO] [range_task.go:113] [“range task started”] [name=resolve-locks-runner] [startKey=] [endKey=] [concurrency=3]

4

看下 dump 下来的 schema 信息，辛苦上传下。

5

这个我后来手动建表，设置no-schema = true 好像也不行

6

额，辛苦更新下 tidb lightning 的版本，与 tidb 版本保持一致，select tidb_version();

7

感谢，lightning版本更新为4.0.2后，正常运行了

8

ok~

9

此话题已在最后回复的 1 分钟后被自动关闭。不再允许新回复。