lightning导入tpcc数据失败 failed to cast `NULL` as int(11)

  • 【TiDB 版本】:v3.1.1
  • 【问题描述】:手动部署importer和lightning

启动lightning后显示

Error: in file /root/csv/tpcc.bmsql_oorder.csv:0 at offset 91669: failed to cast NULL as int(11) for column o_carrier_id (#5): [types:1292]Truncated incorrect FLOAT value: ‘NULL’

lightning log文件最后显示:

[2020/05/28 16:16:00.682 +08:00] [ERROR] [main.go:75] [“tidb lightning encountered error”] [error=“in file /root/csv/tpcc.bmsql_oorder.csv:0 at offset 91669: failed to cast NULL as int(11) for column o_carrier_id (#5): [types:1292]Truncated incorrect FLOAT value: ‘NULL’”] [errorVerbose=“[types:1292]Truncated incorrect FLOAT value: ‘NULL’
github.com/pingcap/errors.AddStack
\t/home/jenkins/agent/workspace/release_tidb_3.1/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20190809092503-95897b64e011/errors.go:174
github.com/pingcap/parser/terror.(*Error).GenWithStackByArgs
\t/home/jenkins/agent/workspace/release_tidb_3.1/go/pkg/mod/github.com/pingcap/parser@v0.0.0-20200317021010-cd90cc2a7d87/terror/terror.go:243
github.com/pingcap/tidb/types.getValidFloatPrefix
\t/home/jenkins/agent/workspace/release_tidb_3.1/go/pkg/mod/github.com/pingcap/tidb@v0.0.0-20200317142013-5268094afe05/types/convert.go:669
github.com/pingcap/tidb/types.getValidIntPrefix
\t/home/jenkins/agent/workspace/release_tidb_3.1/go/pkg/mod/github.com/pingcap/tidb@v0.0.0-20200317142013-5268094afe05/types/convert.go:368
github.com/pingcap/tidb/types.StrToInt
\t/home/jenkins/agent/workspace/release_tidb_3.1/go/pkg/mod/github.com/pingcap/tidb@v0.0.0-20200317142013-5268094afe05/types/convert.go:280
github.com/pingcap/tidb/types.(*Datum).toSignedInteger
\t/home/jenkins/agent/workspace/release_tidb_3.1/go/pkg/mod/github.com/pingcap/tidb@v0.0.0-20200317142013-5268094afe05/types/datum.go:1538
github.com/pingcap/tidb/types.(*Datum).convertToInt
\t/home/jenkins/agent/workspace/release_tidb_3.1/go/pkg/mod/github.com/pingcap/tidb@v0.0.0-20200317142013-5268094afe05/types/datum.go:966
github.com/pingcap/tidb/types.(*Datum).ConvertTo
\t/home/jenkins/agent/workspace/release_tidb_3.1/go/pkg/mod/github.com/pingcap/tidb@v0.0.0-20200317142013-5268094afe05/types/datum.go:792
github.com/pingcap/tidb/table.CastValue
\t/home/jenkins/agent/workspace/release_tidb_3.1/go/pkg/mod/github.com/pingcap/tidb@v0.0.0-20200317142013-5268094afe05/table/column.go:171
github.com/pingcap/tidb-lightning/lightning/backend.(*tableKVEncoder).Encode
\t/home/jenkins/agent/workspace/release_tidb_3.1/go/src/github.com/pingcap/tidb-lightning/lightning/backend/sql2kv.go:169
github.com/pingcap/tidb-lightning/lightning/restore.(*chunkRestore).encodeLoop
\t/home/jenkins/agent/workspace/release_tidb_3.1/go/src/github.com/pingcap/tidb-lightning/lightning/restore/restore.go:1743
github.com/pingcap/tidb-lightning/lightning/restore.(*chunkRestore).restore
\t/home/jenkins/agent/workspace/release_tidb_3.1/go/src/github.com/pingcap/tidb-lightning/lightning/restore/restore.go:1811
github.com/pingcap/tidb-lightning/lightning/restore.(*TableRestore).restoreEngine.func1
\t/home/jenkins/agent/workspace/release_tidb_3.1/go/src/github.com/pingcap/tidb-lightning/lightning/restore/restore.go:934
runtime.goexit
\t/usr/local/go/src/runtime/asm_amd64.s:1357
failed to cast NULL as int(11) for column o_carrier_id (#5)
in file /root/csv/tpcc.bmsql_oorder.csv:0 at offset 91669”]

配置文件如下:

ligtning:

importer:

看下 /root/csv/tpcc.bmsql_oorder 文件中 偏移量 :91669 , o_carrier_id 列的定义

偏移量91669是第91669行吗

是我配置文件跟空值相关的一些参数配置的不对?

你好,

看下数据中对应 o_carrier_id 列的值是否为: ‘NULL’,tidb 不能将 string 插入 int 类型中,应该是数据的问题,或者值和列的对应关系问题

将你的配置 改为这样试试, null = “”

是的,有部分列为NULL

还是报同样的错误

Error: in file /root/csv/tpcc.bmsql_oorder.csv:0 at offset 91669: failed to cast NULL as int(11) for column o_carrier_id (#5): [types:1292]Truncated incorrect FLOAT value: ‘NULL’

image

你好,已经解决,将null =‘NULL’ 可解决此问题

你好,已经解决,我将null =‘NULL’

:call_me_hand:,可否展示下该列在插入成后的值?

你好,这是部分有空值列显示的结果,第五列是o_carrier_id

ok,感谢分享。