tidb4.0日志快速占满磁盘

通过tiup安装的4.0tidb。使用虚拟机4节点(1tidb+1PD,3kv) 用测试程序并发(只有几十)写入测试数据,{data_dir}/tidb-deploy/tidb-4000/log目录中文件巨多。。 平均每分钟占用2G左右。磁盘快速被占满。 问下大神们,个人疑问: 1.此报警需要怎样优化? 2.日志文件是否可以分配最大配额?
测试表的ddl:
CREATE TABLE t_oil_info_09 ( id int(11) NOT NULL AUTO_INCREMENT, msg_seq int(11) DEFAULT NULL, termphonenumber varchar(50) DEFAULT NULL, location_time timestamp(6) NULL DEFAULT NULL, level decimal(8,4) DEFAULT NULL, vol decimal(8,4) DEFAULT NULL, UNIQUE KEY id (id), KEY idx_oil_info_09_time (location_time) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin

日志部分截取如下: image

[2020/04/24 11:00:58.739 +08:00] [WARN] [session.go:1044] [“run statement failed”] [conn=449197] [schemaVersion=110] [error="[types:1264]Out of range value for column ‘vol’ at row 1"] [session="{\n “currDBName”: “test”,\n “id”: 449197,\n “status”: 2,\n “strictMode”: true,\n “user”: {\n “Username”: “tidb”,\n “Hostname”: “192.168.3.251”,\n “CurrentUser”: false,\n “AuthUsername”: “tidb”,\n “AuthHostname”: “%”\n }\n}"] [2020/04/24 11:00:58.739 +08:00] [WARN] [session.go:1044] [“run statement failed”] [conn=449204] [schemaVersion=110] [error="[types:1264]Out of range value for column ‘vol’ at row 1"] [session="{\n “currDBName”: “test”,\n “id”: 449204,\n “status”: 2,\n “strictMode”: true,\n “user”: {\n “Username”: “tidb”,\n “Hostname”: “192.168.3.251”,\n “CurrentUser”: false,\n “AuthUsername”: “tidb”,\n “AuthHostname”: “%”\n }\n}"] [2020/04/24 11:00:58.739 +08:00] [WARN] [conn.go:726] [“command dispatched failed”] [conn=449207] [connInfo=“id:449207, addr:192.168.3.251:59884 status:10, collation:utf8_general_ci, user:tidb”] [command=Query] [status=“inTxn:0, autocommit:1”] [sql=“insert into t_oil_info_09(msg_seq,termPhoneNumber,location_time,level,vol) values(448696432,‘133577’,‘2020-04-24 11:00:55.484’,577.0,44869.645)”] [txn_mode=PESSIMISTIC] [err="[types:1264]Out of range value for column ‘vol’ at row 1\ngithub.com/pingcap/errors.AddStack\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20190809092503-95897b64e011/errors.go:174\ngithub.com/pingcap/parser/terror.(*Error).GenWithStackByArgs\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/pkg/mod/github.com/pingcap/parser@v0.0.0-20200326020624-68d423641be5/terror/terror.go:243\ngithub.com/pingcap/tidb/executor.(*InsertValues).handleErr\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/executor/insert_common.go:283\ngithub.com/pingcap/tidb/executor.(*InsertValues).fastEvalRow\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/executor/insert_common.go:353\ngithub.com/pingcap/tidb/executor.insertRows\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/executor/insert_common.go:223\ngithub.com/pingcap/tidb/executor.(*InsertExec).Next\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/executor/insert.go:262\ngithub.com/pingcap/tidb/executor.Next\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/executor/executor.go:217\ngithub.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelayExecutor\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/executor/adapter.go:511\ngithub.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelay\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/executor/adapter.go:395\ngithub.com/pingcap/tidb/executor.(*ExecStmt).Exec\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/executor/adapter.go:363\ngithub.com/pingcap/tidb/session.runStmt\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/session/tidb.go:281\ngithub.com/pingcap/tidb/session.(*session).executeStatement\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/session/session.go:1041\ngithub.com/pingcap/tidb/session.(*session).execute\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/session/session.go:1153\ngithub.com/pingcap/tidb/session.(*session).Execute\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/session/session.go:1089\ngithub.com/pingcap/tidb/server.(*TiDBContext).Execute\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/server/driver_tidb.go:248\ngithub.com/pingcap/tidb/server.(*clientConn).handleQuery\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/server/conn.go:1256\ngithub.com/pingcap/tidb/server.(*clientConn).dispatch\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/server/conn.go:897\ngithub.com/pingcap/tidb/server.(*clientConn).Run\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/server/conn.go:711\ngithub.com/pingcap/tidb/server.(*Server).onConn\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/server/server.go:415\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357"] [2020/04/24 11:00:58.739 +08:00] [WARN] [conn.go:726] [“command dispatched failed”] [conn=449208] [connInfo=“id:449208, addr:192.168.3.251:59885 status:10, collation:utf8_general_ci, user:tidb”] [command=Query] [status=“inTxn:0, autocommit:1”] [sql=“insert into t_oil_info_09(msg_seq,termPhoneNumber,location_time,level,vol) values(448697396,‘133542’,‘2020-04-24 11:00:55.537’,542.0,44869.74)”] [txn_mode=PESSIMISTIC] [err="[types:1264]Out of range value for column ‘vol’ at row 1\ngithub.com/pingcap/errors.AddStack\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20190809092503-95897b64e011/errors.go:174\ngithub.com/pingcap/parser/terror.(*Error).GenWithStackByArgs\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/pkg/mod/github.com/pingcap/parser@v0.0.0-20200326020624-68d423641be5/terror/terror.go:243\ngithub.com/pingcap/tidb/executor.(*InsertValues).handleErr\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/executor/insert_common.go:283\ngithub.com/pingcap/tidb/executor.(*InsertValues).fastEvalRow\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/executor/insert_common.go:353\ngithub.com/pingcap/tidb/executor.insertRows\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/executor/insert_common.go:223\ngithub.com/pingcap/tidb/executor.(*InsertExec).Next\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/executor/insert.go:262\ngithub.com/pingcap/tidb/executor.Next\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/executor/executor.go:217\ngithub.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelayExecutor\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/executor/adapter.go:511\ngithub.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelay\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/executor/adapter.go:395\ngithub.com/pingcap/tidb/executor.(*ExecStmt).Exec\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/executor/adapter.go:363\ngithub.com/pingcap/tidb/session.runStmt\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/session/tidb.go:281\ngithub.com/pingcap/tidb/session.(*session).executeStatement\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/session/session.go:1041\ngithub.com/pingcap/tidb/session.(*session).execute\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/session/session.go:1153\ngithub.com/pingcap/tidb/session.(*session).Execute\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/session/session.go:1089\ngithub.com/pingcap/tidb/server.(*TiDBContext).Execute\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/server/driver_tidb.go:248\ngithub.com/pingcap/tidb/server.(*clientConn).handleQuery\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/server/conn.go:1256\ngithub.com/pingcap/tidb/server.(*clientConn).dispatch\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/server/conn.go:897\ngithub.com/pingcap/tidb/server.(*clientConn).Run\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/server/conn.go:711\ngithub.com/pingcap/tidb/server.(*Server).onConn\n\t/home/jenkins/agent/workspace/tidb_v4.0.0-rc/go/src/github.com/pingcap/tidb/server/server.go:415\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357"]

收到问题,我们尽快查看下,多谢

您好: 我测试了下,mysql也会报错,您的数据和列的属性不匹配,导致有大量报错

你好,

关于 tidb-server 日志相关配置,滚动重启 tidb 节点方才生效,修改单个 tidb-server 节点配置文件,只对当前节点生效

max-size

  • 日志文件的大小限制。
  • 默认值:300MB
  • 最大设置上限为 4GB。

max-days

  • 日志最大保留的天数。
  • 默认值:0
  • 默认不清理;如果设置了参数值,在 max-days 之后 TiDB 会清理过期的日志文件。

max-backups

  • 保留的日志的最大数量。
  • 默认值:0
  • 默认全部保存;如果设置为 7,会最多保留 7 个老的日志文件。

log-rotate

  • 是否每日创建一个新的日志文件。
  • 默认值:true
  • 如果设置为 true,每天会新建新的日志文件,如果设置为 false,那么只会输出到一个日志文件。X.png)

多谢,不好意思,报错是程序的问题,模拟值超出边界。主要是log文件没有配置配额。

好的,多谢。

:ok_hand: