测试方式:
用jmeter压测应用程序,应用程序从tidb中固定查找一条记录(所有的查询都是查寻这一条相同的记录),在单条查询的时候耗时10ms左右可以查询到该记录,算是可以接受。但在压测的时候,压测每秒3000次的时候,出现了大量查询失败的情况,失败占比90%以上,这种情况下,可以说不能使用了。
另外,当时数据库中,一共有记录40万条。
查询网络连接,在压测机器上查询
netstat -ant|grep 4000 | awk ‘NR>=3 {++State[$6]} END {for (key in State) print key,State[key]}’
返回:
CLOSE_WAIT 11
ESTABLISHED 136
FIN_WAIT1 41
FIN_WAIT2 1983
TIME_WAIT 20394
按说,我们使用的是连接池,不应该有这么多timewait状态。感觉连接池没有生效,请问:应该怎样配置?
同样的压测环境(只修改数据库配置,其他配置包括代码都不改变),mysql基本上没有失败情况;oceanbase(单机部署)有个别超时情况。但整体上仍然可用,查看连接状态,也基本上只有ESTABLISHED,其他状态基本上没有。
我配置的超时50ms超时(tidb单条记录查询耗时10ms)
【TiDB 使用环境】测试环境
【TiDB 版本】
【部署方式】虚机部署(8c16G,3台机器集群部署)
【操作系统/CPU 架构/芯片详情】ctyunOs 8c16G x86_64 x86_64 x86_64 GNU/Linux
【机器部署详情】pd-3台,tidb- 3台,tikv-3台, tiflash-2台
【集群数据量】3台
【集群节点数】3台
【问题复现路径】做过哪些操作出现的问题
set @@global.tidb_enable_parallel_apply=1;
set global tidb_enable_parallel_apply=1;
set tidb_executor_concurrency=500;
set @@global.tidb_max_tiflash_threads = 200;
set global authentication_ldap_sasl_init_pool_size =2000;
set @@global.max_connections=2000;
设置后仍然会出现如下问题:
【遇到的问题:问题现象及影响】
感觉连接池没有生效,应该怎么样配置?谢谢
【资源配置】进入到 TiDB Dashboard -集群信息 (Cluster Info) -主机(Hosts) 截图此页面
【复制黏贴 ERROR 报错的日志】
tidb_slow_query.log记录了很多超时的请求,如下图:
【其他附件:截图/日志/监控】
连接客户端后,执行查询命令:select @@tidb_config;
配置如下:
{
“advertise-address”: “1.1.1.134”,
“alter-primary-key”: false,
“autoscaler-addr”: “tiflash-autoscale-lb.tiflash-autoscale.svc.cluster.local:8081”,
“autoscaler-cluster-id”: “”,
“autoscaler-type”: “aws”,
“ballast-object-size”: 0,
“compatible-kill-query”: false,
“cors”: “”,
“delay-clean-table-lock”: 0,
“deprecate-integer-display-length”: true,
“disaggregated-tiflash”: false,
“enable-32bits-connection-id”: true,
“enable-enum-length-limit”: true,
“enable-forwarding”: false,
“enable-global-kill”: true,
“enable-table-lock”: false,
“enable-tcp4-only”: false,
“enable-telemetry”: false,
“experimental”: {
“allow-expression-index”: false
},
“graceful-wait-before-shutdown”: 0,
“host”: “0.0.0.0”,
“in-mem-slow-query-recent-num”: 500,
“in-mem-slow-query-topn-num”: 30,
“index-limit”: 64,
“initialize-sql-file”: “”,
“instance”: {
“ddl_slow_threshold”: 300,
“max_connections”: 2000,
“plugin_audit_log_buffer_size”: 0,
“plugin_audit_log_flush_interval”: 30,
“plugin_dir”: “/data/deploy/plugin”,
“plugin_load”: “”,
“tidb_check_mb4_value_in_utf8”: true,
“tidb_enable_collect_execution_info”: true,
“tidb_enable_ddl”: true,
“tidb_enable_slow_log”: true,
“tidb_enable_stats_owner”: true,
“tidb_expensive_query_time_threshold”: 60,
“tidb_expensive_txn_time_threshold”: 600,
“tidb_force_priority”: “NO_PRIORITY”,
“tidb_general_log”: false,
“tidb_pprof_sql_cpu”: false,
“tidb_rc_read_check_ts”: false,
“tidb_record_plan_in_slow_log”: 1,
“tidb_service_scope”: “”,
“tidb_slow_log_threshold”: 300,
“tidb_stmt_summary_enable_persistent”: false,
“tidb_stmt_summary_file_max_backups”: 0,
“tidb_stmt_summary_file_max_days”: 3,
“tidb_stmt_summary_file_max_size”: 64,
“tidb_stmt_summary_filename”: “tidb-statements.log”
},
“is-tiflashcompute-fixed-pool”: false,
“isolation-read”: {
“engines”: [
“tikv”,
“tiflash”,
“tidb”
]
},
“keyspace-name”: “”,
“labels”: {},
“lease”: “45s”,
“log”: {
“disable-error-stack”: null,
“disable-timestamp”: null,
“enable-error-stack”: null,
“enable-timestamp”: null,
“file”: {
“buffer-flush-interval”: 0,
“buffer-size”: 0,
“compression”: “”,
“filename”: “/data/tidb-deploy/tidb-4000/log/tidb.log”,
“is-buffered”: false,
“max-backups”: 0,
“max-days”: 0,
“max-size”: 300
},
“format”: “text”,
“general-log-file”: “”,
“level”: “info”,
“slow-query-file”: “/data/tidb-deploy/tidb-4000/log/tidb_slow_query.log”,
“timeout”: 0
},
“max-ballast-object-size”: 0,
“max-index-length”: 3072,
“new_collations_enabled_on_first_bootstrap”: true,
“opentracing”: {
“enable”: false,
“reporter”: {
“buffer-flush-interval”: 0,
“local-agent-host-port”: “”,
“log-spans”: false,
“queue-size”: 0
},
“rpc-metrics”: false,
“sampler”: {
“max-operations”: 0,
“param”: 1,
“sampling-refresh-interval”: 0,
“sampling-server-url”: “”,
“type”: “const”
}
},
“path”: “1.1.1.134:2379,1.1.1.14:2379,1.1.1.15:2379”,
“pd-client”: {
“pd-server-timeout”: 3
},
“performance”: {
“analyze-partition-concurrency-quota”: 16,
“bind-info-lease”: “3s”,
“concurrently-init-stats”: true,
“cross-join”: true,
“distinct-agg-push-down”: false,
“enable-load-fmsketch”: false,
“enable-stats-cache-mem-quota”: true,
“enforce-mpp”: false,
“force-init-stats”: true,
“gogc”: 100,
“lite-init-stats”: true,
“max-procs”: 0,
“max-txn-ttl”: 3600000,
“plan-replayer-dump-worker-concurrency”: 1,
“plan-replayer-gc-lease”: “10m”,
“projection-push-down”: true,
“pseudo-estimate-ratio”: 0.8,
“server-memory-quota”: 0,
“stats-lease”: “3s”,
“stats-load-concurrency”: 0,
“stats-load-queue-size”: 1000,
“stmt-count-limit”: 5000,
“tcp-keep-alive”: true,
“tcp-no-delay”: true,
“txn-entry-size-limit”: 6291456,
“txn-total-size-limit”: 104857600
},
“pessimistic-txn”: {
“constraint-check-in-place-pessimistic”: true,
“deadlock-history-capacity”: 10,
“deadlock-history-collect-retryable”: false,
“max-retry-count”: 256,
“pessimistic-auto-commit”: false
},
“plugin”: {},
“port”: 4000,
“prepared-plan-cache”: {},
“proxy-protocol”: {
“fallbackable”: false,
“header-timeout”: 5,
“networks”: “”
},
“repair-mode”: false,
“repair-table-list”: ,
“security”: {
“auth-token-jwks”: “”,
“auth-token-refresh-interval”: “1h0m0s”,
“auto-tls”: false,
“cluster-ssl-ca”: “”,
“cluster-ssl-cert”: “”,
“cluster-ssl-key”: “”,
“cluster-verify-cn”: null,
“disconnect-on-expired-password”: true,
“enable-sem”: false,
“rsa-key-size”: 4096,
“secure-bootstrap”: false,
“session-token-signing-cert”: “”,
“session-token-signing-key”: “”,
“skip-grant-table”: false,
“spilled-file-encryption-method”: “plaintext”,
“ssl-ca”: “”,
“ssl-cert”: “”,
“ssl-key”: “”,
“tls-version”: “”
},
“server-version”: “”,
“skip-register-to-dashboard”: false,
“socket”: “/tmp/tidb-4000.sock”,
“split-region-max-num”: 1000,
“split-table”: true,
“status”: {
“grpc-concurrent-streams”: 1024,
“grpc-initial-window-size”: 2097152,
“grpc-keepalive-time”: 10,
“grpc-keepalive-timeout”: 3,
“grpc-max-send-msg-size”: 2147483647,
“metrics-addr”: “”,
“metrics-interval”: 15,
“record-db-label”: false,
“record-db-qps”: false,
“report-status”: true,
“status-host”: “0.0.0.0”,
“status-port”: 10080
},
“store”: “tikv”,
“stores-refresh-interval”: 60,
“table-column-count-limit”: 1017,
“temp-dir”: “/tmp/tidb”,
“tidb-edition”: “”,
“tidb-enable-exit-check”: false,
“tidb-max-reuse-chunk”: 64,
“tidb-max-reuse-column”: 256,
“tidb-release-version”: “”,
“tikv-client”: {
“async-commit”: {
“allowed-clock-drift”: 500000000,
“keys-limit”: 256,
“safe-window”: 2000000000,
“total-key-size-limit”: 4096
},
“batch-policy”: “standard”,
“batch-wait-size”: 8,
“commit-timeout”: “41s”,
“copr-cache”: {
“capacity-mb”: 1000
},
“copr-req-timeout”: 60000000000,
“enable-chunk-rpc”: true,
“enable-replica-selector-v2”: true,
“grpc-compression-type”: “none”,
“grpc-connection-count”: 4,
“grpc-initial-conn-window-size”: 134217728,
“grpc-initial-window-size”: 134217728,
“grpc-keepalive-time”: 10,
“grpc-keepalive-timeout”: 3,
“grpc-shared-buffer-pool”: false,
“max-batch-size”: 128,
“max-batch-wait-time”: 0,
“max-concurrency-request-limit”: 9223372036854776000,
“overload-threshold”: 200,
“region-cache-ttl”: 600,
“resolve-lock-lite-threshold”: 16,
“store-limit”: 0,
“store-liveness-timeout”: “1s”,
“ttl-refreshed-txn-size”: 33554432
},
“tmp-storage-path”: “/tmp/1073_tidb/MC4wLjAuMDo0MDAwLzAuMC4wLjA6MTAwODA=/tmp-storage”,
“tmp-storage-quota”: -1,
“token-limit”: 1000,
“top-sql”: {
“receiver-address”: “”
},
“transaction-summary”: {
“transaction-id-digest-min-duration”: 2147483647,
“transaction-summary-capacity”: 500
},
“treat-old-version-utf8-as-utf8mb4”: true,
“use-autoscaler”: false,
“version-comment”: “”
}

