直接操作 tikv 集群，rawkv api 可以正常使用， transactional api 报 PD server timeout 的错

🪐 TiDB 技术问题 部署&运维管理
1

TiKV 版本

TiKV
Release Version:   3.0.7
Git Commit Hash:   ac6f02648a8c6ccb7ccafca20287e1b27007e4a0
Git Commit Branch: HEAD
UTC Build Time:    2019-12-04 10:06:20
Rust Version:      rustc 1.37.0-nightly (0e4a56b4b 2019-06-13)

pd 版本

Release Version: v3.0.7
Git Commit Hash: 7a5909ed3bae74d0c6c728ca931f240233aca03a
Git Branch: HEAD
UTC Build Time:  2019-12-04 10:06:16

问题描述

直接部署的 tikv 集群，没有部署 tidb，使用 golang 的 sdk 操作 tikv 集群，使用 raw kv api 没有任何问题，同时使用 transactional api 报错：

日志如下：

[2020/01/05 04:56:19.162 +08:00] [INFO] [client.go:144] ["[pd] create pd client with endpoints"] [pd-address="[10.0.10.229:2379]"]
[2020/01/05 04:56:19.640 +08:00] [INFO] [client.go:252] ["[pd] switch leader"] [new-leader=http://10.0.10.229:2379] [old-leader=]
[2020/01/05 04:56:19.641 +08:00] [INFO] [client.go:163] ["[pd] init cluster id"] [cluster-id=6777241389820427846]
[2020/01/05 04:56:43.947 +08:00] [WARN] [backoff.go:304] ["pdRPC backoffer.maxSleep 20000ms is exceeded, errors:\
loadRegion from PD failed, key: \"autoIncStepId\", err: invalid marker byte, group bytes \"p\\x00\\b\\xd2S`\\vD\\xa9\" at 2020-01-05T04:56:38.175501+08:00\
loadRegion from PD failed, key: \"autoIncStepId\", err: invalid marker byte, group bytes \"p\\x00\\b\\xd2S`\\vD\\xa9\" at 2020-01-05T04:56:41.258237+08:00\
loadRegion from PD failed, key: \"autoIncStepId\", err: invalid marker byte, group bytes \"p\\x00\\b\\xd2S`\\vD\\xa9\" at 2020-01-05T04:56:43.947372+08:00"]
[2020/01/05 04:56:43.947 +08:00] [ERROR] [main.go:53] ["get auto inc step id failed"] [error="[tikv:9001]PD server timeout"] [errorVerbose="[tikv:9001]PD server timeout\
github.com/pingcap/errors.AddStack\
\t/Users/ys/gopath/pkg/mod/github.com/pingcap/errors@v0.11.4/errors.go:174\
github.com/pingcap/errors.Trace\
\t/Users/ys/gopath/pkg/mod/github.com/pingcap/errors@v0.11.4/juju_adaptor.go:15\
github.com/pingcap/tidb/store/tikv.(*RegionCache).loadRegion\
\t/Users/ys/gopath/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20191203054555-84e4386c7a77/store/tikv/region_cache.go:612\
github.com/pingcap/tidb/store/tikv.(*RegionCache).findRegionByKey\
\t/Users/ys/gopath/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20191203054555-84e4386c7a77/store/tikv/region_cache.go:366\
github.com/pingcap/tidb/store/tikv.(*RegionCache).LocateKey\
\t/Users/ys/gopath/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20191203054555-84e4386c7a77/store/tikv/region_cache.go:326\
github.com/pingcap/tidb/store/tikv.(*tikvSnapshot).get\
\t/Users/ys/gopath/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20191203054555-84e4386c7a77/store/tikv/snapshot.go:301\
github.com/pingcap/tidb/store/tikv.(*tikvSnapshot).Get\
\t/Users/ys/gopath/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20191203054555-84e4386c7a77/store/tikv/snapshot.go:265\
github.com/pingcap/tidb/kv.(*unionStore).Get\
\t/Users/ys/gopath/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20191203054555-84e4386c7a77/kv/union_store.go:194\
github.com/pingcap/tidb/store/tikv.(*tikvTxn).Get\
\t/Users/ys/gopath/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20191203054555-84e4386c7a77/store/tikv/txn.go:135\
main.test\
\t/Users/ys/gopath/src/bitbucket.org/ldnvnbl/tikv-client/main/main.go:51\
main.main\
\t/Users/ys/gopath/src/bitbucket.org/ldnvnbl/tikv-client/main/main.go:81\
runtime.main\
\t/Users/ys/go/src/runtime/proc.go:203\
runtime.goexit\
\t/Users/ys/go/src/runtime/asm_amd64.s:1357"] [stack="github.com/pingcap/log.Error\
\t/Users/ys/gopath/pkg/mod/github.com/pingcap/log@v0.0.0-20190715063458-479153f07ebd/global.go:42\
main.test\
\t/Users/ys/gopath/src/bitbucket.org/ldnvnbl/tikv-client/main/main.go:53\
main.main\
\t/Users/ys/gopath/src/bitbucket.org/ldnvnbl/tikv-client/main/main.go:81\
runtime.main\
\t/Users/ys/go/src/runtime/proc.go:203"]

我确定 pd server 是可以连上的，网络没有问题

2

是使用 https://github.com/tikv/client-go 调用 txnkv 吗，具体是如何操作的呢？

3

我是根据这篇文档：https://tikv.org/docs/3.0/reference/clients/go/ 我的操作方法也很简单，这是我的测试代码：

var autoIncStepId = []byte("autoIncStepId")

func test() {

	pdAddrs := []string{"10.0.10.229:2379"}

	store, err := txnkv.NewClient(context.Background(), pdAddrs, cfg2.Config{})
	//
	//path := "tikv://10.0.10.229:2379"
	//store, err := tikv.Driver{}.Open(path)
	if err != nil {
		//log.Error("open tikv storage failed", zap.Error(err))
		fmt.Println("new client err:", err)
		return
	}

	ctx := context.Background()

	tx, err := store.Begin(context.Background())
	if err != nil {
		//log.Error("store begin failed", zap.Error(err))
		fmt.Println("begin err:", err)
		return
	}

	defer func() {
		err := tx.Commit(context.Background())
		if err != nil {
			//log.Error("commit failed", zap.Error(err))
			fmt.Println("commit err:", err)
			return
		}
	}()

	dat, err := tx.Get(ctx, autoIncStepId)
	if err != nil {
		//log.Error("get auto inc step id failed", zap.Error(err))
		fmt.Println("get err:", err)
		return
	}

	var n int
	if dat == nil {
		n = 1
	} else {
		n, err = strconv.Atoi(string(dat))
		if err != nil {
			//log.Error("parse step id to int failed", zap.Error(err), zap.String("stepIdData", string(dat)))
			fmt.Println("parse err", err)
			return
		}
		n++
	}

	//log.Info("set auto inc step id", zap.Int("stepId", n))
	fmt.Println("n:", n)

	err = tx.Set(autoIncStepId, []byte(strconv.Itoa(n)))
	if err != nil {
		//log.Error("set auto inc step id failed", zap.Error(err))
		fmt.Println("set err:", err)
		return
	}

}

我试了你发的这个 SDK，又报其他错误：

panic: non-positive interval for NewTicker

goroutine 29 [running]:
time.NewTicker(0x0, 0x0)
        /Users/ys/go/src/time/tick.go:23 +0x147
github.com/tikv/client-go/txnkv/oracle/oracles.(*pdOracle).updateTS(0xc0001589c0, 0x1a51ba0, 0xc0000ba008, 0x0)
        /Users/ys/gopath/pkg/mod/github.com/tikv/client-go@v0.0.0-20191226061530-ddd42ffad4d2/txnkv/oracle/oracles/pd.go:121 +0x40
created by github.com/tikv/client-go/txnkv/oracle/oracles.NewPdOracle
        /Users/ys/gopath/pkg/mod/github.com/tikv/client-go@v0.0.0-20191226061530-ddd42ffad4d2/txnkv/oracle/oracles/pd.go:50 +0xe2

Process finished with exit code 2
4

建议参考 clinet-go 下面 example 中 txnkv 的例子再测试下代码 https://github.com/tikv/client-go/blob/master/examples/txnkv/txnkv.go

5

你好，我看文档上说:

It is not recommended or supported to use both the raw and transactional APIs on the same keyspace.

我前面的问题应该是在同一集群上使用了两种 API 导致。但我现在的需求就是要同时用两种 API，以满足不同的需求。 上面说的是 keyspace, 那么是不是在同一个 tikv cluster 上可以创建多个 keyspace 来满足我的需求呢？

6
  1. 首先单个 TiKV 集群使用 2 个 API 是不建议的这样使用，会出现很多问题，而且目前不支持；
  2. 另外增加多个 keyspace 方式也不是强烈不推荐的。

有兴趣可以看一下 TiKV 的官网