直接操作 tikv 集群,rawkv api 可以正常使用, transactional api 报 PD server timeout 的错

TiKV 版本

TiKV
Release Version:   3.0.7
Git Commit Hash:   ac6f02648a8c6ccb7ccafca20287e1b27007e4a0
Git Commit Branch: HEAD
UTC Build Time:    2019-12-04 10:06:20
Rust Version:      rustc 1.37.0-nightly (0e4a56b4b 2019-06-13)

pd 版本

Release Version: v3.0.7
Git Commit Hash: 7a5909ed3bae74d0c6c728ca931f240233aca03a
Git Branch: HEAD
UTC Build Time:  2019-12-04 10:06:16

问题描述

直接部署的 tikv 集群,没有部署 tidb,使用 golang 的 sdk 操作 tikv 集群,使用 raw kv api 没有任何问题,同时使用 transactional api 报错:

日志如下:

[2020/01/05 04:56:19.162 +08:00] [INFO] [client.go:144] ["[pd] create pd client with endpoints"] [pd-address="[10.0.10.229:2379]"]
[2020/01/05 04:56:19.640 +08:00] [INFO] [client.go:252] ["[pd] switch leader"] [new-leader=http://10.0.10.229:2379] [old-leader=]
[2020/01/05 04:56:19.641 +08:00] [INFO] [client.go:163] ["[pd] init cluster id"] [cluster-id=6777241389820427846]
[2020/01/05 04:56:43.947 +08:00] [WARN] [backoff.go:304] ["pdRPC backoffer.maxSleep 20000ms is exceeded, errors:\nloadRegion from PD failed, key: \"autoIncStepId\", err: invalid marker byte, group bytes \"p\\x00\\b\\xd2S`\\vD\\xa9\" at 2020-01-05T04:56:38.175501+08:00\nloadRegion from PD failed, key: \"autoIncStepId\", err: invalid marker byte, group bytes \"p\\x00\\b\\xd2S`\\vD\\xa9\" at 2020-01-05T04:56:41.258237+08:00\nloadRegion from PD failed, key: \"autoIncStepId\", err: invalid marker byte, group bytes \"p\\x00\\b\\xd2S`\\vD\\xa9\" at 2020-01-05T04:56:43.947372+08:00"]
[2020/01/05 04:56:43.947 +08:00] [ERROR] [main.go:53] ["get auto inc step id failed"] [error="[tikv:9001]PD server timeout"] [errorVerbose="[tikv:9001]PD server timeout\ngithub.com/pingcap/errors.AddStack\n\t/Users/ys/gopath/pkg/mod/github.com/pingcap/errors@v0.11.4/errors.go:174\ngithub.com/pingcap/errors.Trace\n\t/Users/ys/gopath/pkg/mod/github.com/pingcap/errors@v0.11.4/juju_adaptor.go:15\ngithub.com/pingcap/tidb/store/tikv.(*RegionCache).loadRegion\n\t/Users/ys/gopath/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20191203054555-84e4386c7a77/store/tikv/region_cache.go:612\ngithub.com/pingcap/tidb/store/tikv.(*RegionCache).findRegionByKey\n\t/Users/ys/gopath/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20191203054555-84e4386c7a77/store/tikv/region_cache.go:366\ngithub.com/pingcap/tidb/store/tikv.(*RegionCache).LocateKey\n\t/Users/ys/gopath/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20191203054555-84e4386c7a77/store/tikv/region_cache.go:326\ngithub.com/pingcap/tidb/store/tikv.(*tikvSnapshot).get\n\t/Users/ys/gopath/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20191203054555-84e4386c7a77/store/tikv/snapshot.go:301\ngithub.com/pingcap/tidb/store/tikv.(*tikvSnapshot).Get\n\t/Users/ys/gopath/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20191203054555-84e4386c7a77/store/tikv/snapshot.go:265\ngithub.com/pingcap/tidb/kv.(*unionStore).Get\n\t/Users/ys/gopath/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20191203054555-84e4386c7a77/kv/union_store.go:194\ngithub.com/pingcap/tidb/store/tikv.(*tikvTxn).Get\n\t/Users/ys/gopath/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20191203054555-84e4386c7a77/store/tikv/txn.go:135\nmain.test\n\t/Users/ys/gopath/src/bitbucket.org/ldnvnbl/tikv-client/main/main.go:51\nmain.main\n\t/Users/ys/gopath/src/bitbucket.org/ldnvnbl/tikv-client/main/main.go:81\nruntime.main\n\t/Users/ys/go/src/runtime/proc.go:203\nruntime.goexit\n\t/Users/ys/go/src/runtime/asm_amd64.s:1357"] [stack="github.com/pingcap/log.Error\n\t/Users/ys/gopath/pkg/mod/github.com/pingcap/log@v0.0.0-20190715063458-479153f07ebd/global.go:42\nmain.test\n\t/Users/ys/gopath/src/bitbucket.org/ldnvnbl/tikv-client/main/main.go:53\nmain.main\n\t/Users/ys/gopath/src/bitbucket.org/ldnvnbl/tikv-client/main/main.go:81\nruntime.main\n\t/Users/ys/go/src/runtime/proc.go:203"]

我确定 pd server 是可以连上的,网络没有问题

是使用 https://github.com/tikv/client-go 调用 txnkv 吗,具体是如何操作的呢?

我是根据这篇文档:https://tikv.org/docs/3.0/reference/clients/go/ 我的操作方法也很简单,这是我的测试代码:

var autoIncStepId = []byte("autoIncStepId")

func test() {

	pdAddrs := []string{"10.0.10.229:2379"}

	store, err := txnkv.NewClient(context.Background(), pdAddrs, cfg2.Config{})
	//
	//path := "tikv://10.0.10.229:2379"
	//store, err := tikv.Driver{}.Open(path)
	if err != nil {
		//log.Error("open tikv storage failed", zap.Error(err))
		fmt.Println("new client err:", err)
		return
	}

	ctx := context.Background()

	tx, err := store.Begin(context.Background())
	if err != nil {
		//log.Error("store begin failed", zap.Error(err))
		fmt.Println("begin err:", err)
		return
	}

	defer func() {
		err := tx.Commit(context.Background())
		if err != nil {
			//log.Error("commit failed", zap.Error(err))
			fmt.Println("commit err:", err)
			return
		}
	}()

	dat, err := tx.Get(ctx, autoIncStepId)
	if err != nil {
		//log.Error("get auto inc step id failed", zap.Error(err))
		fmt.Println("get err:", err)
		return
	}

	var n int
	if dat == nil {
		n = 1
	} else {
		n, err = strconv.Atoi(string(dat))
		if err != nil {
			//log.Error("parse step id to int failed", zap.Error(err), zap.String("stepIdData", string(dat)))
			fmt.Println("parse err", err)
			return
		}
		n++
	}

	//log.Info("set auto inc step id", zap.Int("stepId", n))
	fmt.Println("n:", n)

	err = tx.Set(autoIncStepId, []byte(strconv.Itoa(n)))
	if err != nil {
		//log.Error("set auto inc step id failed", zap.Error(err))
		fmt.Println("set err:", err)
		return
	}

}

我试了你发的这个 SDK,又报其他错误:

panic: non-positive interval for NewTicker

goroutine 29 [running]:
time.NewTicker(0x0, 0x0)
        /Users/ys/go/src/time/tick.go:23 +0x147
github.com/tikv/client-go/txnkv/oracle/oracles.(*pdOracle).updateTS(0xc0001589c0, 0x1a51ba0, 0xc0000ba008, 0x0)
        /Users/ys/gopath/pkg/mod/github.com/tikv/client-go@v0.0.0-20191226061530-ddd42ffad4d2/txnkv/oracle/oracles/pd.go:121 +0x40
created by github.com/tikv/client-go/txnkv/oracle/oracles.NewPdOracle
        /Users/ys/gopath/pkg/mod/github.com/tikv/client-go@v0.0.0-20191226061530-ddd42ffad4d2/txnkv/oracle/oracles/pd.go:50 +0xe2

Process finished with exit code 2

建议参考 clinet-go 下面 example 中 txnkv 的例子再测试下代码 https://github.com/tikv/client-go/blob/master/examples/txnkv/txnkv.go

你好,我看文档上说:

It is not recommended or supported to use both the raw and transactional APIs on the same keyspace.

我前面的问题应该是在同一集群上使用了两种 API 导致。但我现在的需求就是要同时用两种 API,以满足不同的需求。 上面说的是 keyspace, 那么是不是在同一个 tikv cluster 上可以创建多个 keyspace 来满足我的需求呢?

  1. 首先单个 TiKV 集群使用 2 个 API 是不建议的这样使用,会出现很多问题,而且目前不支持;
  2. 另外增加多个 keyspace 方式也不是强烈不推荐的。

有兴趣可以看一下 TiKV 的官网