Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
rpc: replace with gRPC
This completely removes the old "cockroach RPC", and replaces it with a
very thin wrapper around gRPC. Notable omissions:
- Heartbeats are no longer used for health checking, and no replacement
is provided. gRPC has a similar mechanism, but it is not currently
used.
- gRPC client connections are never closed. This was not exactly
implemented in the old code, but we probably should implement it so
that clusters do not attempt to connect to downed nodes indefinitely.

SQL benchmarks have regressed. With local calls disabled:
```
name                          old time/op    new time/op    delta
Select1_Cockroach-4             61.8µs ± 3%    59.5µs ± 1%    -3.65%          (p=0.008 n=5+5)
Select2_Cockroach-4             1.01ms ±14%    1.31ms ± 7%   +30.13%          (p=0.008 n=5+5)
Insert1_Cockroach-4              480µs ± 1%     815µs ± 0%   +69.73%          (p=0.008 n=5+5)
Insert10_Cockroach-4             815µs ± 5%    1160µs ± 1%   +42.42%          (p=0.008 n=5+5)
Insert100_Cockroach-4           3.81ms ± 1%    4.33ms ± 3%   +13.55%          (p=0.008 n=5+5)
Insert1000_Cockroach-4          34.3ms ± 1%    36.3ms ± 3%    +6.07%          (p=0.008 n=5+5)
Update1_Cockroach-4              807µs ± 5%    1435µs ± 3%   +77.87%          (p=0.008 n=5+5)
Update10_Cockroach-4            1.47ms ± 6%    2.09ms ± 1%   +41.85%          (p=0.008 n=5+5)
Update100_Cockroach-4           6.93ms ± 1%    7.87ms ± 1%   +13.52%          (p=0.008 n=5+5)
Update1000_Cockroach-4          56.4ms ± 3%    59.2ms ± 3%    +4.98%          (p=0.016 n=5+5)
Delete1_Cockroach-4              725µs ± 4%    1096µs ± 2%   +51.10%          (p=0.008 n=5+5)
Delete10_Cockroach-4            1.94ms ± 1%    2.36ms ± 3%   +21.48%          (p=0.008 n=5+5)
Delete100_Cockroach-4           14.5ms ± 1%    15.4ms ± 1%    +5.78%          (p=0.008 n=5+5)
Delete1000_Cockroach-4           144ms ± 5%     139ms ± 2%      ~             (p=0.310 n=5+5)
Scan1_Cockroach-4                291µs ± 1%     610µs ± 2%  +109.77%          (p=0.008 n=5+5)
Scan10_Cockroach-4               334µs ± 1%     662µs ± 4%   +98.20%          (p=0.008 n=5+5)
Scan100_Cockroach-4              665µs ± 3%     988µs ± 1%   +48.62%          (p=0.008 n=5+5)
Scan1000_Cockroach-4            3.68ms ± 1%    4.29ms ± 1%   +16.39%          (p=0.008 n=5+5)
Scan10000_Cockroach-4           36.9ms ± 1%    40.9ms ± 3%   +10.62%          (p=0.008 n=5+5)
Scan1000Limit1_Cockroach-4       304µs ± 2%     621µs ± 1%  +104.08%          (p=0.008 n=5+5)
Scan1000Limit10_Cockroach-4      347µs ± 1%     670µs ± 1%   +93.05%          (p=0.008 n=5+5)
Scan1000Limit100_Cockroach-4     691µs ± 3%    1016µs ± 0%   +47.06%          (p=0.016 n=5+4)

name                          old alloc/op   new alloc/op   delta
Select1_Cockroach-4             3.91kB ± 0%    4.00kB ± 1%    +2.32%          (p=0.008 n=5+5)
Select2_Cockroach-4              101kB ± 0%     197kB ± 0%   +95.89%          (p=0.008 n=5+5)
Insert1_Cockroach-4             27.8kB ± 0%   109.5kB ± 0%  +293.66%          (p=0.008 n=5+5)
Insert10_Cockroach-4            83.2kB ± 0%   168.1kB ± 1%  +101.89%          (p=0.008 n=5+5)
Insert100_Cockroach-4            639kB ± 0%     753kB ± 0%   +17.92%          (p=0.008 n=5+5)
Insert1000_Cockroach-4          5.72MB ± 1%    6.06MB ± 0%    +5.99%          (p=0.008 n=5+5)
Update1_Cockroach-4             43.6kB ± 0%   203.7kB ± 0%  +367.60%          (p=0.008 n=5+5)
Update10_Cockroach-4             132kB ± 0%     298kB ± 0%  +126.62%          (p=0.008 n=5+5)
Update100_Cockroach-4            980kB ± 0%    1203kB ± 0%   +22.78%          (p=0.008 n=5+5)
Update1000_Cockroach-4          8.45MB ± 0%    9.16MB ± 1%    +8.40%          (p=0.008 n=5+5)
Delete1_Cockroach-4             34.8kB ± 0%   116.0kB ± 0%  +233.17%          (p=0.008 n=5+5)
Delete10_Cockroach-4            93.6kB ± 0%   179.5kB ± 0%   +91.74%          (p=0.008 n=5+5)
Delete100_Cockroach-4            689kB ± 0%     809kB ± 0%   +17.46%          (p=0.008 n=5+5)
Delete1000_Cockroach-4          6.15MB ± 0%    6.55MB ± 0%    +6.50%          (p=0.008 n=5+5)
Scan1_Cockroach-4               15.2kB ± 0%    95.0kB ± 0%  +526.70%          (p=0.008 n=5+5)
Scan10_Cockroach-4              21.1kB ± 0%   102.1kB ± 0%  +384.48%          (p=0.008 n=5+5)
Scan100_Cockroach-4             69.1kB ± 0%   161.9kB ± 0%  +134.27%          (p=0.016 n=4+5)
Scan1000_Cockroach-4             522kB ± 0%     721kB ± 0%   +38.13%          (p=0.008 n=5+5)
Scan10000_Cockroach-4           9.48MB ± 0%   10.66MB ± 0%   +12.45%          (p=0.016 n=4+5)
Scan1000Limit1_Cockroach-4      16.1kB ± 0%    96.0kB ± 0%  +496.44%          (p=0.008 n=5+5)
Scan1000Limit10_Cockroach-4     21.8kB ± 0%   103.1kB ± 0%  +372.11%          (p=0.008 n=5+5)
Scan1000Limit100_Cockroach-4    69.8kB ± 0%   162.6kB ± 0%  +132.85%          (p=0.008 n=5+5)

name                          old allocs/op  new allocs/op  delta
Select1_Cockroach-4               97.0 ± 0%      97.0 ± 0%      ~     (all samples are equal)
Select2_Cockroach-4              2.95k ± 0%     3.13k ± 0%    +6.14%          (p=0.016 n=5+4)
Insert1_Cockroach-4                402 ± 0%       584 ± 0%   +45.27%          (p=0.016 n=5+4)
Insert10_Cockroach-4             1.01k ± 0%     1.19k ± 0%   +18.50%          (p=0.008 n=5+5)
Insert100_Cockroach-4            6.73k ± 0%     6.97k ± 0%    +3.46%          (p=0.008 n=5+5)
Insert1000_Cockroach-4           64.3k ± 0%     64.8k ± 0%    +0.76%          (p=0.008 n=5+5)
Update1_Cockroach-4                795 ± 0%      1151 ± 0%   +44.73%          (p=0.016 n=4+5)
Update10_Cockroach-4             1.64k ± 0%     2.00k ± 0%   +21.75%          (p=0.016 n=5+4)
Update100_Cockroach-4            9.54k ± 0%     9.93k ± 0%    +4.16%          (p=0.008 n=5+5)
Update1000_Cockroach-4           84.5k ± 0%     85.3k ± 1%    +0.85%          (p=0.008 n=5+5)
Delete1_Cockroach-4                620 ± 0%       801 ± 0%   +29.09%          (p=0.008 n=5+5)
Delete10_Cockroach-4             1.41k ± 0%     1.60k ± 0%   +13.08%          (p=0.016 n=4+5)
Delete100_Cockroach-4            9.04k ± 0%     9.25k ± 0%    +2.34%          (p=0.008 n=5+5)
Delete1000_Cockroach-4           84.7k ± 0%     85.2k ± 0%    +0.54%          (p=0.008 n=5+5)
Scan1_Cockroach-4                  304 ± 0%       482 ± 0%   +58.55%          (p=0.016 n=4+5)
Scan10_Cockroach-4                 405 ± 0%       584 ± 0%   +44.06%          (p=0.008 n=5+5)
Scan100_Cockroach-4              1.31k ± 0%     1.50k ± 0%   +13.69%          (p=0.016 n=4+5)
Scan1000_Cockroach-4             10.3k ± 0%     10.5k ± 0%    +2.01%          (p=0.016 n=5+4)
Scan10000_Cockroach-4             101k ± 0%      101k ± 0%    +0.60%          (p=0.008 n=5+5)
Scan1000Limit1_Cockroach-4         330 ± 0%       508 ± 0%   +54.00%          (p=0.008 n=5+5)
Scan1000Limit10_Cockroach-4        430 ± 0%       608 ± 0%   +41.53%          (p=0.008 n=5+5)
Scan1000Limit100_Cockroach-4     1.34k ± 0%     1.52k ± 0%   +13.44%          (p=0.029 n=4+4)
```

SQL benchmarks have regressed. With local calls enabled:
```
name                          old time/op    new time/op    delta
Select1_Cockroach-4             60.1µs ± 0%    58.7µs ± 1%   -2.36%          (p=0.016 n=5+4)
Select2_Cockroach-4              846µs ±18%     818µs ±15%     ~             (p=0.310 n=5+5)
Insert1_Cockroach-4              383µs ± 2%     363µs ± 4%   -5.25%          (p=0.016 n=5+5)
Insert10_Cockroach-4             689µs ± 9%     658µs ± 1%   -4.51%          (p=0.008 n=5+5)
Insert100_Cockroach-4           3.49ms ± 2%    3.45ms ± 2%     ~             (p=0.310 n=5+5)
Insert1000_Cockroach-4          33.2ms ± 3%    33.1ms ± 1%     ~             (p=0.841 n=5+5)
Update1_Cockroach-4              594µs ± 3%     524µs ± 2%  -11.83%          (p=0.008 n=5+5)
Update10_Cockroach-4            1.15ms ± 4%    1.13ms ± 2%     ~             (p=0.310 n=5+5)
Update100_Cockroach-4           6.17ms ± 2%    6.22ms ± 1%     ~             (p=0.421 n=5+5)
Update1000_Cockroach-4          52.7ms ± 3%    53.4ms ± 2%     ~             (p=0.151 n=5+5)
Delete1_Cockroach-4              625µs ± 1%     590µs ± 5%   -5.64%          (p=0.008 n=5+5)
Delete10_Cockroach-4            1.83ms ± 4%    1.74ms ± 1%   -5.08%          (p=0.008 n=5+5)
Delete100_Cockroach-4           14.8ms ± 2%    14.4ms ± 1%   -2.96%          (p=0.016 n=5+5)
Delete1000_Cockroach-4           140ms ± 5%     142ms ± 4%     ~             (p=0.548 n=5+5)
Scan1_Cockroach-4                207µs ± 2%     166µs ± 9%  -19.96%          (p=0.008 n=5+5)
Scan10_Cockroach-4               242µs ± 4%     209µs ± 5%  -13.74%          (p=0.008 n=5+5)
Scan100_Cockroach-4              500µs ± 2%     508µs ± 4%     ~             (p=0.690 n=5+5)
Scan1000_Cockroach-4            2.98ms ± 1%    3.02ms ± 1%   +1.50%          (p=0.008 n=5+5)
Scan10000_Cockroach-4           30.5ms ± 1%    31.4ms ± 1%   +2.91%          (p=0.008 n=5+5)
Scan1000Limit1_Cockroach-4       218µs ± 1%     177µs ± 8%  -18.47%          (p=0.008 n=5+5)
Scan1000Limit10_Cockroach-4      253µs ± 4%     215µs ± 7%  -15.05%          (p=0.008 n=5+5)
Scan1000Limit100_Cockroach-4     524µs ± 4%     523µs ± 2%     ~             (p=0.841 n=5+5)

name                          old alloc/op   new alloc/op   delta
Select1_Cockroach-4             3.91kB ± 0%    3.90kB ± 0%   -0.22%          (p=0.016 n=5+4)
Select2_Cockroach-4             80.7kB ± 0%    81.1kB ± 0%   +0.46%          (p=0.008 n=5+5)
Insert1_Cockroach-4             23.0kB ± 0%    23.4kB ± 0%   +1.51%          (p=0.008 n=5+5)
Insert10_Cockroach-4            67.2kB ± 0%    67.6kB ± 0%   +0.51%          (p=0.008 n=5+5)
Insert100_Cockroach-4            507kB ± 0%     508kB ± 0%     ~             (p=0.222 n=5+5)
Insert1000_Cockroach-4          4.53MB ± 0%    4.54MB ± 1%     ~             (p=0.548 n=5+5)
Update1_Cockroach-4             36.7kB ± 0%    37.5kB ± 0%   +2.03%          (p=0.016 n=5+4)
Update10_Cockroach-4            99.4kB ± 0%   100.2kB ± 0%   +0.81%          (p=0.008 n=5+5)
Update100_Cockroach-4            704kB ± 0%     704kB ± 0%     ~             (p=0.690 n=5+5)
Update1000_Cockroach-4          6.08MB ± 1%    6.12MB ± 1%     ~             (p=0.095 n=5+5)
Delete1_Cockroach-4             29.8kB ± 0%    30.3kB ± 0%   +1.51%          (p=0.008 n=5+5)
Delete10_Cockroach-4            75.7kB ± 0%    76.2kB ± 0%   +0.72%          (p=0.008 n=5+5)
Delete100_Cockroach-4            537kB ± 0%     538kB ± 0%   +0.31%          (p=0.008 n=5+5)
Delete1000_Cockroach-4          4.81MB ± 0%    4.81MB ± 0%     ~             (p=1.000 n=4+4)
Scan1_Cockroach-4               13.3kB ± 0%    13.6kB ± 0%   +2.66%          (p=0.008 n=5+5)
Scan10_Cockroach-4              17.1kB ± 0%    17.5kB ± 0%   +2.12%          (p=0.008 n=5+5)
Scan100_Cockroach-4             49.4kB ± 0%    49.8kB ± 0%   +0.76%          (p=0.008 n=5+5)
Scan1000_Cockroach-4             331kB ± 0%     332kB ± 0%   +0.18%          (p=0.008 n=5+5)
Scan10000_Cockroach-4           5.72MB ± 0%    5.72MB ± 0%   +0.03%          (p=0.008 n=5+5)
Scan1000Limit1_Cockroach-4      14.0kB ± 0%    14.4kB ± 0%   +2.67%          (p=0.008 n=5+5)
Scan1000Limit10_Cockroach-4     17.9kB ± 0%    18.2kB ± 0%   +2.04%          (p=0.008 n=5+5)
Scan1000Limit100_Cockroach-4    50.0kB ± 0%    50.4kB ± 0%   +0.73%          (p=0.008 n=5+5)

name                          old allocs/op  new allocs/op  delta
Select1_Cockroach-4               97.0 ± 0%      97.0 ± 0%     ~     (all samples are equal)
Select2_Cockroach-4              2.70k ± 0%     2.70k ± 0%   +0.19%          (p=0.008 n=5+5)
Insert1_Cockroach-4                357 ± 0%       361 ± 0%   +1.29%          (p=0.008 n=5+5)
Insert10_Cockroach-4               907 ± 0%       912 ± 0%   +0.53%          (p=0.008 n=5+5)
Insert100_Cockroach-4            6.17k ± 0%     6.18k ± 0%     ~             (p=0.095 n=5+5)
Insert1000_Cockroach-4           58.8k ± 0%     58.8k ± 0%     ~             (p=0.421 n=5+5)
Update1_Cockroach-4                719 ± 0%       729 ± 0%   +1.45%          (p=0.008 n=5+5)
Update10_Cockroach-4             1.41k ± 0%     1.42k ± 0%   +0.77%          (p=0.016 n=5+4)
Update100_Cockroach-4            7.93k ± 0%     7.92k ± 0%     ~             (p=0.667 n=5+5)
Update1000_Cockroach-4           70.0k ± 0%     70.1k ± 0%     ~             (p=0.087 n=5+5)
Delete1_Cockroach-4                567 ± 0%       573 ± 0%   +0.99%          (p=0.016 n=4+5)
Delete10_Cockroach-4             1.24k ± 0%     1.24k ± 0%   +0.49%          (p=0.016 n=4+5)
Delete100_Cockroach-4            7.68k ± 0%     7.69k ± 0%   +0.07%          (p=0.024 n=5+5)
Delete1000_Cockroach-4           71.7k ± 0%     71.7k ± 0%     ~             (p=0.905 n=4+5)
Scan1_Cockroach-4                  276 ± 0%       280 ± 0%   +1.60%          (p=0.016 n=5+4)
Scan10_Cockroach-4                 356 ± 0%       360 ± 0%   +1.35%          (p=0.008 n=5+5)
Scan100_Cockroach-4              1.08k ± 0%     1.09k ± 0%   +0.46%          (p=0.016 n=4+5)
Scan1000_Cockroach-4             8.29k ± 0%     8.30k ± 0%   +0.08%          (p=0.008 n=5+5)
Scan10000_Cockroach-4            80.4k ± 0%     80.5k ± 0%   +0.02%          (p=0.008 n=5+5)
Scan1000Limit1_Cockroach-4         298 ± 0%       303 ± 0%   +1.68%          (p=0.008 n=5+5)
Scan1000Limit10_Cockroach-4        378 ± 0%       383 ± 0%   +1.32%          (p=0.029 n=4+4)
Scan1000Limit100_Cockroach-4     1.10k ± 0%     1.11k ± 0%   +0.45%          (p=0.016 n=5+4)
```

Closes #1283.
Fixes #1521.
Closes #1633.
Closes #2642.
Fixes #3942.
Closes #4854.
  • Loading branch information
tamird committed Mar 9, 2016
commit 4f27ef42d59604b6c20685dc0fab6cbd37aee20f
2 changes: 2 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,8 @@ dupl:
check:
@echo "checking for proto.Clone calls (use util.CloneProto instead)"
@! git grep -E '\.Clone\([^)]+\)' | grep -vE '^util/clone_proto(_test)?\.go:'
@echo "checking for grpc.NewServer calls (use rpc.NewServer instead)"
@! git grep -E 'grpc\.NewServer\(\)' | grep -vE '^rpc/context(_test)?\.go:'
@echo "misspell"
@! git ls-files | xargs misspell | grep -vF 'No Exceptions'
@echo "checking for tabs in shell scripts"
Expand Down
9 changes: 6 additions & 3 deletions acceptance/cluster/cluster.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ package cluster
import (
"net"
"testing"

"github.com/cockroachdb/cockroach/client"
"github.com/cockroachdb/cockroach/util/stop"
)

// A Cluster is an abstraction away from a concrete cluster deployment (i.e.
Expand All @@ -27,12 +30,12 @@ import (
type Cluster interface {
// NumNodes returns the number of nodes in the cluster, running or not.
NumNodes() int
// ConnString returns a connection string for the given node.
ConnString(int) string
// NewClient returns a kv client for the given node.
NewClient(*testing.T, int) (*client.DB, *stop.Stopper)
// PGUrl returns a URL string for the given node postgres server.
PGUrl(int) string
// Addr returns the TCP address for the given node.
Addr(i int) *net.TCPAddr
Addr(int) *net.TCPAddr
// Assert verifies that the cluster state is as expected (i.e. no unexpected
// restarts or node deaths occurred). Tests can call this periodically to
// ascertain cluster health.
Expand Down
29 changes: 20 additions & 9 deletions acceptance/cluster/localcluster.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,17 +31,21 @@ import (
"testing"
"time"

"github.com/cockroachdb/cockroach/base"
"github.com/cockroachdb/cockroach/security"
"github.com/cockroachdb/cockroach/util"
"github.com/cockroachdb/cockroach/util/log"
"github.com/docker/engine-api/client"
"github.com/docker/engine-api/types"
"github.com/docker/engine-api/types/container"
"github.com/docker/engine-api/types/events"
"github.com/docker/engine-api/types/network"
"github.com/docker/go-connections/nat"
"golang.org/x/net/context"

"github.com/cockroachdb/cockroach/base"
roachClient "github.com/cockroachdb/cockroach/client"
"github.com/cockroachdb/cockroach/rpc"
"github.com/cockroachdb/cockroach/security"
"github.com/cockroachdb/cockroach/util"
"github.com/cockroachdb/cockroach/util/log"
"github.com/cockroachdb/cockroach/util/stop"
)

const (
Expand Down Expand Up @@ -631,11 +635,18 @@ func (l *LocalCluster) stop() {
}
}

// ConnString creates a connections string.
func (l *LocalCluster) ConnString(i int) string {
return "rpcs://" + security.NodeUser + "@" +
l.Addr(i).String() +
"?certs=" + l.CertsDir
// NewClient implements the Cluster interface.
func (l *LocalCluster) NewClient(t *testing.T, i int) (*roachClient.DB, *stop.Stopper) {
stopper := stop.NewStopper()
rpcContext := rpc.NewContext(&base.Context{
User: security.NodeUser,
Certs: l.CertsDir,
}, nil, stopper)
sender, err := roachClient.NewSender(rpcContext, l.Addr(i).String())
if err != nil {
t.Fatal(err)
}
return roachClient.NewDB(sender), stopper
}

// Addr returns the TCP address for the given node.
Expand Down
2 changes: 1 addition & 1 deletion acceptance/gossip_peerings_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ func testGossipRestartInner(t *testing.T, c cluster.Cluster, cfg cluster.TestCon
checkGossip(t, c, waitTime, hasSentinel)

for i := 0; i < num; i++ {
db, dbStopper := makeClient(t, c.ConnString(i))
db, dbStopper := c.NewClient(t, i)
if i == 0 {
if err := db.Del("count"); err != nil {
t.Fatal(err)
Expand Down
2 changes: 1 addition & 1 deletion acceptance/put_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ func TestPut(t *testing.T) {
}

func testPutInner(t *testing.T, c cluster.Cluster, cfg cluster.TestConfig) {
db, dbStopper := makeClient(t, c.ConnString(0))
db, dbStopper := c.NewClient(t, 0)
defer dbStopper.Stop()

errs := make(chan error, c.NumNodes())
Expand Down
2 changes: 1 addition & 1 deletion acceptance/replication_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ func countRangeReplicas(db *client.DB) (int, error) {

func checkRangeReplication(t *testing.T, c cluster.Cluster, d time.Duration) {
// Always talk to node 0.
client, dbStopper := makeClient(t, c.ConnString(0))
client, dbStopper := c.NewClient(t, 0)
defer dbStopper.Stop()

wantedReplicas := 3
Expand Down
4 changes: 2 additions & 2 deletions acceptance/single_key_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ func testSingleKeyInner(t *testing.T, c cluster.Cluster, cfg cluster.TestConfig)

// Initialize the value for our test key to zero.
const key = "test-key"
initDB, initDBStopper := makeClient(t, c.ConnString(0))
initDB, initDBStopper := c.NewClient(t, 0)
defer initDBStopper.Stop()
if err := initDB.Put(key, 0); err != nil {
t.Fatal(err)
Expand All @@ -58,7 +58,7 @@ func testSingleKeyInner(t *testing.T, c cluster.Cluster, cfg cluster.TestConfig)
// key. Each worker is configured to talk to a different node in the
// cluster.
for i := 0; i < num; i++ {
db, dbStopper := makeClient(t, c.ConnString(i))
db, dbStopper := c.NewClient(t, i)
defer dbStopper.Stop()
go func() {
var r result
Expand Down
19 changes: 14 additions & 5 deletions acceptance/terrafarm/farmer.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,11 @@ import (
"time"

"github.com/cockroachdb/cockroach/base"
"github.com/cockroachdb/cockroach/client"
"github.com/cockroachdb/cockroach/rpc"
"github.com/cockroachdb/cockroach/security"
"github.com/cockroachdb/cockroach/util/retry"
"github.com/cockroachdb/cockroach/util/stop"
)

// A Farmer sets up and manipulates a test cluster via terraform.
Expand Down Expand Up @@ -163,12 +166,18 @@ func (f *Farmer) Exec(i int, cmd string) error {
return nil
}

// ConnString returns a connection string to pass to client.Open().
func (f *Farmer) ConnString(i int) string {
// NewClient implements the Cluster interface.
func (f *Farmer) NewClient(t *testing.T, i int) (*client.DB, *stop.Stopper) {
// TODO(tschottdorf,mberhault): TLS all the things!
return "rpc://" + security.RootUser + "@" +
net.JoinHostPort(f.Nodes()[i], base.DefaultPort) +
"?certs=" + "certswhocares"
stopper := stop.NewStopper()
rpcContext := rpc.NewContext(&base.Context{
User: security.RootUser,
}, nil, stopper)
sender, err := client.NewSender(rpcContext, f.Addr(i).String())
if err != nil {
t.Fatal(err)
}
return client.NewDB(sender), stopper
}

// PGUrl returns a URL string for the given node postgres server.
Expand Down
11 changes: 0 additions & 11 deletions acceptance/util_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,8 @@ import (
"github.com/cockroachdb/cockroach/acceptance/cluster"
"github.com/cockroachdb/cockroach/acceptance/terrafarm"
"github.com/cockroachdb/cockroach/base"
"github.com/cockroachdb/cockroach/client"
"github.com/cockroachdb/cockroach/util/caller"
"github.com/cockroachdb/cockroach/util/log"
"github.com/cockroachdb/cockroach/util/stop"
)

var flagDuration = flag.Duration("d", cluster.DefaultDuration, "duration to run the test")
Expand Down Expand Up @@ -178,15 +176,6 @@ func SkipUnlessLocal(t *testing.T) {
}
}

func makeClient(t *testing.T, str string) (*client.DB, *stop.Stopper) {
stopper := stop.NewStopper()
db, err := client.Open(stopper, str)
if err != nil {
t.Fatal(err)
}
return db, stopper
}

func makePGClient(t *testing.T, dest string) *sql.DB {
db, err := sql.Open("postgres", dest)
if err != nil {
Expand Down
19 changes: 9 additions & 10 deletions cli/kv.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,11 @@ import (
"strconv"
"strings"

"github.com/cockroachdb/cockroach/base"
"github.com/cockroachdb/cockroach/client"
"github.com/cockroachdb/cockroach/keys"
"github.com/cockroachdb/cockroach/roachpb"
"github.com/cockroachdb/cockroach/rpc"
"github.com/cockroachdb/cockroach/security"
"github.com/cockroachdb/cockroach/util/stop"

Expand All @@ -33,20 +35,17 @@ import (

func makeDBClient() (*client.DB, *stop.Stopper) {
stopper := stop.NewStopper()

// TODO(marc): KV endpoints are now restricted to node users.
// This should probably be made more explicit.
db, err := client.Open(stopper, fmt.Sprintf(
"%s://%s@%s?certs=%s",
cliContext.RPCRequestScheme(),
security.NodeUser,
cliContext.Addr,
cliContext.Certs))
context := &base.Context{
User: security.NodeUser,
Certs: cliContext.Certs,
Insecure: cliContext.Insecure,
}
sender, err := client.NewSender(rpc.NewContext(context, nil, stopper), cliContext.Addr)
if err != nil {
stopper.Stop()
panicf("failed to initialize KV client: %s", err)
}
return db, stopper
return client.NewDB(sender), stopper
}

// unquoteArg unquotes the provided argument using Go double-quoted
Expand Down
31 changes: 11 additions & 20 deletions client/client_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,14 +28,15 @@ import (

"golang.org/x/net/context"

"github.com/cockroachdb/cockroach/base"
"github.com/cockroachdb/cockroach/client"
"github.com/cockroachdb/cockroach/config"
"github.com/cockroachdb/cockroach/roachpb"
"github.com/cockroachdb/cockroach/rpc"
"github.com/cockroachdb/cockroach/security"
"github.com/cockroachdb/cockroach/server"
"github.com/cockroachdb/cockroach/testutils"
"github.com/cockroachdb/cockroach/util/leaktest"
"github.com/cockroachdb/cockroach/util/log"
"github.com/cockroachdb/cockroach/util/retry"
"github.com/cockroachdb/cockroach/util/stop"
"github.com/gogo/protobuf/proto"
Expand Down Expand Up @@ -75,25 +76,21 @@ func createTestClient(t *testing.T, stopper *stop.Stopper, addr string) *client.
}

func createTestClientForUser(t *testing.T, stopper *stop.Stopper, addr, user string) *client.DB {
db, err := client.Open(stopper, fmt.Sprintf("rpcs://%s@%s?certs=%s",
user, addr, security.EmbeddedCertsDir))
rpcContext := rpc.NewContext(&base.Context{
User: user,
Certs: security.EmbeddedCertsDir,
}, nil, stopper)
sender, err := client.NewSender(rpcContext, addr)
if err != nil {
t.Fatal(err)
}
return db
return client.NewDB(sender)
}

// createTestNotifyClient creates a new client which connects using an HTTP
// sender to the server at addr. It contains a waitgroup to allow waiting.
func createTestNotifyClient(stopper *stop.Stopper, addr string, priority roachpb.UserPriority) (*client.DB, *notifyingSender) {
db, err := client.Open(stopper, fmt.Sprintf("rpcs://%s@%s?certs=%s",
security.NodeUser,
addr,
security.EmbeddedCertsDir))
if err != nil {
log.Fatal(err)
}

func createTestNotifyClient(t *testing.T, stopper *stop.Stopper, addr string, priority roachpb.UserPriority) (*client.DB, *notifyingSender) {
db := createTestClient(t, stopper, addr)
sender := &notifyingSender{wrapped: db.GetSender()}
return client.NewDBWithPriority(sender, priority), sender
}
Expand Down Expand Up @@ -143,7 +140,7 @@ func TestClientRetryNonTxn(t *testing.T) {
txnPri = 2
}

db, sender := createTestNotifyClient(s.Stopper(), s.ServingAddr(), -clientPri)
db, sender := createTestNotifyClient(t, s.Stopper(), s.ServingAddr(), -clientPri)

// doneCall signals when the non-txn read or write has completed.
doneCall := make(chan struct{})
Expand Down Expand Up @@ -627,12 +624,6 @@ func TestClientPermissions(t *testing.T) {
s := server.StartTestServer(t)
defer s.Stop()

oldHealthyTimeout := client.HealthyTimeout
client.HealthyTimeout = 250 * time.Millisecond
defer func() {
client.HealthyTimeout = oldHealthyTimeout
}()

// NodeUser certs are required for all KV operations.
// RootUser has no KV privileges whatsoever.
nodeClient := createTestClientForUser(t, s.Stopper(), s.ServingAddr(), security.NodeUser)
Expand Down
69 changes: 0 additions & 69 deletions client/db.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,17 +19,13 @@ package client
import (
"bytes"
"fmt"
"net/url"
"strconv"
"time"

"golang.org/x/net/context"

"github.com/cockroachdb/cockroach/base"
"github.com/cockroachdb/cockroach/roachpb"
"github.com/cockroachdb/cockroach/util/log"
"github.com/cockroachdb/cockroach/util/retry"
"github.com/cockroachdb/cockroach/util/stop"
"github.com/cockroachdb/cockroach/util/tracing"
"github.com/gogo/protobuf/proto"
)
Expand Down Expand Up @@ -198,71 +194,6 @@ func NewDBWithPriority(sender Sender, userPriority roachpb.UserPriority) *DB {
return db
}

// TODO(pmattis): Allow setting the sender/txn retry options.

// Open creates a new database handle to the cockroach cluster specified by
// addr. The cluster is identified by a URL with the format:
//
// [<sender>:]//[<user>@]<host>:<port>[?certs=<dir>,priority=<val>]
//
// The URL scheme (<sender>) specifies which transport to use for talking to
// the cockroach cluster. Currently allowable values are: http, https, rpc,
// rpcs. The rpc and rpcs senders use a variant of Go's builtin rpc library for
// communication with the cluster. This protocol is lower overhead and more
// efficient than http. The decision between the encrypted (https, rpcs) and
// unencrypted senders (http, rpc) depends on the settings of the cluster. A
// given cluster supports either encrypted or unencrypted traffic, but not
// both.
//
// If not specified, the <user> field defaults to "root".
//
// The certs parameter can be used to override the default directory to use for
// client certificates. In tests, the directory "test_certs" uses the embedded
// test certificates.
//
// The priority parameter can be used to override the default priority for
// operations.
func Open(stopper *stop.Stopper, addr string) (*DB, error) {
u, err := url.Parse(addr)
if err != nil {
return nil, err
}
ctx := &base.Context{}
ctx.InitDefaults()
if u.User != nil {
ctx.User = u.User.Username()
}

q := u.Query()
if dir := q["certs"]; len(dir) > 0 {
ctx.Certs = dir[0]
}

sender, err := newSender(u, ctx, stopper)
if err != nil {
return nil, err
}
if sender == nil {
return nil, fmt.Errorf("\"%s\" no sender specified", addr)
}

db := &DB{
sender: sender,
userPriority: roachpb.NormalUserPriority,
txnRetryOptions: DefaultTxnRetryOptions,
}

if priority := q["priority"]; len(priority) > 0 {
p, err := strconv.ParseFloat(priority[0], 64)
if err != nil {
return nil, err
}
db.userPriority = roachpb.UserPriority(p)
}

return db, nil
}

// NewBatch creates and returns a new empty batch object for use with the DB.
// TODO(tschottdorf): it appears this can be unexported.
func (db *DB) NewBatch() *Batch {
Expand Down
Loading