You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am seriously struggling with this code base and design of TiUP. It assumes there is always a ~/.tiup directory, a root.json file and a mirror defined. I am not able to override the root.json lookup by simply loading an existing system-wide /usr/share/tiup/root.json as loading the profile, home directory and having the root.json specifically inside the home directory is mandatory through multiple levels of abstractions and simply injecting a custom path in local.go or local_manifests.go won't work.
Also, by default it is not possible to run any commands unless the repository object is loaded. By default, running any command requires network connection even though the command does not do anything that would actually need network, e.g.:
./bin/tiup tiup-client --help
The component `tiup-client` version is not installed; downloading from repository.
Error: fetch /timestamp.json from mirror(https://tiup-mirrors.pingcap.com/) failed: download from https://tiup-mirrors.pingcap.com/timestamp.json failed: Get "https://tiup-mirrors.pingcap.com/timestamp.json": dial tcp: lookup tiup-mirrors.pingcap.com on 172.20.91.201:53: dial udp 172.20.91.201:53: connect: network is unreachable
The PR 2538 fixes the above issue, but the code base is full of assumptions that are inherently incompatible with installing the binaries at /usr/bin/tiup, /usr/bin/tiup-client etc and allowing users to run them as-is without additional mandatory setup and creating new .tiup directories etc. This makes it very challenging to package this for any Linux distribution.
Doing a regular Go build should produce the binary using regular Go source code structure and dependencies.
The name of the binary needs to be specific for the project, e.g. not just client or server or a.
The binary should run as-is without requiring that user has any extra ENV set or extra magic files. Extra stuff is allowed when it enables more functionality, but basic functionality should work as-is (e.g. program --help should produce help text and not an error).
It should be possible to run the binary on a read-only filesystem without network. Functionality can be limited, but it should still run.
Currently, tiup can be built, but tiup-client is produced only by make build that has a custom rename step.
Currently, tiup can't run at all without extra magic file (root.json) available, a writeable .tiup directory and live network connectivity. For example tiup --help won't produce any help text unless the listed prerequisited are satisfied. I assumed I could hack around these with a few Go patches, but this structure seems quite deep and a fundamental part of how TiUP is designed.
This will also block #2507 and #2509 as they have the same generic requirements of being able to run a binary without magic ENV or extra files.
I am currently iterating on #2538 to maybe get past these issues, but not sure if I will make progress.
While going through the dependencies I noticed that https://github.com/AstroProfundis/sysinfo is a fork of https://github.com/zcalusic/sysinfo that seems to now be maintained in a PingCAP employee personal GitHub account. As PingCAP relies on this repository, it should probably be in PingCAP's namespace. Alternatively all improvements could be sent upstream to not need to have a fork at all.
TiUP depends on old github.com/r3labs/diff/v2 but Debian golang-github-r3labs-diff-dev has only v3, which does not build on TiUP. Would it be feasible to port TiUP to use this newer library version?
TiUP depends on old github.com/grpc-ecosystem/grpc-gateway v1.16.0 but the Debian golang-github-grpc-ecosystem-grpc-gateway-dev is of version 2.x, and does not build with TiUP. Would it be feasible to port TiUP to use this newer library version?
Additionally the code needst to be updated to use new functions or build fails with:
# github.com/pingcap/tiup/pkg/cluster/api/dmpb
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1152:141: too many arguments in call to runtime.NewPattern
have (number, []int, []string, string, unknown type)
want (int, []int, []string, string)
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1152:149: undefined: runtime.AssumeColonVerbOpt
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1154:169: too many arguments in call to runtime.NewPattern
have (number, []int, []string, string, unknown type)
want (int, []int, []string, string)
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1154:177: undefined: runtime.AssumeColonVerbOpt
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1156:170: too many arguments in call to runtime.NewPattern
have (number, []int, []string, string, unknown type)
want (int, []int, []string, string)
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1156:178: undefined: runtime.AssumeColonVerbOpt
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1158:147: too many arguments in call to runtime.NewPattern
have (number, []int, []string, string, unknown type)
want (int, []int, []string, string)
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1158:155: undefined: runtime.AssumeColonVerbOpt
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1160:199: too many arguments in call to runtime.NewPattern
have (number, []int, []string, string, unknown type)
want (int, []int, []string, string)
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1160:207: undefined: runtime.AssumeColonVerbOpt
src/github.com/pingcap/tiup/pkg/cluster/api/dmpb/dmmaster.pb.gw.go:1160:199: too many errors
Other PingCAP/TiDB specific issues when trying to build dependencies as independent packages for Debian:
golang-github-pingcap-tidb-insight
dh_auto_build -O--builddirectory=debian/.build/upstream -O--buildsystem=golang
cd debian/.build/upstream && go install -trimpath -v -p 4 github.com/pingcap/tidb-insight/collector github.com/pingcap/tidb-insight/collector/insight github.com/pingcap/tidb-insight/collector/kmsg github.com/pingcap/tidb-insight/docs github.com/pingcap/tidb-insight/tools
src/github.com/pingcap/tidb-insight/collector/insight/insight.go:22:2: cannot find package "github.com/AstroProfundis/sysinfo" in any of:
/usr/lib/go-1.24/src/github.com/AstroProfundis/sysinfo (from $GOROOT)
/tmp/build/source/debian/.build/upstream/src/github.com/AstroProfundis/sysinfo (from $GOPATH)
src/github.com/pingcap/tidb-insight/collector/insight/syscfg.go:21:2: cannot find package "github.com/lorenzosaino/go-sysctl" in any of:
/usr/lib/go-1.24/src/github.com/lorenzosaino/go-sysctl (from $GOROOT)
/tmp/build/source/debian/.build/upstream/src/github.com/lorenzosaino/go-sysctl (from $GOPATH)
src/github.com/pingcap/tidb-insight/collector/insight/process.go:10:2: cannot find package "github.com/shirou/gopsutil/cpu" in any of:
/usr/lib/go-1.24/src/github.com/shirou/gopsutil/cpu (from $GOROOT)
/tmp/build/source/debian/.build/upstream/src/github.com/shirou/gopsutil/cpu (from $GOPATH)
src/github.com/pingcap/tidb-insight/collector/insight/pd.go:11:2: cannot find package "github.com/shirou/gopsutil/process" in any of:
/usr/lib/go-1.24/src/github.com/shirou/gopsutil/process (from $GOROOT)
/tmp/build/source/debian/.build/upstream/src/github.com/shirou/gopsutil/process (from $GOPATH)
src/github.com/pingcap/tidb-insight/collector/insight/socket.go:16:2: cannot find package "github.com/vishvananda/netlink" in any of:
/usr/lib/go-1.24/src/github.com/vishvananda/netlink (from $GOROOT)
/tmp/build/source/debian/.build/upstream/src/github.com/vishvananda/netlink (from $GOPATH)
src/github.com/pingcap/tidb-insight/collector/insight/epoll.go:16:2: cannot find package "golang.org/x/sys/unix" in any of:
/usr/lib/go-1.24/src/golang.org/x/sys/unix (from $GOROOT)
/tmp/build/source/debian/.build/upstream/src/golang.org/x/sys/unix (from $GOPATH)
src/github.com/pingcap/tidb-insight/tools/prom2influx.go:13:2: cannot find package "github.com/influxdata/influxdb/client/v2" in any of:
/usr/lib/go-1.24/src/github.com/influxdata/influxdb/client/v2 (from $GOROOT)
/tmp/build/source/debian/.build/upstream/src/github.com/influxdata/influxdb/client/v2 (from $GOPATH)
src/github.com/pingcap/tidb-insight/tools/prom2influx.go:14:2: cannot find package "github.com/prometheus/common/model" in any of:
/usr/lib/go-1.24/src/github.com/prometheus/common/model (from $GOROOT)
/tmp/build/source/debian/.build/upstream/src/github.com/prometheus/common/model (from $GOPATH)
golang-github-pingcap-kvproto
Investigating (0) golang-google-genproto-dev:amd64 < none -> 0.0~git20210726.e7812ac-4 @un uN Ib >
Broken golang-google-genproto-dev:amd64 Depends on golang-github-golang-protobuf-1-5-dev:amd64 < none | 1.5.4-1 @un uH >
Considering golang-github-golang-protobuf-1-5-dev:amd64 5 as a solution to golang-google-genproto-dev:amd64 11
Holding Back golang-google-genproto-dev:amd64 rather than change golang-github-golang-protobuf-1-5-dev:amd64
Investigating (0) golang-go.opencensus-dev:amd64 < none -> 0.24.0-2 @un uN Ib >
Broken golang-go.opencensus-dev:amd64 Depends on golang-github-golang-protobuf-1-5-dev:amd64 < none | 1.5.4-1 @un uH >
Considering golang-github-golang-protobuf-1-5-dev:amd64 5 as a solution to golang-go.opencensus-dev:amd64 8
Holding Back golang-go.opencensus-dev:amd64 rather than change golang-github-golang-protobuf-1-5-dev:amd64
Investigating (0) golang-google-cloud-dev:amd64 < none -> 0.56.0-6 @un uN Ib >
Broken golang-google-cloud-dev:amd64 Depends on golang-github-golang-protobuf-1-5-dev:amd64 < none | 1.5.4-1 @un uH >
Considering golang-github-golang-protobuf-1-5-dev:amd64 5 as a solution to golang-google-cloud-dev:amd64 6
Holding Back golang-google-cloud-dev:amd64 rather than change golang-github-golang-protobuf-1-5-dev:amd64
Investigating (0) golang-github-google-cel-go-dev:amd64 < none -> 0.18.2+ds-5 @un uN Ib >
Broken golang-github-google-cel-go-dev:amd64 Depends on golang-github-golang-protobuf-1-5-dev:amd64 < none | 1.5.4-1 @un uH >
Considering golang-github-golang-protobuf-1-5-dev:amd64 5 as a solution to golang-github-google-cel-go-dev:amd64 6
Holding Back golang-github-google-cel-go-dev:amd64 rather than change golang-github-golang-protobuf-1-5-dev:amd64
Investigating (0) golang-github-grpc-ecosystem-grpc-gateway-dev:amd64 < none -> 2.20.0-2 @un uN Ib >
Broken golang-github-grpc-ecosystem-grpc-gateway-dev:amd64 Depends on golang-google-genproto-dev:amd64 < none | 0.0~git20210726.e7812ac-4 @un uH > (>= 0.0~git20210726.e7812ac~)
Considering golang-google-genproto-dev:amd64 11 as a solution to golang-github-grpc-ecosystem-grpc-gateway-dev:amd64 0
Holding Back golang-github-grpc-ecosystem-grpc-gateway-dev:amd64 rather than change golang-google-genproto-dev:amd64
Investigating (0) golang-google-api-dev:amd64 < none -> 0.61.0-6 @un uN Ib >
Broken golang-google-api-dev:amd64 Depends on golang-go.opencensus-dev:amd64 < none | 0.24.0-2 @un uH > (>= 0.23.0)
Considering golang-go.opencensus-dev:amd64 8 as a solution to golang-google-api-dev:amd64 0
Holding Back golang-google-api-dev:amd64 rather than change golang-go.opencensus-dev:amd64
Investigating (0) golang-github-pingcap-kvproto-build-deps:amd64 < 1.0 @iU K Nb Ib >
Broken golang-github-pingcap-kvproto-build-deps:amd64 Depends on golang-github-grpc-ecosystem-grpc-gateway-dev:amd64 < none | 2.20.0-2 @un uH >
Considering golang-github-grpc-ecosystem-grpc-gateway-dev:amd64 0 as a solution to golang-github-pingcap-kvproto-build-deps:amd64 -2
Removing golang-github-pingcap-kvproto-build-deps:amd64 rather than change golang-github-grpc-ecosystem-grpc-gateway-dev:amd64
Investigating (1) golang-google-grpc-dev:amd64 < none -> 1.64.0-7 @un uN Ib >
Broken golang-google-grpc-dev:amd64 Depends on golang-github-google-cel-go-dev:amd64 < none | 0.18.2+ds-5 @un uH >
Considering golang-github-google-cel-go-dev:amd64 6 as a solution to golang-google-grpc-dev:amd64 14
Holding Back golang-google-grpc-dev:amd64 rather than change golang-github-google-cel-go-dev:amd64
Investigating (1) golang-github-googleapis-gax-go-dev:amd64 < none -> 2.14.1-1 @un uN Ib >
Broken golang-github-googleapis-gax-go-dev:amd64 Depends on golang-google-grpc-dev:amd64 < none | 1.64.0-7 @un uH >
Considering golang-google-grpc-dev:amd64 14 as a solution to golang-github-googleapis-gax-go-dev:amd64 1
Holding Back golang-github-googleapis-gax-go-dev:amd64 rather than change golang-google-grpc-dev:amd64
golang-github-pingcap-errors
=== RUN TestFormatNew
format_test.go:38: test 3: line 2: fmt.Sprintf("%+v", err):
got: "error\ngithub.com/pingcap/errors.TestFormatNew\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:26\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700"
want: "error\ngithub.com/pkg/errors.TestFormatNew\n\t.+/github.com/pkg/errors/format_test.go:26"
format_test.go:38: test 3: line 3: fmt.Sprintf("%+v", err):
got: "error\ngithub.com/pingcap/errors.TestFormatNew\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:26\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700"
want: "error\ngithub.com/pkg/errors.TestFormatNew\n\t.+/github.com/pkg/errors/format_test.go:26"
--- FAIL: TestFormatNew (0.00s)
=== RUN TestFormatErrorf
format_test.go:64: test 3: line 2: fmt.Sprintf("%+v", err):
got: "error\ngithub.com/pingcap/errors.TestFormatErrorf\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:56\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700"
want: "error\ngithub.com/pkg/errors.TestFormatErrorf\n\t.+/github.com/pkg/errors/format_test.go:56"
format_test.go:64: test 3: line 3: fmt.Sprintf("%+v", err):
got: "error\ngithub.com/pingcap/errors.TestFormatErrorf\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:56\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700"
want: "error\ngithub.com/pkg/errors.TestFormatErrorf\n\t.+/github.com/pkg/errors/format_test.go:56"
--- FAIL: TestFormatErrorf (0.00s)
=== RUN TestFormatWrap
format_test.go:116: test 3: line 2: fmt.Sprintf("%+v", err):
got: "error\ngithub.com/pingcap/errors.TestFormatWrap\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:82\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700\nerror2"
want: "error\ngithub.com/pkg/errors.TestFormatWrap\n\t.+/github.com/pkg/errors/format_test.go:82"
format_test.go:116: test 3: line 3: fmt.Sprintf("%+v", err):
got: "error\ngithub.com/pingcap/errors.TestFormatWrap\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:82\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700\nerror2"
want: "error\ngithub.com/pkg/errors.TestFormatWrap\n\t.+/github.com/pkg/errors/format_test.go:82"
format_test.go:116: test 6: line 3: fmt.Sprintf("%+v", err):
got: "EOF\nerror\ngithub.com/pingcap/errors.TestFormatWrap\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:96\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700"
want: "EOF\nerror\ngithub.com/pkg/errors.TestFormatWrap\n\t.+/github.com/pkg/errors/format_test.go:96"
format_test.go:116: test 6: line 4: fmt.Sprintf("%+v", err):
got: "EOF\nerror\ngithub.com/pingcap/errors.TestFormatWrap\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:96\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700"
want: "EOF\nerror\ngithub.com/pkg/errors.TestFormatWrap\n\t.+/github.com/pkg/errors/format_test.go:96"
format_test.go:116: test 7: line 3: fmt.Sprintf("%+v", err):
got: "EOF\nerror1\ngithub.com/pingcap/errors.TestFormatWrap\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:103\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700\nerror2"
want: "EOF\nerror1\ngithub.com/pkg/errors.TestFormatWrap\n\t.+/github.com/pkg/errors/format_test.go:103\n"
format_test.go:116: test 7: line 4: fmt.Sprintf("%+v", err):
got: "EOF\nerror1\ngithub.com/pingcap/errors.TestFormatWrap\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:103\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700\nerror2"
want: "EOF\nerror1\ngithub.com/pkg/errors.TestFormatWrap\n\t.+/github.com/pkg/errors/format_test.go:103\n"
--- FAIL: TestFormatWrap (0.00s)
=== RUN TestFormatWrapf
format_test.go:157: test 3: line 3: fmt.Sprintf("%+v", err):
got: "EOF\nerror2\ngithub.com/pingcap/errors.TestFormatWrapf\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:134\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700"
want: "EOF\nerror2\ngithub.com/pkg/errors.TestFormatWrapf\n\t.+/github.com/pkg/errors/format_test.go:134"
format_test.go:157: test 3: line 4: fmt.Sprintf("%+v", err):
got: "EOF\nerror2\ngithub.com/pingcap/errors.TestFormatWrapf\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:134\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700"
want: "EOF\nerror2\ngithub.com/pkg/errors.TestFormatWrapf\n\t.+/github.com/pkg/errors/format_test.go:134"
format_test.go:157: test 6: line 2: fmt.Sprintf("%+v", err):
got: "error\ngithub.com/pingcap/errors.TestFormatWrapf\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:149\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700\nerror2"
want: "error\ngithub.com/pkg/errors.TestFormatWrapf\n\t.+/github.com/pkg/errors/format_test.go:149"
format_test.go:157: test 6: line 3: fmt.Sprintf("%+v", err):
got: "error\ngithub.com/pingcap/errors.TestFormatWrapf\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:149\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700\nerror2"
want: "error\ngithub.com/pkg/errors.TestFormatWrapf\n\t.+/github.com/pkg/errors/format_test.go:149"
--- FAIL: TestFormatWrapf (0.00s)
=== RUN TestFormatWithStack
format_test.go:476: test 3: block 2: fmt.Sprintf("%+v", err):
got:
"github.com/pingcap/errors.TestFormatWithStack\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:175\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700"
want:
"github.com/pkg/errors.TestFormatWithStack\n\t.+/github.com/pkg/errors/format_test.go:175"
all-got:
EOF
github.com/pingcap/errors.TestFormatWithStack
/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:175
testing.tRunner
/usr/lib/go-1.24/src/testing/testing.go:1792
runtime.goexit
/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700
all-want:
EOF
github.com/pkg/errors.TestFormatWithStack
.+/github.com/pkg/errors/format_test.go:175
--- FAIL: TestFormatWithStack (0.00s)
=== RUN TestFormatWithMessage
format_test.go:476: test 3: block 2: fmt.Sprintf("%+v", err):
got:
"github.com/pingcap/errors.TestFormatWithMessage\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:244\ntesting.tRunner\n\t/usr/lib/go-1.24/src/testing/testing.go:1792\nruntime.goexit\n\t/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700"
want:
"github.com/pkg/errors.TestFormatWithMessage\n\t.+/github.com/pkg/errors/format_test.go:244"
all-got:
error
github.com/pingcap/errors.TestFormatWithMessage
/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/format_test.go:244
testing.tRunner
/usr/lib/go-1.24/src/testing/testing.go:1792
runtime.goexit
/usr/lib/go-1.24/src/runtime/asm_amd64.s:1700
error2
all-want:
error
github.com/pkg/errors.TestFormatWithMessage
.+/github.com/pkg/errors/format_test.go:244
error2
--- FAIL: TestFormatWithMessage (0.00s)
=== RUN TestFrameLine
stack_test.go:40: Frame(5391080): want: 9, got: 10
stack_test.go:40: Frame(5433480): want: 20, got: 21
stack_test.go:40: Frame(5433563): want: 28, got: 24
--- FAIL: TestFrameLine (0.00s)
=== RUN TestFrameFormat
stack_test.go:125: test 2: line 1: fmt.Sprintf("%+s", err):
got: "github.com/pingcap/errors.init\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go"
want: "github.com/pkg/errors.init\n\t.+/github.com/pkg/errors/stack_test.go"
stack_test.go:125: test 2: line 2: fmt.Sprintf("%+s", err):
got: "github.com/pingcap/errors.init\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go"
want: "github.com/pkg/errors.init\n\t.+/github.com/pkg/errors/stack_test.go"
stack_test.go:125: test 5: line 1: fmt.Sprintf("%d", err):
got: "10"
want: "9"
stack_test.go:125: test 11: line 1: fmt.Sprintf("%v", err):
got: "stack_test.go:10"
want: "stack_test.go:9"
stack_test.go:125: test 12: line 1: fmt.Sprintf("%+v", err):
got: "github.com/pingcap/errors.init\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:10"
want: "github.com/pkg/errors.init\n\t.+/github.com/pkg/errors/stack_test.go:9"
stack_test.go:125: test 12: line 2: fmt.Sprintf("%+v", err):
got: "github.com/pingcap/errors.init\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:10"
want: "github.com/pkg/errors.init\n\t.+/github.com/pkg/errors/stack_test.go:9"
--- FAIL: TestFrameFormat (0.00s)
=== RUN TestFuncname
--- PASS: TestFuncname (0.00s)
=== RUN TestStackTrace
stack_test.go:203: test 1: line 1: fmt.Sprintf("%+v", err):
got: "github.com/pingcap/errors.TestStackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:155"
want: "github.com/pkg/errors.TestStackTrace\n\t.+/github.com/pkg/errors/stack_test.go:154"
stack_test.go:203: test 1: line 2: fmt.Sprintf("%+v", err):
got: "github.com/pingcap/errors.TestStackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:155"
want: "github.com/pkg/errors.TestStackTrace\n\t.+/github.com/pkg/errors/stack_test.go:154"
stack_test.go:203: test 2: line 1: fmt.Sprintf("%+v", err):
got: "github.com/pingcap/errors.TestStackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:160"
want: "github.com/pkg/errors.TestStackTrace\n\t.+/github.com/pkg/errors/stack_test.go:159"
stack_test.go:203: test 2: line 2: fmt.Sprintf("%+v", err):
got: "github.com/pingcap/errors.TestStackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:160"
want: "github.com/pkg/errors.TestStackTrace\n\t.+/github.com/pkg/errors/stack_test.go:159"
stack_test.go:203: test 3: line 1: fmt.Sprintf("%+v", err):
got: "github.com/pingcap/errors.TestStackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:165"
want: "github.com/pkg/errors.TestStackTrace\n\t.+/github.com/pkg/errors/stack_test.go:164"
stack_test.go:203: test 3: line 2: fmt.Sprintf("%+v", err):
got: "github.com/pingcap/errors.TestStackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:165"
want: "github.com/pkg/errors.TestStackTrace\n\t.+/github.com/pkg/errors/stack_test.go:164"
stack_test.go:203: test 4: line 1: fmt.Sprintf("%+v", err):
got: "github.com/pingcap/errors.TestStackTrace.func1\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:170"
want: "github.com/pkg/errors.(func·009|TestStackTrace.func1)\n\t.+/github.com/pkg/errors/stack_test.go:169"
stack_test.go:203: test 4: line 2: fmt.Sprintf("%+v", err):
got: "github.com/pingcap/errors.TestStackTrace.func1\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:170"
want: "github.com/pkg/errors.(func·009|TestStackTrace.func1)\n\t.+/github.com/pkg/errors/stack_test.go:169"
stack_test.go:203: test 4: line 1: fmt.Sprintf("%+v", err):
got: "github.com/pingcap/errors.TestStackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:170"
want: "github.com/pkg/errors.TestStackTrace\n\t.+/github.com/pkg/errors/stack_test.go:169"
stack_test.go:203: test 4: line 2: fmt.Sprintf("%+v", err):
got: "github.com/pingcap/errors.TestStackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:170"
want: "github.com/pkg/errors.TestStackTrace\n\t.+/github.com/pkg/errors/stack_test.go:169"
stack_test.go:203: test 5: line 1: fmt.Sprintf("%+v", err):
got: "github.com/pingcap/errors.TestStackTrace.TestStackTrace.func2.func3\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:179"
want: "github.com/pkg/errors.(func·010|TestStackTrace.func2.1)\n\t.+/github.com/pkg/errors/stack_test.go:178"
stack_test.go:203: test 5: line 2: fmt.Sprintf("%+v", err):
got: "github.com/pingcap/errors.TestStackTrace.TestStackTrace.func2.func3\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:179"
want: "github.com/pkg/errors.(func·010|TestStackTrace.func2.1)\n\t.+/github.com/pkg/errors/stack_test.go:178"
stack_test.go:203: test 5: line 1: fmt.Sprintf("%+v", err):
got: "github.com/pingcap/errors.TestStackTrace.func2\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:180"
want: "github.com/pkg/errors.(func·011|TestStackTrace.func2)\n\t.+/github.com/pkg/errors/stack_test.go:179"
stack_test.go:203: test 5: line 2: fmt.Sprintf("%+v", err):
got: "github.com/pingcap/errors.TestStackTrace.func2\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:180"
want: "github.com/pkg/errors.(func·011|TestStackTrace.func2)\n\t.+/github.com/pkg/errors/stack_test.go:179"
stack_test.go:203: test 5: line 1: fmt.Sprintf("%+v", err):
got: "github.com/pingcap/errors.TestStackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:181"
want: "github.com/pkg/errors.TestStackTrace\n\t.+/github.com/pkg/errors/stack_test.go:180"
stack_test.go:203: test 5: line 2: fmt.Sprintf("%+v", err):
got: "github.com/pingcap/errors.TestStackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:181"
want: "github.com/pkg/errors.TestStackTrace\n\t.+/github.com/pkg/errors/stack_test.go:180"
--- FAIL: TestStackTrace (0.00s)
=== RUN TestStackTraceFormat
stack_test.go:276: test 11: line 2: fmt.Sprintf("%+v", err):
got: "\ngithub.com/pingcap/errors.stackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:211\ngithub.com/pingcap/errors.TestStackTraceFormat\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:262"
want: "\ngithub.com/pkg/errors.stackTrace\n\t.+/github.com/pkg/errors/stack_test.go:210\ngithub.com/pkg/errors.TestStackTraceFormat\n\t.+/github.com/pkg/errors/stack_test.go:261"
stack_test.go:276: test 11: line 3: fmt.Sprintf("%+v", err):
got: "\ngithub.com/pingcap/errors.stackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:211\ngithub.com/pingcap/errors.TestStackTraceFormat\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:262"
want: "\ngithub.com/pkg/errors.stackTrace\n\t.+/github.com/pkg/errors/stack_test.go:210\ngithub.com/pkg/errors.TestStackTraceFormat\n\t.+/github.com/pkg/errors/stack_test.go:261"
stack_test.go:276: test 11: line 4: fmt.Sprintf("%+v", err):
got: "\ngithub.com/pingcap/errors.stackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:211\ngithub.com/pingcap/errors.TestStackTraceFormat\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:262"
want: "\ngithub.com/pkg/errors.stackTrace\n\t.+/github.com/pkg/errors/stack_test.go:210\ngithub.com/pkg/errors.TestStackTraceFormat\n\t.+/github.com/pkg/errors/stack_test.go:261"
stack_test.go:276: test 11: line 5: fmt.Sprintf("%+v", err):
got: "\ngithub.com/pingcap/errors.stackTrace\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:211\ngithub.com/pingcap/errors.TestStackTraceFormat\n\t/tmp/build/source/debian/.build/upstream/src/github.com/pingcap/errors/stack_test.go:262"
want: "\ngithub.com/pkg/errors.stackTrace\n\t.+/github.com/pkg/errors/stack_test.go:210\ngithub.com/pkg/errors.TestStackTraceFormat\n\t.+/github.com/pkg/errors/stack_test.go:261"
stack_test.go:276: test 12: line 1: fmt.Sprintf("%#v", err):
got: "[]errors.Frame{stack_test.go:211, stack_test.go:270}"
want: "\\[\\]errors.Frame{stack_test.go:210, stack_test.go:269}"
--- FAIL: TestStackTraceFormat (0.00s)
Daniel will take a look into updating r3labs/diff v2 -> v3 and grpc-gateway v1 -> v2.
TiUP has had a policy of not updating dependencies proactively, but only if there are specific errors that absolutely require dependencies to be updates. This explains why many dependencies are outdated, and the Debian packaging may serve as the reason to update some of the dependencies now.
Daniel will also check what is the status of these dependencies:
github.com/pingcap/check v0.0.0-20211026125417-57bd13f7b5f0
github.com/pingcap/errors v0.11.5-0.20201126102027-b0a155152ca3
github.com/pingcap/failpoint v0.0.0-20240528011301-b51a646c7c86
github.com/pingcap/fn v1.0.0
github.com/pingcap/kvproto v0.0.0-20230331024443-349815129e6d
github.com/pingcap/tidb-insight/collector v0.0.0-20220902034607-fb5ae0ddc8c1
github.com/pkg/errors v0.9.1
Maybe they should be updated to latest to ensure TiUP benefits from fixes done in them, and also to avoid risk of anybody re-doing fixes on old versions that actually already existed in latest version?
Maybe the PingCAP maintained modules themselves should have some updates, or at least clarified if the latest tagged version or latest mainline commit should be used?
For the pingcap/errors library, consider removing references to original pkg/errors, and in TiUP stop depending direcly on pkg/errors to not have two libraries that are basically the same?
Looks like the replace rule for yaml.v2 is needed because the upstream is abandoned/unmaintained. Not sure what the best way forward is. Switch from yaml.v2 to yaml.v3 #2558
For pingcap/tidb-insight/collector: I think we should move this to the tiup repo as it isn't actively used elsewhere afaik and it also doensn't seem to be really maintained. I've created Move insight to tiup #2553 for this
I pushed now tiup_1.16.2+20250802~f57cbe37 to my test PPA at https://launchpad.net/~otto/+archive/ubuntu/tidb. The package is only buildable for Ubuntu devel (questing) as it depends on new dependencies not yet in any actual release.
The package can be tested in e.g. a Ubuntu devel container with:
Unfortunately the tiup commands that depend on the real-time update check are segfaulting due to a bug I have in #2538, but the point here is now to demonstrate the build and dependency management.
Thanks to the latest changes on master, I was able to remove these vendored dependencies:
Are you sure the list is correct? Some of these are still in the go.mod on the master branch
You are right, sorry for getting mixed in up in dependency tracking. The only dependencies we got completely rid of was pingcap/tidb-insight and grpc-ecosystem/grpc-gateway.
While working on the latest git head of TiUP I came across a new finding about very outdated transitive dependency zaf/temp in #2588
Alright, so recent changes on TiUP git master branch helped get rid of: golang-github-alecthomas-assert-dev, golang-github-grpc-ecosystem-grpc-gateway-dev, golang-github-otiai10-copy-dev and golang-github-pingcap-check-dev.
Names are Debian package names and screenshot below shows diff between earlier Debian packaging prototype and latest version, which also has new dependencies after I've packaged them for Debian.
FYI: The Debian NEW queue process finally started working again after taken over by a new team, and TiUP got accepted into Debian, and by extension also Ubuntu:
Activity
ottok commentedon Mar 20, 2025
This is a subtask of #2506.
I am currently a bit baffled about these module dependencies that point to a subdirectory inside a specific branch in a module:
These all stem from https://github.com/pingcap/tiup/blob/master/components/client/main.go#L27-L35
What is the point of this code, why isn't it just importing the module in a normal fashion?
dveeden commentedon Mar 20, 2025
I faced the same in #2507. I just created #2517 for this.
ottok commentedon Mar 20, 2025
Seems TiUp has a bunch of Go dependencies that are not yet in Debian at all. One major one is xo/usql, for which I started packaging at https://salsa.debian.org/go-team/packages/usql/-/merge_requests/1
ottok commentedon Apr 10, 2025
For the record,
dh-make-golang estimate github.com/pingcap/tiupreports on latest v.1.16.2:The usql is one of the largest dependencies with a large number of dependencies of its own.
ottok commentedon Apr 11, 2025
I have the build working on multiple platforms now, but tests not passing. Still requires quite a lot of work.
ottok commentedon May 31, 2025
I am seriously struggling with this code base and design of TiUP. It assumes there is always a
~/.tiupdirectory, aroot.jsonfile and a mirror defined. I am not able to override theroot.jsonlookup by simply loading an existing system-wide/usr/share/tiup/root.jsonas loading the profile, home directory and having theroot.jsonspecifically inside the home directory is mandatory through multiple levels of abstractions and simply injecting a custom path inlocal.goorlocal_manifests.gowon't work.Also, by default it is not possible to run any commands unless the repository object is loaded. By default, running any command requires network connection even though the command does not do anything that would actually need network, e.g.:
The PR 2538 fixes the above issue, but the code base is full of assumptions that are inherently incompatible with installing the binaries at
/usr/bin/tiup,/usr/bin/tiup-clientetc and allowing users to run them as-is without additional mandatory setup and creating new.tiupdirectories etc. This makes it very challenging to package this for any Linux distribution.ottok commentedon May 31, 2025
Generic requirements for a Go program in Linux:
program --helpshould produce help text and not an error).Currently,
tiupcan be built, buttiup-clientis produced only bymake buildthat has a custom rename step.Currently,
tiupcan't run at all without extra magic file (root.json) available, a writeable.tiupdirectory and live network connectivity. For exampletiup --helpwon't produce any help text unless the listed prerequisited are satisfied. I assumed I could hack around these with a few Go patches, but this structure seems quite deep and a fundamental part of how TiUP is designed.This will also block #2507 and #2509 as they have the same generic requirements of being able to run a binary without magic ENV or extra files.
I am currently iterating on #2538 to maybe get past these issues, but not sure if I will make progress.
ottok commentedon Jun 9, 2025
While going through the dependencies I noticed that https://github.com/AstroProfundis/sysinfo is a fork of https://github.com/zcalusic/sysinfo that seems to now be maintained in a PingCAP employee personal GitHub account. As PingCAP relies on this repository, it should probably be in PingCAP's namespace. Alternatively all improvements could be sent upstream to not need to have a fork at all.
ottok commentedon Jun 11, 2025
More notes on dependencies:
TiUP depends on old github.com/r3labs/diff/v2 but Debian golang-github-r3labs-diff-dev has only v3, which does not build on TiUP. Would it be feasible to port TiUP to use this newer library version?
TiUP depends on old github.com/grpc-ecosystem/grpc-gateway v1.16.0 but the Debian golang-github-grpc-ecosystem-grpc-gateway-dev is of version 2.x, and does not build with TiUP. Would it be feasible to port TiUP to use this newer library version?
ottok commentedon Jun 25, 2025
Other PingCAP/TiDB specific issues when trying to build dependencies as independent packages for Debian:
golang-github-pingcap-tidb-insight
golang-github-pingcap-kvproto
golang-github-pingcap-errors
ottok commentedon Jun 25, 2025
Ttest PPA is now available:
ottok commentedon Jun 25, 2025
Notes from chat with @dveeden
github.com/pingcap/check v0.0.0-20211026125417-57bd13f7b5f0
github.com/pingcap/errors v0.11.5-0.20201126102027-b0a155152ca3
github.com/pingcap/failpoint v0.0.0-20240528011301-b51a646c7c86
github.com/pingcap/fn v1.0.0
github.com/pingcap/kvproto v0.0.0-20230331024443-349815129e6d
github.com/pingcap/tidb-insight/collector v0.0.0-20220902034607-fb5ae0ddc8c1
github.com/pkg/errors v0.9.1
pingcap/errorslibrary, consider removing references to originalpkg/errors, and in TiUP stop depending direcly onpkg/errorsto not have two libraries that are basically the same?dveeden commentedon Jun 25, 2025
This doesn't seem to work with a
debian:latestorubuntu:latestcontainer.It does work with a
ubuntu:questingcontainer with these steps:Some things I noticed:
tiup --versionmight be usefultiup --versionoutput doesn't show the git hash etctiup clientdoesn't seem to be ale to find/executetiup-clienttiup-clientis directly in the path, but probably this shouldn't be the case as it should not be called directly.tiup completion bashthings could be shipped as a file in/etc/bash_completion.d/?dveeden commentedon Jun 26, 2025
For golang-github-pingcap-errors:
go teston current masterdveeden commentedon Jun 26, 2025
The usage for this:
This summary mixes pingcap/errors with errors from the standard library.
dveeden commentedon Jul 11, 2025
Some updates / notes:
dveeden commentedon Aug 1, 2025
#2569 should help to get rid of grpc-gateway
ottok commentedon Aug 9, 2025
I pushed now tiup_1.16.2+20250802~f57cbe37 to my test PPA at https://launchpad.net/~otto/+archive/ubuntu/tidb. The package is only buildable for Ubuntu devel (questing) as it depends on new dependencies not yet in any actual release.
The package can be tested in e.g. a Ubuntu devel container with:
Unfortunately the tiup commands that depend on the real-time update check are segfaulting due to a bug I have in #2538, but the point here is now to demonstrate the build and dependency management.
Thanks to the latest changes on
master, I was able to remove these vendored dependencies:dveeden commentedon Aug 20, 2025
@ottok so these dependencies changed from vendored to non-vendored?
ottok commentedon Aug 20, 2025
No, they were removed completely.
Thanks for f6aa0ac too, now we can rely on yaml v3 alone.
dveeden commentedon Aug 20, 2025
Are you sure the list is correct? Some of these are still in the
go.modon the master branchottok commentedon Aug 22, 2025
You are right, sorry for getting mixed in up in dependency tracking. The only dependencies we got completely rid of was pingcap/tidb-insight and grpc-ecosystem/grpc-gateway.
While working on the latest git head of TiUP I came across a new finding about very outdated transitive dependency zaf/temp in #2588
dveeden commentedon Aug 25, 2025
Didn't we also get rid of
pingcap/checkand maybe some others?ottok commentedon Aug 26, 2025
Alright, so recent changes on TiUP git master branch helped get rid of: golang-github-alecthomas-assert-dev, golang-github-grpc-ecosystem-grpc-gateway-dev, golang-github-otiai10-copy-dev and golang-github-pingcap-check-dev.
Names are Debian package names and screenshot below shows diff between earlier Debian packaging prototype and latest version, which also has new dependencies after I've packaged them for Debian.
Latest proto version available in the TiUP PPA. Build logs at https://launchpad.net/~otto/+archive/ubuntu/tidb/+builds?build_text=&build_state=all show what went into it recently.
ottok commentedon Sep 10, 2025
FYI @dveeden I posted now #2602 and also non-native official packaging draft at https://salsa.debian.org/go-team/packages/tiup/-/merge_requests/1
ottok commentedon Feb 13, 2026
FYI: The Debian NEW queue process finally started working again after taken over by a new team, and TiUP got accepted into Debian, and by extension also Ubuntu:
https://tracker.debian.org/pkg/tiup
https://launchpad.net/ubuntu/+source/tiup
I plan to refresh my PRs in this repo, test Go 1.26 compatibility, update the TiUP version in Debian and do other maintenance tasks in coming weeks.
ottok commentedon Feb 28, 2026
I've now uploaded TiUP 1.16.4-1 to Debian unstable.