It now fetches and installs packages in parallel. This has a side
effect that the order of unzipping of packages in a manifest is
non-deterministic and depends on when particular packages finish
downloading.
To be able to show progress of parallel operations there's also
a new terminal UI with progress bars. It activates when running
CIPD at `info` logging level (default) if the stderr is attached
to a terminal (i.e. not piping to a file or the recipe engine).
Two new env vars control the new behavior:
* CIPD_PARALLEL_DOWNLOADS: set to 0 to completely disable
parallelism and restore pre 2.6.0 behavior. Set to 1 to fetch
and unzip packages in paralle, but fetch at most one package
at once. Default is 4.
* CIPD_SIMPLE_TERMINAL_UI: set to 1 to use log-based UI instead
of progress bars.
R=iannucci@chromium.org
BUG=1217435
Change-Id: I16815dbc397137fbb8ef09e09d0a04c77660c0c6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2993750
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
All packages except mac-amd64 ones are built with go1.16. mac-amd64
are built with go1.15, to preserve OSX 10.10/10.11 compatibility.
Note that we still use mac-amd64 even on arm64 OSX, since
the bootstrap script still installs mac-amd64 version of CIPD.
R=iannucci@chromium.org, tandrii@chromium.org
BUG=1102967
Change-Id: I8fdb27397fbbf238fe39e81305d4642a0500ab33
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2699690
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
It was built with go1.15 which has a bug in deflate implementation
which may result in corruption when uploading >2GB files.
R=iannucci@chromium.org
BUG=1129704
Change-Id: Id61eaaed5dd18d8e51ac5150a9d29a5825805a17
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2418899
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Includes bug fix for crbug.com/936911 and no other changes.
R=tandrii@chromium.org, iannucci@chromium.org
BUG=936911
Change-Id: I92e6725dc16b513579db87b81e389f9d61bdf62b
Reviewed-on: https://chromium-review.googlesource.com/c/1496034
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Includes a bug fix for incorrect handling of subcommands with typos in them. No
other changes.
BUG=931755
R=iannucci@chromium.org
Change-Id: I4a3c523452939bdb5a6f26637f1f7b4a2eeb4b0c
Reviewed-on: https://chromium-review.googlesource.com/c/1474373
Auto-Submit: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
The most notable change is in how the integrity of the tag cache file is
checked. Once deployed, existing tag cache files will be considered invalid and
wiped.
The only observable side effect is the following line in the log:
"can't deserialize tag cache - no sha256 is recorded in the file"
R=tandrii@chromium.org, iannucci@chromium.org
Change-Id: I7ea300a4e7ad8be0c1d42ae561c5202420d2db62
Reviewed-on: https://chromium-review.googlesource.com/c/1327823
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
The new version uses SHA256 for uploads by default. It also fully supports
$ResolvedVersions and 'selfupdate-roll'.
The roll was generated by:
$ ./cipd selfupdate-roll \
-version-file cipd_client_version \
-version git_revision:fb963f0f43e265a65fb7f1f202e17ea23e947063
BUG=821194, 870166
R=nodir@chromium.org, iannucci@chromium.org
Change-Id: If7c23c43ff2c1bf50c9c50f4e9e17a23e84859b9
Reviewed-on: https://chromium-review.googlesource.com/1227431
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Linux and OSX only for now. This also rolls CIPD client to a version that
supports pinned hashes (v2.2.5).
CIPD_CLIENT_VER and CIPD_CLIENT_SRV are no longer supported as env vars, since
it makes no sense when pinning hashes of the binaries at specific version on
the specific backend.
Also somewhat cleanup 'cipd' script to use "${VAR}", stderr and colored output
consistently.
R=iannucci@chromium.org, nodir@chromium.org
BUG=870166, 874586
Change-Id: Iac67fbb6b5d07dcd81d44536737b03b146f1ad14
Reviewed-on: https://chromium-review.googlesource.com/1176727
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
This reverts commit eebc3d8232.
Reason for revert: crbug.com/874586
Original change's description:
> [cipd] Check CIPD client hash against pinned SHA256 during updates.
>
> Linux and OSX only for now. This also rolls CIPD client to a version that
> supports pinned hashes (v2.2.5).
>
> CIPD_CLIENT_VER and CIPD_CLIENT_SRV are no longer supported as env vars, since
> it makes no sense when pinning hashes of the binaries at specific version on
> the specific backend.
>
> Also somewhat cleanup 'cipd' script to use "${VAR}", stderr and colored output
> consistently.
>
> R=iannucci@chromium.org, nodir@chromium.org
> BUG=870166
>
> Change-Id: I9e61f9f8fbdcf10985c52828b2bfbec64b4234f0
> Reviewed-on: https://chromium-review.googlesource.com/1171957
> Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
> Reviewed-by: Nodir Turakulov <nodir@chromium.org>
TBR=iannucci@chromium.org,vadimsh@chromium.org,nodir@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 870166
Change-Id: I9aa8e7a7f07520aa69d366c76e4dbccae345bc00
Reviewed-on: https://chromium-review.googlesource.com/1175294
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Linux and OSX only for now. This also rolls CIPD client to a version that
supports pinned hashes (v2.2.5).
CIPD_CLIENT_VER and CIPD_CLIENT_SRV are no longer supported as env vars, since
it makes no sense when pinning hashes of the binaries at specific version on
the specific backend.
Also somewhat cleanup 'cipd' script to use "${VAR}", stderr and colored output
consistently.
R=iannucci@chromium.org, nodir@chromium.org
BUG=870166
Change-Id: I9e61f9f8fbdcf10985c52828b2bfbec64b4234f0
Reviewed-on: https://chromium-review.googlesource.com/1171957
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Vadim Shtayura
Matt Kotsenas
Robert Iannucci
Andrii Shyshkalov