There are other retries (e.g. in git wrapper, gclient_scm) so we
shouldn't be retrying that much in gclient_utils.
In ideal case, we have a dedicated place where retries are determined,
and we make a retry decision based on stderr / exit code.
R=jojwang@google.com
Bug: 1359109
Change-Id: I97daa0d991a7294635e54b7a3d85a349c03c04c6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3871979
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Our Windows CI machine may take longer than 10s to process hooks
resulting in an extra block of information being printed. This change
updates the affected test to handle both fast and slow cases.
R=gavinmak@google.com
Fixed: 1352364
Change-Id: I62edbe49c0efc16cfed2bc218220840120c81c0a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3867544
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Vanilla git has a git-send-email(1) subcommand. `git cl upload` should
follow suit by using the same flag name to be consistent and avoid
confusion.
[1]: https://git-scm.com/docs/git-send-email
Bug: none
Change-Id: I47ff8e03bd11915e99804fe1695046d874efc4e0
Tested: `git cl upload --send-email` to upload this very own CL
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3854330
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Auto-Submit: Thiago Perrotta <tperrotta@chromium.org>
This change also updates version of git documentation to match one used in
depot_tools, asciidoc, and ansi2html. It also fixes reset signal.
R=gavinmak@google.com
Bug: 1357152
Change-Id: I998e7434dde8d82b491376b47a5993c6d91d64a8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3856830
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Change-Id: I7c05247729f4b569c9d6a95813d8849972c9b985
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3849003
Reviewed-by: Cindy Lin <xcl@google.com>
Commit-Queue: Cindy Lin <xcl@google.com>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Mike Frysinger <vapier@chromium.org>
This removes backslash from log like
```
Python 2 scripts were run during Python 2 presubmits. Please see https://bugs.chromium.org/p/chromium/issues/detail?id=1313804#c61 for tips on resolving this.
scripts/common/env.py print from /usr/local/google/home/tikuta/chrome_infra/build \
/usr/local/google/home/tikuta/chrome_infra/build/recipes/unittests/recipe_test.py from /usr/local/google/home/tikuta/chrome_infra/build \
-u /usr/local/google/home/tikuta/chrome_infra/build/recipes/.recipe_deps/recipe_engine/recipe_engine/main.py --package /usr/local/google/home/tikuta/chrome_infra/build/infra/config/recipes.cfg test run from /usr/local/google/home/tikuta/chrome_infra/build \
-u /usr/local/google/home/tikuta/chrome_infra/build/recipes/.recipe_deps/recipe_engine/recipe_engine/main.py --package /usr/local/google/home/tikuta/chrome_infra/build/infra/config/recipes.cfg --proto-override /usr/local/google/home/tikuta/chrome_infra/build/recipes/.recipe_deps/_pb --log-level ERROR -O depot_too
ls=/usr/local/google/home/tikuta/chrome_infra/build/recipes/.recipe_deps/depot_tools -O chromiumos_proto=/usr/local/google/home/tikuta/chrome_infra/build/recipes/.recipe_deps/chromiumos_proto -O infra=/usr/local/google/home/tikuta/chrome_infra/build/recipes/.recipe_deps/infra -O recipe_engine=/usr/local/google/home
/tikuta/chrome_infra/build/recipes/.recipe_deps/recipe_engine -O chromeos=/usr/local/google/home/tikuta/chrome_infra/build/recipes/.recipe_deps/chromeos -O chromiumos_config=/usr/local/google/home/tikuta/chrome_infra/build/recipes/.recipe_deps/chromiumos_config test _runner --cov-file /tmp/tmpqNhYvj.recipe_test_cov
erage/thread-0.coverage --cover-module-imports from /usr/local/google/home/tikuta/chrome_infra/build \
```
I think this backslash is bit confusing there.
Change-Id: If5911e27808cfcde029d5ea2b34dda06847085ff
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3827922
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
This improves output from ninja a bit on Windows when
NINJA_SUMMARIZE_BUILD is set.
The trailing space was removed in https://crrev.com/c/3400399.
Change-Id: Ifb14129a436a2673969f529d0c9143118dd87d6b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3833272
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
It's possible to change DEPS git repository while keeping the same
source. This is useful feature, for example if upstream repository moves
from github to gitlab. The repositories can be out of sync, or even
diverge.
We use git alternatives to save disk space. If a builder flip-flops
between mirror repositories (e.g. case with different release branches),
it's possible that new mirror doesn't contain all necessary objects that
are present in the old repositories. That will result in invalid git
pointer entries and fetch operations will fail.
Instead of replacing git alternatives, we append new git alternatives to
avoid such problem.
R=bpastene@google.com, jojwang@google.com
Bug: b/241802975
Change-Id: I3ef025603af58a84692a68bf3b780c47b6d3ae46
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3843133
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
On windows paths with spaces some cipd and depot_tools bootstrap commands were failing due to misquoting or syntax that didn't support such paths. This ensures that a clean bootstrap on a path such as "C:/Foo Bar/depot_tools" works as expected.
Change-Id: Ib516d16ed7809635369399f3df0eff2269ff6e2b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3834965
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
At the moment reclient isn't supported for developer builds. Add
messaging to autoninja to indicate this if a developer attempts a build
with use_remoteexec set.
Bug:b/199192766
Change-Id: I9e016a09c98b756018505f661afdb6eaf3db3243
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3839070
Commit-Queue: Michael Savigny <msavigny@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
This will be used in ChromeOS build_packages.
Bug: b/242680623, b/203606263
Change-Id: I3eedec7b245405c24091db43a450181a270d1305
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3833266
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
If a wrong command name is used (e.g. typo), cmdhelp is triggered.
However, if user specifies any additional arguments (other than -h),
then user is presented with not so useful message.
This CL calls cmdhelp with no arguments.
Fixed: 1352093
Change-Id: I30c0b5f580f18b4fa2cb59d3195b67bb083f3442
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3828794
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
The roll in depot_tools enables uploading Xcode in legacy format and
not excluding some folders at upload.
Bug: 1348212
Change-Id: I2a6b635a6c236bf2eedbbf7855da3c0d58b4abaa
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3834034
Reviewed-by: Chan Li <chanli@chromium.org>
Commit-Queue: Zhaoyang Li <zhaoyangli@chromium.org>
When running presubmits you can tell if Python 2 presubmits are running,
but it's subtle, and it isn't obvious which presubmits are running under
Python 2.
This makes it obvious by always printing the path to PRESUBMIT.py
scripts that are run under Python 2. This change also fixes a bug in the
--verbose logic - it would print that a script was running before
evaluating _ShouldRunPresubmit.
Bug: 1352347
Change-Id: I45213acdb3b37f7be0713e5caba775841704817c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3826917
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
This reverts commit ba352e00ba.
Reason for revert: Unfortunately this breaks this command when run
from a normal cmd.exe prompt:
autoninja ..\..\base\win\access_token.cc^^
The escaping of ^^ as it passes through batch files is very finicky, but important.
Original change's description:
> Update autoninja.bat to work with other shells.
>
> Other shells, such as TCC, have subtle differences in how batch files are processed. This will not break cmd.exe compatibility.
>
> Without the "call" command, running a nested batch file should, according to CMD.exe "rules", cancel the current batch file and run the new one. However, the 'FOR /f "usebackq"' command will call the batch file and return. One could argue for both approaches, however, adding the "call" command doesn't seem to affect cmd.exe and fixes the other shell.
>
> There is a similar issue with the "do" (in the FOR command) block and the use of the '&' separator. The '%a' variable isn't expanded, but adding the '()' block works in both instances.
>
> Bug: 1351817
> Change-Id: Idd685a64404ea950b71e1f3cc0f5d1e3bf13b8b6
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3824199
> Commit-Queue: Allen Bauer <kylixrd@chromium.org>
> Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Bug: 1351817
Change-Id: I9d204a3369c468664c0cff247ed1a53065dfebfd
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3829729
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
ListOwners returns all OWNERS for a file, which is undesirable in cases
when only direct OWNERS are needed.
Bug: 1351212, 1351519
Change-Id: I693b6645c780aa589e8ab24d0b58691f4aeb30f6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3823299
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Other shells, such as TCC, have subtle differences in how batch files are processed. This will not break cmd.exe compatibility.
Without the "call" command, running a nested batch file should, according to CMD.exe "rules", cancel the current batch file and run the new one. However, the 'FOR /f "usebackq"' command will call the batch file and return. One could argue for both approaches, however, adding the "call" command doesn't seem to affect cmd.exe and fixes the other shell.
There is a similar issue with the "do" (in the FOR command) block and the use of the '&' separator. The '%a' variable isn't expanded, but adding the '()' block works in both instances.
Bug: 1351817
Change-Id: Idd685a64404ea950b71e1f3cc0f5d1e3bf13b8b6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3824199
Commit-Queue: Allen Bauer <kylixrd@chromium.org>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Pylint uses parallelism to improve performance, but high startup costs
means that this only makes sense if there are lots of files to be
processed. So, a while ago a change was made such that if there are
fewer than ten files to be analyzed then no parallelism is used.
Our Pylint wrapper also has a hack where it does one type of check in
serial mode, because that is the only time it is reliable. This
requires running Pylint twice, which is expensive.
If there are a small enough number of files to analyze then we will be
doing serial analysis anyway, so there is no need to do two separate
runs. In this test case:
git cl presubmit -v --files tools\code_coverage\create_js_source_maps\test\create_js_source_maps_test.py
the cost of Pylint is dropped roughly in half, from six seconds to
three seconds, by eliminating one of the three-second runs.
Bug: 1309977
Change-Id: I2e5e96a86d1d76b127f481af7478d807c042b609
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3812436
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
This update is per http://crrev.com/1031130 where OSS licensing
requested removal of "All rights reserved" from new files.
Change-Id: I52a9deab99539ea90403798606c7bbeb261df994
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3811744
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Auto-Submit: Robert Liao <robliao@chromium.org>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
code-owners should have been enabled for most hosts that depot_tools
supports by now. Remove our own implementation and rely on code-owners.
Change-Id: Iaf0d3db65b2e5063b67d42b92188c4ec51d2cd9a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3783475
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Recent gerrit issues made it obvious that "git cl presubmit" relies on
gerrit much more than most people would expect (the expectation is zero
for many people). This makes presubmits flaky or much slower under poor
network conditions, and it means that the presubmit step may drastically
underestimate how long it takes to run because of a
cl.FetchDescription() that may occur outside of the timed portion of the
presubmits.
This change wraps more network-touching steps in try/except blocks, to
make them robust. It also gets them to check for the existence of a
PRESUBMIT_SKIP_NETWORK environment variable. And, it prints the elapsed
time to get the CL description if this is inordinately long.
Bug: 1350227
Change-Id: I7954fd50e928fd24975a4f61a316cb280542ebbd
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3813095
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
The try/except was added for one specific situation but the 'except' portion accepts a wide range of errors. This results in confusing logs and nothing clearly states what actually caused the error.
Bug: 1349905, 1350210
Change-Id: Ia86c71b0b77658992fca5d3de215e659ccd90d96
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3810798
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
This reverts commit a2267932e6.
Reason for revert: this version has an issue with led edit-recipe-bundle https://crbug.com/1350060
Original change's description:
> Update led to the current version.
>
> This will pull in the change that allows for "led edit-cr-crl" to accept
> URLs in the form output by "git cl issue": URLs that do not contain the
> gerrit project.
>
> Bug: 1348269
> Change-Id: Icb360686c9ca1548636e755bc9a611351d78d2f7
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3807651
> Commit-Queue: Garrett Beaty <gbeaty@google.com>
> Reviewed-by: Chan Li <chanli@chromium.org>
Bug: 1348269
Change-Id: Idad964cf47b0cbc7174201add38ab8799c1ae9b3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3812817
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
In case of PartitionAlloc project in Chromium, there is a case that
it's useful to declare 'noparent' in the DEPS due to the situation
that a sub directory is exported to a standalone git project.
//base/allocator/partition_allocator/ (sub directory) is exported to
https://chromium.googlesource.com/chromium/src/base/allocator/partition_allocator.git
Change-Id: Ic51b6cd3928002962f25620b4d8797bed7802d1e
Bug: chromium:1151236
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3802719
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Yuki Shiino <yukishiino@chromium.org>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
This will pull in the change that allows for "led edit-cr-crl" to accept
URLs in the form output by "git cl issue": URLs that do not contain the
gerrit project.
Bug: 1348269
Change-Id: Icb360686c9ca1548636e755bc9a611351d78d2f7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3807651
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Chan Li <chanli@chromium.org>
Gavin Mak
Josip Sokcevic
recipe-roller
Joanna Wang
Thiago Perrotta
Junji Watanabe
Garrett Beaty
Mike Frysinger
Takuto Ikuta
Samuel Attard
Michael Savigny
Zhaoyang Li
Yuanjun Huang
Bruce Dawson
Chan Li
Vadim Shtayura
Allen Bauer
Robert Liao
Brian Ryner
Yuki Shiino