Add helpers to gerrit_util for:
* Setting the commit message on a ChangeEdit,
* Creating a cherry-pick,
* Getting the contents of a file in a change.
Bug: v8:12849
Change-Id: I1a5de3054a366d480def32942fd42be179ffb749
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3683383
Auto-Submit: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
presubmit --all should succeed regardless of the current state of the
tree. Otherwise a presubmit --all bot will be flaky due to circumstances
beyond its control.
This is a logical follow-on to crrev.com/c/3628368 which turned some
other errors into warnings for presubmit --all.
Bug: 1309977
Change-Id: I6d26708f8c47916cf015dec75e27833db7ca465b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3696070
Auto-Submit: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
It's sometimes possible to end up with a depot_tools checkout that isn't
a git repository, which breaks the update mechanism. Rather than
allowing the updater to silently fail and have things break several
weeks/months later, we should emit a warning.
Bug: 1333997
Change-Id: I70ae4f3b479e24b87987847927184687caf08b33
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3692864
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Will Cassella <cassew@chromium.org>
Reviewed-by: Brian Sheedy <bsheedy@chromium.org>
gerrit_util.ChangeEdit was passing the result of b64encode into a %s
format, which was printing the contents with a bytes prefix (b'...').
Decode the bytes object back to a str object before passing it to '%s'
Bug: v8:12849
Change-Id: I8b54b427bbbe8b914444d7486b5a8afbd743de70
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3683382
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Auto-Submit: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
This reverts commit c90a982106.
Reason for revert:
https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket/8811949917908303825/+/u/compile/stdout
Original change's description:
> Switch to use CIPD ninja v1.8.2
>
> https://crrev.com/c/3674981 installed CIPD ninja packages.
> This CL switches from the old ninja binaries to them.
>
> Note that this CL doesn't change ninja version. the old/CIPD ninja binaries are v1.8.2.
>
> The differences from the old binaries are
> - the CIPD packages don't use chromium's clang.
> - the CIPD package for Linux doesn't use chromium's sysroot.
> - the min macOS version is 10.13, which was 10.6.
> Please also see https://crsrc.org/i/3pp/ninja/install_bootstrap.sh
>
>
> On Windows, ninja.exe will be installed under .cipd_bin.
> But there are many places that assume that ninja.exe exists on depot_tools root. So this CL also copies the ninja.exe from .cipd_bin to root in cipd_bin_setup.bat.
>
> Bug: 931218
> Change-Id: Ib67eee5e9b6ad9b2937b789626970d9c85867dbd
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3674982
> Commit-Queue: Junji Watanabe <jwata@google.com>
> Reviewed-by: Philipp Wollermann <philwo@google.com>
> Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
> Reviewed-by: Nico Weber <thakis@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Bug: 931218
Change-Id: I08d61dad119a0d23ae9ec8b1ed787dd3915de697
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3686854
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Owners-Override: Takuto Ikuta <tikuta@chromium.org>
https://crrev.com/c/3674981 installed CIPD ninja packages.
This CL switches from the old ninja binaries to them.
Note that this CL doesn't change ninja version. the old/CIPD ninja binaries are v1.8.2.
The differences from the old binaries are
- the CIPD packages don't use chromium's clang.
- the CIPD package for Linux doesn't use chromium's sysroot.
- the min macOS version is 10.13, which was 10.6.
Please also see https://crsrc.org/i/3pp/ninja/install_bootstrap.sh
On Windows, ninja.exe will be installed under .cipd_bin.
But there are many places that assume that ninja.exe exists on depot_tools root. So this CL also copies the ninja.exe from .cipd_bin to root in cipd_bin_setup.bat.
Bug: 931218
Change-Id: Ib67eee5e9b6ad9b2937b789626970d9c85867dbd
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3674982
Commit-Queue: Junji Watanabe <jwata@google.com>
Reviewed-by: Philipp Wollermann <philwo@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
This reverts commit 7c4f7ec408.
Reason for revert: bugs reported by users (see comments in Gerrit).
error: block. was unexpected at this time.
Original change's description:
> Make depot_tools scripts return exit codes properly on all platforms.
>
> Changes:
> 1. Windows: exit /b %errorlevel% should be used instead of goto :EOF to get valid exit codes during cmd /c <script>.bat invocation.
> 2. Windows: delayed var expansion is required in update_depot_tools.bat exit code generation.
> 3. Posix: update_depot_tools returns exit code from update_git_repo function in case of a failure.
>
> A rule of thumb on Windows: goto :EOF should not be used if %errorlevel% must be returned for all possible invocations.
>
> Test case for update_depot_tools changes:
> 1. Make a change to depot_tools sources that will conflict with next depot_tools update
> 2. Run update_depot_tools either directly or via gclient
> 3. Expect a git error is triggered
> 4. Inspect %errorlevel% or $? depending on platform
> 5. Expected 1, but the actual result is 0.
>
> Test case for changes in other .bat files:
> 1. Make a change to depot_tools sources that will conflict with next depot_tools update
> 2. Run cmd /c gclient
> 3. Expect a git error is triggered
> 4. Inspect %errorlevel%
> 5. Expected 1, but the actual result is 0.
>
> Change-Id: I64459982bcd9cc3db1319a9b39224b7a7af8c5aa
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3688632
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
> Auto-Submit: Aleksey Khoroshilov <akhoroshilov@brave.com>
> Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Change-Id: I85d598af01d75588cdee77165d6af22270ee031d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3694139
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
AFAIK, flash is long gone from Chromium.
Bug: 1064651
Change-Id: I25b98ac2b44e52fa3e3cfdbc9d7716ee736a4b6a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3687701
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Scott Violet <sky@chromium.org>
Auto-Submit: Victor Vianna <victorvianna@google.com>
This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.
The build that created this CL was
https://ci.chromium.org/b/8811985614603312065
recipe_engine:
51b3aed56d
51b3aed (jkusuma@google.com)
Update go.chromium.org/luci protos related to resultdb.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
R=iannucci@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Ignore-Freeze: Autoroller
Bugdroid-Send-Email: False
Change-Id: I60997a4ef19b45a3ce381ea885ba0932bf78f6d8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3694138
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Bot-Commit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Changes:
1. Windows: exit /b %errorlevel% should be used instead of goto :EOF to get valid exit codes during cmd /c <script>.bat invocation.
2. Windows: delayed var expansion is required in update_depot_tools.bat exit code generation.
3. Posix: update_depot_tools returns exit code from update_git_repo function in case of a failure.
A rule of thumb on Windows: goto :EOF should not be used if %errorlevel% must be returned for all possible invocations.
Test case for update_depot_tools changes:
1. Make a change to depot_tools sources that will conflict with next depot_tools update
2. Run update_depot_tools either directly or via gclient
3. Expect a git error is triggered
4. Inspect %errorlevel% or $? depending on platform
5. Expected 1, but the actual result is 0.
Test case for changes in other .bat files:
1. Make a change to depot_tools sources that will conflict with next depot_tools update
2. Run cmd /c gclient
3. Expect a git error is triggered
4. Inspect %errorlevel%
5. Expected 1, but the actual result is 0.
Change-Id: I64459982bcd9cc3db1319a9b39224b7a7af8c5aa
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3688632
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Auto-Submit: Aleksey Khoroshilov <akhoroshilov@brave.com>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.
The build that created this CL was
https://ci.chromium.org/b/8812059854989420705
recipe_engine:
92f7100ca3
92f7100 (iannucci@chromium.org)
[proto] Add ability to inject python files when scanning protos.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
R=iannucci@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Ignore-Freeze: Autoroller
Bugdroid-Send-Email: False
Change-Id: I11df85101f6262d6e3529db3598b2a294d170216
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3691023
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Bot-Commit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
A run of "git cl presubmit --all" shows that PyLint 1.5 is being run
from nine locations, however the deprecation warnings don't say where.
This makes it difficult to file bugs or fix the remaining instances.
This changes the message to list the path to the presubmit that is
running PyLint 1.5.
This also adds instructions on how to change to version 2.7.
Before:
pylint-1.5 is deprecated, please switch to 2.7 before 2022-07-11
After (with word wrapping):
pylint-1.5 is being run on ...src\tools\find_runtime_symbols
and is deprecated, please switch to 2.7 before 2022-07-11 (add
version='2.7' to RunPylint call)
Change-Id: Iece2cb904f5d26ad66e3ab78f7ce7aef1878bfd1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3688839
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
When a repository doesn't work with Gerrit, it doesn't make sense to
create GerritAccessor and pass it to presubmit checks.
gerrit.host is stored into git config when codereview.settings file contains GERRIT_HOST item.
Change-Id: I9740950caf85a1da9a6e4ae12e3612625cf0cec5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3687235
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Auto-Submit: Aleksey Khoroshilov <akhoroshilov@brave.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
When PRESUBMIT is run with a custom --upstream, dry run cl format should
also use this value, otherwise format warnings would be inconsistent.
Change-Id: I1370f614618ecc03706e54cd4b151a207d9b3bae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3688631
Auto-Submit: Aleksey Khoroshilov <akhoroshilov@brave.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Currently, the ninja binaries in depot_tools are committed to the repo directly.
This CL installs CIPD ninja packages to replace them.
Switching to CIPD ninja will be done in the next CL.
Also, see the performance comparison and CIPD packages in the comments below.
https://crbug.com/931218#c84https://crbug.com/931218#c86
Bug: 931218
Change-Id: I44d5e9ea1447588b32a10db20cfba088dd748f48
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3674981
Commit-Queue: Junji Watanabe <jwata@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
This CL makes fetch default to sso:// within the cog env. It also adds a helper within gclient_scm to check if the env is cog.
Bug: chrome-operations:170
Change-Id: I8c35348653406549fadbd08df3d9991bb24ca776
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3670721
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Remove unnecessary os.listdir call and fix infinite loop on Windows if
no file was found in a tree.
Change-Id: I82a8763e807bbc0ce6fcae6b35a370ffe3b34943
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3687234
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Auto-Submit: Aleksey Khoroshilov <akhoroshilov@brave.com>
This reverts commit 013e47be09.
Reason for revert: This will be relanded as a warning, although the
bugs which the original change exposed should be fixed soon.
Original change's description:
> Revert "Fix not-run-test detection."
>
> This reverts commit 41691abe86.
>
> Reason for revert: I thought https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3683378 would solve the problem but turns out this is the issue where it now reports it as a critical error.
>
> Original change's description:
> > Fix not-run-test detection.
> >
> > In crrev.com/c/2986400 the skip_shebang_check flag was added to address
> > the problem of some tests being run neither on Python 2 or Python 3. At
> > the same time reporting was added to warn if tests were run on neither
> > Python version.
> >
> > Unfortunately the reporting code was incorrect. A PresubmitPromptWarning
> > object was created but was not added to the list of results, so it had
> > no effect. Also the skipped tests name was added where a list was
> > expected which meant that each character of the test name showed up on
> > its own line.
> >
> > This change fixes the reporting. It also changes the format of the
> > report and makes it an error - a failure to run tests at all deserves
> > that severity level.
> >
> > A test run with the fixed check showed that no errors have crept in -
> > all tests were being run with one or the other. However the fix did
> > find a bug in an in-progress change I was working on.
> >
> > Bug: 1223478
> > Change-Id: Ibb44b5e60e7a7a5de08302f19ee4035cdfac5212
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3674199
> > Auto-Submit: Bruce Dawson <brucedawson@chromium.org>
> > Commit-Queue: Fabrice de Gans <fdegans@chromium.org>
> > Reviewed-by: Fabrice de Gans <fdegans@chromium.org>
>
> Bug: 1223478
> Change-Id: If4395e90784c3f91692f398b4e5230a51139f78a
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3683105
> Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Bug: 1223478, 1330859
Change-Id: I6a0a78e70db77c9c5fda5bab27960cd4e71279e5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3685349
Reviewed-by: Fabrice de Gans <fdegans@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
This reverts commit c03ebf0e1a.
Reason for revert: the original change was not the cause of the test
failures. The actual cause was crrev.com/c/3683105 which was reverted.
Original change's description:
> Revert "Add Python 3 support to GetPythonUnitTests"
>
> This reverts commit dfc71bbe01.
>
> Reason for revert: Breaks all presubmits that do not have skip_shebang_check=True set.
>
> Original change's description:
> > Add Python 3 support to GetPythonUnitTests
> >
> > GetPythonUnitTests (and by extension RunPythonUnitTests) always ran
> > scripts using Python 2. This change adds a python3=False parameter to
> > GetPythonUnitTests to give the option of switching to Python 3, while
> > leaving the default as Python 2. The child scripts are run under one or
> > the other, based on this parameter.
> >
> > This change is needed in support of crrev.com/c/3679801.
> >
> > Bug: 1313804
> > Change-Id: Ic59287352d4941707adaf7981ed7af4201b7d526
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3680099
> > Reviewed-by: Jesse McKenna <jessemckenna@google.com>
> > Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
>
> Bug: 1313804
> Change-Id: I1bd7096b6cfdfed6205e6dc0b5d4498b5b821b97
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3683378
> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> Commit-Queue: Christoffer Jansson <jansson@chromium.org>
> Owners-Override: Ben Pastene <bpastene@chromium.org>
> Reviewed-by: Tomas Gunnarsson <tommi@chromium.org>
Bug: 1313804
Change-Id: I15ed90cc35d88ece71eb681acc0acc06cb5c3bcc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3685229
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Jesse McKenna <jessemckenna@google.com>
This is a reland of commit 6817010e83
Additional Changes:
This reland fixes https://crbug.com/1330995 by stopping gclient from accidentally updating the scheme when one is not present.
Original change's description:
> Update fetch protocol using --protocol-override flag
>
> This CL updates gclient sync to use the protocol of the URL specified in the solutions for cloning all the child dependencies of it.
>
> Bug: chrome-operations:170
> Change-Id: I33588059788b677fbae8c3b434100af5c7979a67
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3631600
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Bug: 1330995
Change-Id: I1447a5e884e41d671d8556c35193f1635f2f6936
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3684112
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Currently cipd packages listed in src/DEPS are being created and
installed in symlink mode (see bug for more info). This CL adds `$OverrideInstallMode copy` to the ensure files for cipd packages installed during gclient runhooks. `$OverrideInstallMode` was
implemented here, https://chromium-review.googlesource.com/c/infra/luci/luci-go/+/3124202,
so that every cipd package can be installed directly to the installation
target, instead of being symlinked to an executable inside the cache/builder/.cipd dir.
The added `$OverrideInstallMode copy` can be removed once all cipd
packages themselves or the default mode is updated to be created in
copy mode.
Bug: 1329641
Change-Id: I68dc718e20f193c384d005c9b6b87603753943d4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3673698
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
This reverts commit 6817010e83.
Reason for revert: Breaks gclient sync for flutter
Original change's description:
> Update fetch protocol using --protocol-override flag
>
> This CL updates gclient sync to use the protocol of the URL specified in the solutions for cloning all the child dependencies of it.
>
> Bug: chrome-operations:170
> Change-Id: I33588059788b677fbae8c3b434100af5c7979a67
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3631600
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Bug: chrome-operations:170
Change-Id: I76a49030f48fa266eeea336307e7153b22ded4e9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3684110
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Should be landed once the new cipd is rolled out everywhere (including
bots)
Bug: 1329641
Change-Id: I32edaa167dd26dbf891435215eeb7c833aba091f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3680100
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
This reverts commit 41691abe86.
Reason for revert: I thought https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3683378 would solve the problem but turns out this is the issue where it now reports it as a critical error.
Original change's description:
> Fix not-run-test detection.
>
> In crrev.com/c/2986400 the skip_shebang_check flag was added to address
> the problem of some tests being run neither on Python 2 or Python 3. At
> the same time reporting was added to warn if tests were run on neither
> Python version.
>
> Unfortunately the reporting code was incorrect. A PresubmitPromptWarning
> object was created but was not added to the list of results, so it had
> no effect. Also the skipped tests name was added where a list was
> expected which meant that each character of the test name showed up on
> its own line.
>
> This change fixes the reporting. It also changes the format of the
> report and makes it an error - a failure to run tests at all deserves
> that severity level.
>
> A test run with the fixed check showed that no errors have crept in -
> all tests were being run with one or the other. However the fix did
> find a bug in an in-progress change I was working on.
>
> Bug: 1223478
> Change-Id: Ibb44b5e60e7a7a5de08302f19ee4035cdfac5212
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3674199
> Auto-Submit: Bruce Dawson <brucedawson@chromium.org>
> Commit-Queue: Fabrice de Gans <fdegans@chromium.org>
> Reviewed-by: Fabrice de Gans <fdegans@chromium.org>
Bug: 1223478
Change-Id: If4395e90784c3f91692f398b4e5230a51139f78a
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3683105
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
This CL updates gclient sync to use the protocol of the URL specified in the solutions for cloning all the child dependencies of it.
Bug: chrome-operations:170
Change-Id: I33588059788b677fbae8c3b434100af5c7979a67
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3631600
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
This reverts commit dfc71bbe01.
Reason for revert: Breaks all presubmits that do not have skip_shebang_check=True set.
Original change's description:
> Add Python 3 support to GetPythonUnitTests
>
> GetPythonUnitTests (and by extension RunPythonUnitTests) always ran
> scripts using Python 2. This change adds a python3=False parameter to
> GetPythonUnitTests to give the option of switching to Python 3, while
> leaving the default as Python 2. The child scripts are run under one or
> the other, based on this parameter.
>
> This change is needed in support of crrev.com/c/3679801.
>
> Bug: 1313804
> Change-Id: Ic59287352d4941707adaf7981ed7af4201b7d526
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3680099
> Reviewed-by: Jesse McKenna <jessemckenna@google.com>
> Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Bug: 1313804
Change-Id: I1bd7096b6cfdfed6205e6dc0b5d4498b5b821b97
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3683378
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Christoffer Jansson <jansson@chromium.org>
Owners-Override: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Tomas Gunnarsson <tommi@chromium.org>
GetPythonUnitTests (and by extension RunPythonUnitTests) always ran
scripts using Python 2. This change adds a python3=False parameter to
GetPythonUnitTests to give the option of switching to Python 3, while
leaving the default as Python 2. The child scripts are run under one or
the other, based on this parameter.
This change is needed in support of crrev.com/c/3679801.
Bug: 1313804
Change-Id: Ic59287352d4941707adaf7981ed7af4201b7d526
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3680099
Reviewed-by: Jesse McKenna <jessemckenna@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
In crrev.com/c/2986400 the skip_shebang_check flag was added to address
the problem of some tests being run neither on Python 2 or Python 3. At
the same time reporting was added to warn if tests were run on neither
Python version.
Unfortunately the reporting code was incorrect. A PresubmitPromptWarning
object was created but was not added to the list of results, so it had
no effect. Also the skipped tests name was added where a list was
expected which meant that each character of the test name showed up on
its own line.
This change fixes the reporting. It also changes the format of the
report and makes it an error - a failure to run tests at all deserves
that severity level.
A test run with the fixed check showed that no errors have crept in -
all tests were being run with one or the other. However the fix did
find a bug in an in-progress change I was working on.
Bug: 1223478
Change-Id: Ibb44b5e60e7a7a5de08302f19ee4035cdfac5212
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3674199
Auto-Submit: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Fabrice de Gans <fdegans@chromium.org>
Reviewed-by: Fabrice de Gans <fdegans@chromium.org>
Some presubmit failures result in uncaught exceptions which historically
have halted presubmit runs. This behavior makes it more difficult to
find all failures, measure presubmit times, or detect other behavior
over time. This is particularly problematic when running "git cl
presubmit --all".
As an example, presubmit --all was broken and would not complete from
when crrev.com/3642633 landed until crrev.com/c/3657600. The minimal
repro was this presubmit command:
git cl presubmit --files ui\accessibility\ax_mode.h
This command, or presubmit --all, triggered this error, which halts the
presubmits:
Evaluation of CheckChangeOnCommit failed: [Errno 2] No such file or directory: 'chrome/browser/resources/accessibility/accessibility.js'
Traceback (most recent call last):
File "presubmit_support.py", line 2038, in <module>
sys.exit(main())
File "presubmit_support.py", line 2015, in main
return DoPresubmitChecks(
File "presubmit_support.py", line 1738, in DoPresubmitChecks
results += executer.ExecPresubmitScript(presubmit_script, filename)
File "presubmit_support.py", line 1602, in ExecPresubmitScript
self._run_check_function(function_name, context, sink,
File "presubmit_support.py", line 1640, in _run_check_function
six.reraise(e_type, e_value, e_tb)
File "C:\Users\brucedawson\.vpython-root\e726d2\lib\site-packages\six.py", line 686, in reraise
raise value
File "presubmit_support.py", line 1630, in _run_check_function
result = eval(function_name + '(*__args)', context)
File "<string>", line 1, in <module>
File "PRESUBMIT.py", line 314, in CheckChangeOnCommit
errs.extend(CheckModesMatch(input_api, output_api))
File "PRESUBMIT.py", line 283, in CheckModesMatch
ax_modes_in_js = GetAccessibilityModesFromFile(
File "PRESUBMIT.py", line 245, in GetAccessibilityModesFromFile
for line in open(fullpath).readlines():
FileNotFoundError: [Errno 2] No such file or directory: 'chrome/browser/resources/accessibility/accessibility.js'
With this change the exception is handled and presubmits continue after
printing this message:
Evaluation of CheckChangeOnCommit failed: [Errno 2] No such file or directory: 'chrome/browser/resources/accessibility/accessibility.js', Traceback (most recent call last):
File "presubmit_support.py", line 1630, in _run_check_function
result = eval(function_name + '(*__args)', context)
File "<string>", line 1, in <module>
File "PRESUBMIT.py", line 314, in CheckChangeOnCommit
errs.extend(CheckModesMatch(input_api, output_api))
File "PRESUBMIT.py", line 283, in CheckModesMatch
ax_modes_in_js = GetAccessibilityModesFromFile(
File "PRESUBMIT.py", line 245, in GetAccessibilityModesFromFile
for line in open(fullpath).readlines():
FileNotFoundError: [Errno 2] No such file or directory: 'chrome/browser/resources/accessibility/accessibility.js'
That is, the crucial information regarding the failure is printed, but
subsequent presubmits are not skipped.
Bug: 1309977
Change-Id: I7cfeda85c830e6c8e567c0df3c50f27af1dbe835
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3653978
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
A typical cpplint error report during a presubmit looks like this:
.../linux_ui.cc:64: Almost always, snprintf is better than strcpy [runtime/printf] [4]
...
Changelist failed cpplint.py check.
The problem is that the details of the error go to stderr when
CheckChangeLintsClean is run, whereas the "Changelist failed cpplint.py
check." message is printed at the end of the run. If there a lot of
errors, warnings, or messages then the association between them can be
completely lost.
This change adds the tag (cpplint) to the error messages, and adds a
comment to the "failed cpplint.py check" message to suggest looking for
the tag.
Bug: 1309977
Change-Id: I0d073b57f215e28495cbc3e009cab6ac2f23b65e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3632947
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Some of the expensive checks when running presubmit --all, such as
CheckStableMojomChanges (~300 s) and CheckAddedDepsHaveTargetApprovals
(~200 s) only look at diffs and are therefore guaranteed to be NOPs when
running presubmit --all or --files=. Passing along the no_diffs state
lets these expensive checks be skipped, thus allowing for faster
iteration times.
Initial testing suggests that (with some supporting changes in the
Chromium repo) this reduces "presubmit --all" times by about ten
minutes, or a bit more than 10%, and additional improvements may be
possible.
Special handling for the no-diffs case also offers a simple way to avoid
presubmit failures that happen whenever all files are flagged as being
changed.
Finally, and perhaps most importantly for having a presubmit --all bot,
when --no_diffs is passed we can treat errors like "Issue wasn't
uploaded" and "Add a description to the CL" as messages, thus making it
possible to have zero presubmit errors when run on origin/main.
Bug: 1320937, 1322936
Change-Id: I0d09dd4aae8fdaa48c8b2f89337441cf96dcff72
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3628368
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Presubmit message types were printed in an inconsistent order, generally
determined by which type was encountered first. This unpredictability
can be confusing when comparing runs. This change enforces a consistent
order, in increasing order of severity. ERRORS always go last so that
they will be the most visible when running presubmits locally.
This also adds a "There were Python %d presubmit errors." message to
the end to make it easier to tell when there were presubmit errors.
This is particularly useful when manually running presubmits.
Bug: 1309977
Change-Id: Ib2b73c7625789bad5b21ae12abf238b746cd11e1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3668724
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
In https://crbug.com/1327371, we started encountering hanging gsutil
calls. These were triggered under very specific circumstances, but
were essentially due to a multi-processing bug in python on MacOS:
https://bugs.python.org/issue33725
When running gsutil on mac, it explicitly suggests disabling
multi-processing for this reason:
https://source.chromium.org/chromium/chromium/src/+/main:third_party/catapult/third_party/gsutil/gslib/command.py;drc=3a12d6ccdec28da8bda09d9ff826aae1f9504e59;l=1331
So this CL simply puts that suggestion into practice. There are two
options for doing so: adding a line to an active Boto file, or adding
a "-o" arg to every gsutil invocation. Since chromium has multiple
copies of gsutil and multiple different locations where `gsutil` is
invoked, the latter option is considered intractable.
Instead, we choose the Boto file option by adding a context manager
to the gsutil recipe module that will insert a custom Boto file into
the environment that only disables multi-threading. Due to the fact
that the BOTO_PATH env var takes multiple paths, we can safely
incorporate both our custom Boto along with LUCI's per-build Boto file
in the same gsutil invocation. And when the context manager exits, the
environment should revert back to its original state.
Chromium incorporates this method in crrev.com/c/3662023. See it take
effect during the "gclient runhooks" step in this led build:
https://ci.chromium.org/swarming/task/5b0c5174e70cb010?server=chromium-swarm.appspot.com
Bug: 1327371
Change-Id: I75cfdf16d0ec023bf81ea57991dde996694a46c6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3661949
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
It's possible that remote closes connection, which would result in a
stacktrace being shown in the stderr. Silence this exception, as we do
with other exceptions.
R=gavinmak@google.com
Bug: 1328518
Change-Id: Ifae2aca64fe5d572c0b71008f6dacbb63ba77e06
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3664178
Reviewed-by: Gavin Mak <gavinmak@google.com>
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Leszek Swirski
Bruce Dawson
Rebecca Chang Swee Fun
Will Cassella
Junji Watanabe
Takuto Ikuta
Josip Sokcevic
Ryan Heise
Victor Vianna
recipe-roller
Aleksey Khoroshilov
Fumitoshi Ukai
Joanna Wang
Aravind Vasudevan
Stephanie Kim
Christoffer Jansson
Ben Pastene