This reverts commit db0dcbbe7c.
Reason for revert: `gclient sync` no longer works on my local checkout with this change due to the error `No such file or directory: 'vpython'.`.
Original change's description:
> Delete vpython, leaving only vpython3.
>
> Bug: 1376538
> Change-Id: Ia7795f75fd05db63f155c5feffcc43822fb45bb1
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5187892
> Reviewed-by: Mike Frysinger <vapier@chromium.org>
> Commit-Queue: Brian Ryner <bryner@google.com>
> Reviewed-by: George Engelbrecht <engeg@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
> Reviewed-by: Chenlin Fan <fancl@chromium.org>
Bug: 1376538
Change-Id: I7c32eea1f8ae07644ab81a35213de5c68c64dfb6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5260042
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Andy Perelson <ajp@chromium.org>
Commit-Queue: Brian Ryner <bryner@google.com>
Reviewed-by: Brian Ryner <bryner@google.com>
+ Change scm GetAllFiles back to returning all files
+ should be safe to do since it's only used by presubmit code
+ add ignore_submodules option to CaptureStatus
+ It's used by other parts of depot_tools code
+ Add AffectedSubmodules to return submodules only
+ AffectedFiles still only returns non-submodule files.
Bug: 1475770
Change-Id: I457ad96099fb20eff4cc1a4fc84f59e7ae707abe
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5216005
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Change-Id: Ia37291aca59b429b59bdc6424a2f36394d558e3f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5214265
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Assigning os.environ the result of os.environ.copy() replaces it
with a standard dict. This breaks reflection into the actual
environment, as well as features such as case-insensitivity on
Windows, leading to undefined standard library behavior.
R=bryner, gavinmak
Bug: 1511316
Change-Id: I63d026387104ae92669256567854f8bf9d2397a2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5133887
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Brian Ryner <bryner@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
All __future__ imports (unicode_literals, print_function) are already
mandatory in py3. Also remove an outdated py2 comment in
presubmit_canned_checks.py
Bug: 1475402
Change-Id: I27cf6a8268f6dd1081f22af782c4c29a975376ae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4867135
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Leave the recipes/ code at 2 space to match the rest of the recipes
project in other repos.
Reformatted using:
files=( $(
git ls-tree -r --name-only HEAD | \
grep -Ev -e '^(third_party|recipes)/' | \
grep '\.py$';
git grep -l '#!/usr/bin/env.*python' | grep -v '\.py$'
) )
parallel ./yapf -i -- "${files[@]}"
~/chromiumos/chromite/contrib/reflow_overlong_comments "${files[@]}"
The files that still had strings that were too long were manually
reformatted because they were easy and only a few issues.
autoninja.py
clang_format.py
download_from_google_storage.py
fix_encoding.py
gclient_utils.py
git_cache.py
git_common.py
git_map_branches.py
git_reparent_branch.py
gn.py
my_activity.py
owners_finder.py
presubmit_canned_checks.py
reclient_helper.py
reclientreport.py
roll_dep.py
rustfmt.py
siso.py
split_cl.py
subcommand.py
subprocess2.py
swift_format.py
upload_to_google_storage.py
These files still had lines (strings) that were too long, so the pylint
warnings were suppressed with a TODO.
auth.py
gclient.py
gclient_eval.py
gclient_paths.py
gclient_scm.py
gerrit_util.py
git_cl.py
presubmit_canned_checks.py
presubmit_support.py
scm.py
Change-Id: Ia6535c4f2c48d46b589ec1e791dde6c6b2ea858f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4836379
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
The new flow is that if any affected file is in the config directory
of a config set, the script will call `lucicfg validate` to validate
the whole config directory and then filter out the validation result
that does not belong to affected file(s). This will introduce a bit
more latency in valdiation because before, only affected files are
sent to LUCI Config for validation. However, this is not latency
senstively operation and always validating the full snapshot will
faciliate us breaking down the config as it's possible in order to
validate the affected file, LUCI Config will need an unaffected file.
R=iannucci, yuanjunh
Bug: 1449933
Change-Id: Id330973f59b882569a8ece36edb4b6a8ff0ed2d0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4591233
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Yuanjun Huang <yuanjunh@google.com>
Calling CheckChangeHasNoUnwantedTags and
CheckChangeHasBugFieldFromChange from a PostUploadHook is useful because
it ensures that users will see a message if they add a bad tag when
creating the commit message during upload. Allowing this requires
creating entry points for these functions that take a change parameter
instead of an input_api parameter. It also requires fixing our use of
the deprecated inspect.getargspec function.
In order to not tell users twice (once before and once after upload)
that they haven't used a Bug: tag (it is optional, after all) a flag
to make these suggestions optional is also added.
This change is required in support of crrev.com/c/4490087.
Bug: 1434702
Change-Id: I3fd909a71c6c7d7671d6f986ad60d5375143a2f1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4519532
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
PRESUBMIT.py files can invoke other scripts, and those scripts may be
run under Python 2 or Python 3. There are multiple mechanisms that
govern which version will be used, sometimes requiring several flags
to be passed.
This change removes support for Python 2 from this system, thus making
it simpler to invoke Python 3 scripts.
The function parameters that are used to select Python versions are
passed in from multiple places so they still need to be supported, but
they are now ignored. The parameters are deleted to prevent accidental
use.
This change was tested by running this command in a Chromium repo:
git cl presubmit --force --all
Bug: 1207012
Change-Id: I4adc7164417e155ff80d3a039cf36ed030756456
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4328470
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
When using async jobs, information about presubmit is not present in
callstack and it's just noise.
Change-Id: Ib95af92f9e83f1fb8b8d68c7aece6bbb4b03e61c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4505697
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
This changes all references to the LUCI_OMIT_PYTHON2 environment
variable to instead assume that it is `'true'`.
R=sokcevic, tikuta
Bug: 1441784
Recipe-Nontrivial-Roll: build
Change-Id: Iea161a372adb68cdcb330d131df2c19ca2fe7b37
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4522480
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
This reverts commit 119cff3d2f.
Reason for revert: some builders still call py2 presubmit
http://go/bbid/8781466817788761185
We need to clean up recipe first before relanding this
Original change's description:
> Reland "Remove Python 2 support for PRESUBMIT.py"
>
> This reverts commit d4c6cbeb61.
>
> Reason for revert: Python 2 invocations of the presubmit system
> should be gone now.
>
> Original change's description:
> > Revert "Remove Python 2 support for PRESUBMIT.py"
> >
> > This reverts commit 8454fc2458.
> >
> > Reason for revert: post submit hooks failing
> > https://crbug.com/1422416
> >
> > Original change's description:
> > > Remove Python 2 support for PRESUBMIT.py
> > >
> > > The presubmit system still supports invoking PRESUBMIT.py files using
> > > Python 2. This has recently been turned off on the bots so this change
> > > removes support more completely.
> > >
> > > There are still some python3 parameters being passed around - it seemed
> > > better to do the simplest possible removal now, with a follow-up change
> > > to remove more support code after this has sat for a while.
> > >
> > > Tests run from PRESUBMIT.py files could still be run using Python 2, but
> > > those should also have been addressed already. Removing support for that
> > > will be done in a subsequent change.
> > >
> > > Bug: 1207012
> > > Change-Id: Id244d547a04438f83734dba269c3cc180c148b37
> > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4315183
> > > Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> > > Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
> >
> > Bug: 1207012
> > Bug: 1422416
> > Change-Id: Iaf3102e63ec3c698d0258fac5746dbd92c30edbb
> > No-Presubmit: true
> > No-Tree-Checks: true
> > No-Try: true
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4317176
> > Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
> > Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> > Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
>
> Bug: 1207012
> Bug: 1422416
> Change-Id: I9521095989c708188fca0251fa13881e086b1395
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4516015
> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Bug: 1207012
Bug: 1422416
Change-Id: Ifa775ca19f8c7b3c166eb89a255c25c20aaf4a98
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4521578
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
This reverts commit d4c6cbeb61.
Reason for revert: Python 2 invocations of the presubmit system
should be gone now.
Original change's description:
> Revert "Remove Python 2 support for PRESUBMIT.py"
>
> This reverts commit 8454fc2458.
>
> Reason for revert: post submit hooks failing
> https://crbug.com/1422416
>
> Original change's description:
> > Remove Python 2 support for PRESUBMIT.py
> >
> > The presubmit system still supports invoking PRESUBMIT.py files using
> > Python 2. This has recently been turned off on the bots so this change
> > removes support more completely.
> >
> > There are still some python3 parameters being passed around - it seemed
> > better to do the simplest possible removal now, with a follow-up change
> > to remove more support code after this has sat for a while.
> >
> > Tests run from PRESUBMIT.py files could still be run using Python 2, but
> > those should also have been addressed already. Removing support for that
> > will be done in a subsequent change.
> >
> > Bug: 1207012
> > Change-Id: Id244d547a04438f83734dba269c3cc180c148b37
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4315183
> > Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> > Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
>
> Bug: 1207012
> Bug: 1422416
> Change-Id: Iaf3102e63ec3c698d0258fac5746dbd92c30edbb
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4317176
> Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bug: 1207012
Bug: 1422416
Change-Id: I9521095989c708188fca0251fa13881e086b1395
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4516015
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
This reverts commit 512fd3bd85.
Reason for revert: breaks some presubmits
https://crbug.com/1421441
Original change's description:
> Extract searching relevat files in utils function
>
> This change introduces lib/ directory which will be used for classes
> that are considered libraries and used by multiple executables.
>
> Change-Id: I3da126778bf5ffdb28d7a3c2b966c85de08ba717
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4289679
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
> Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Bug: 1421441
Change-Id: I20e40f452a544a1188a258131a620e2afcd64f13
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4328468
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
This reverts commit e9ece0f581.
Reason for revert: broke infra presubmits:
https://ci.chromium.org/p/infra/builders/try/infra-try-presubmit
Original change's description:
> Reland "Remove Python 2 support for PRESUBMIT.py"
>
> This is a reland of commit 8454fc2458
>
> Original change's description:
> > Remove Python 2 support for PRESUBMIT.py
> >
> > The presubmit system still supports invoking PRESUBMIT.py files using
> > Python 2. This has recently been turned off on the bots so this change
> > removes support more completely.
> >
> > There are still some python3 parameters being passed around - it seemed
> > better to do the simplest possible removal now, with a follow-up change
> > to remove more support code after this has sat for a while.
> >
> > Tests run from PRESUBMIT.py files could still be run using Python 2, but
> > those should also have been addressed already. Removing support for that
> > will be done in a subsequent change.
> >
> > Bug: 1207012
> > Change-Id: Id244d547a04438f83734dba269c3cc180c148b37
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4315183
> > Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> > Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
>
> Bug: 1207012
> Change-Id: If542cac21d8ec8704b28d03fd8407c5c2899ca2c
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4317177
> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
> Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Bug: 1207012
Change-Id: Iebf76d9e2580761fc773791bac07439160503c98
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4323198
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
This is a reland of commit 8454fc2458
Original change's description:
> Remove Python 2 support for PRESUBMIT.py
>
> The presubmit system still supports invoking PRESUBMIT.py files using
> Python 2. This has recently been turned off on the bots so this change
> removes support more completely.
>
> There are still some python3 parameters being passed around - it seemed
> better to do the simplest possible removal now, with a follow-up change
> to remove more support code after this has sat for a while.
>
> Tests run from PRESUBMIT.py files could still be run using Python 2, but
> those should also have been addressed already. Removing support for that
> will be done in a subsequent change.
>
> Bug: 1207012
> Change-Id: Id244d547a04438f83734dba269c3cc180c148b37
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4315183
> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Bug: 1207012
Change-Id: If542cac21d8ec8704b28d03fd8407c5c2899ca2c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4317177
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
This reverts commit 8454fc2458.
Reason for revert: post submit hooks failing
https://crbug.com/1422416
Original change's description:
> Remove Python 2 support for PRESUBMIT.py
>
> The presubmit system still supports invoking PRESUBMIT.py files using
> Python 2. This has recently been turned off on the bots so this change
> removes support more completely.
>
> There are still some python3 parameters being passed around - it seemed
> better to do the simplest possible removal now, with a follow-up change
> to remove more support code after this has sat for a while.
>
> Tests run from PRESUBMIT.py files could still be run using Python 2, but
> those should also have been addressed already. Removing support for that
> will be done in a subsequent change.
>
> Bug: 1207012
> Change-Id: Id244d547a04438f83734dba269c3cc180c148b37
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4315183
> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Bug: 1207012
Bug: 1422416
Change-Id: Iaf3102e63ec3c698d0258fac5746dbd92c30edbb
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4317176
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
The presubmit system still supports invoking PRESUBMIT.py files using
Python 2. This has recently been turned off on the bots so this change
removes support more completely.
There are still some python3 parameters being passed around - it seemed
better to do the simplest possible removal now, with a follow-up change
to remove more support code after this has sat for a while.
Tests run from PRESUBMIT.py files could still be run using Python 2, but
those should also have been addressed already. Removing support for that
will be done in a subsequent change.
Bug: 1207012
Change-Id: Id244d547a04438f83734dba269c3cc180c148b37
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4315183
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
This change introduces lib/ directory which will be used for classes
that are considered libraries and used by multiple executables.
Change-Id: I3da126778bf5ffdb28d7a3c2b966c85de08ba717
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4289679
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
While we're at it, ensure that the working directory is reverted even if
exceptions are raised by moving that part into a finally block.
Bug: 1404222
Change-Id: I8606bc5810202ff1ecdc7f202e3f574bf643f91d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4129515
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Pavol Marko <pmarko@chromium.org>
.rs files were not part of DEFAULT_FILES_TO_CHECK which means that they
were not being checked for proper licenses, among other things.
Testing with this command reveals that six missing or bad licenses have
crept in:
git cl presubmit --force --files "*.rs"
Output was:
Found a bad license header in these files:
build\rust\tests\test_aliased_deps\main.rs
build\rust\tests\test_aliased_deps\real_name.rs
mojo\public\rust\tests\util\mojom_validation.rs
testing\rust_gtest_interop\gtest_attribute.rs
tools\crates\gnrt\sample_package\foo\src\lib.rs
tools\crates\gnrt\sample_package\src\main.rs
Change-Id: Ia7788d92d428a4060f4a0b854fdcee0450b0a3f6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4128762
Commit-Queue: Gavin Mak <gavinmak@google.com>
Auto-Submit: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Collin Baker <collinbaker@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
For reasons unknown, the presubmit seems to make multiple passes
sometimes; this sometimes results in an email address being inserted
into the list of additional CCs multiple times. While this is de-duped
by Gerrit, it still generates a fairly spammy message on the client
side.
Change-Id: I38688462d02ff17f3765c91af326161107027351
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3957428
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Daniel Cheng <dcheng@chromium.org>
The initial version of detecting Python 2 usage in presubmit child
scripts used a text file in depot_tools to record Python 2 usage. This
caused problems in chromeos so this change moves to the file to the temp
directory, which is where it should have been in the first place.
Bug: 1366661
Change-Id: I003b813d7bd5f70956dd3418f86e8ed6edf88220
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3954470
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
When a presubmit message, warning, or error strikes it is sometimes due
to a bug or weakness in the presubmit. Examining the presubmit or fixing
it can be important. However it can be hard to find the relevant code
(hint: many presubmits are in depot_tools/presubmit_canned_checks.py).
With this change you can just run the presubmits with -v -v (double
verbose) and a call stack will be recorded when each presubmit result
object is created. For instance:
>git cl presubmit --force --files ash/public/cpp/app_list/vector_icons/google_black.icon -v -v
** Presubmit Messages: 1 **
Trademarked images should not be added to the public repo. See crbug.com/944754
ash/public/cpp/app_list/vector_icons/google_black.icon
***************
Presubmit result call stack is:
File "depot_tools/presubmit_support.py", line 2098, in <module>
sys.exit(main())
File "depot_tools/presubmit_support.py", line 2074, in main
return DoPresubmitChecks(
File "depot_tools/presubmit_support.py", line 1771, in DoPresubmitChecks
results += executer.ExecPresubmitScript(presubmit_script, filename)
File "depot_tools/presubmit_support.py", line 1612, in ExecPresubmitScript
self._run_check_function(function_name, context, sink,
File "depot_tools/presubmit_support.py", line 1653, in _run_check_function
result = eval(function_name + '(*__args)', context)
File "<string>", line 1, in <module>
File "chromium/src/PRESUBMIT.py", line 2225, in CheckNoProductIconsAddedToPublicRepo
message_type(
File "depot_tools/presubmit_support.py", line 352, in __init__
self._long_text += ' '.join(traceback.format_stack(None, 8))
This changes tracking down presubmits from a dark art to a trivial operation.
Bug: 1309977
Change-Id: Ia0a6adfbbab04041f97c56cd2064a1627e252561
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3896076
Reviewed-by: Dmitrii Kuragin <kuragin@chromium.org>
Commit-Queue: Dmitrii Kuragin <kuragin@chromium.org>
The FilterSourceFile comment block said that the filter lists use
regular expressions, but that is not specific enough because .search and
.match have different semantics. crrev.com/c/3679163 added a filter that
only matched on PRESUBMIT.py instead of all of them because of this
difference - better documentation might have avoided this.
The new documentation also makes it clear that Linux-style path
separators in the regex expressions are fine (they have been supported
since crrev.com/c/3609117).
Bug: 1309977
Change-Id: I49d37fbb93ef4a85f143d928a56af8ce22850c76
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3889880
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Jesse McKenna <jessemckenna@google.com>
The warning that a PRESUBMIT.py file was still being run under Python 2
was apparently too polite as one was just added. This change adds
instructions on how to prevent this, which should make it clear that
the behavior is not desirable.
The warning was added in crrev.com/c/3826917.
Bug: 1313804
Change-Id: I81a6ea45907b5ea00e7d945c3131a329708cc377
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3878875
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
This is a reland of commit 10dbd7babd
Original change's description:
> Remove old DepotToolsOwners implementation
>
> code-owners should have been enabled for most hosts that depot_tools
> supports by now. Remove our own implementation and rely on code-owners.
>
> Change-Id: Iaf0d3db65b2e5063b67d42b92188c4ec51d2cd9a
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3783475
> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
> Commit-Queue: Gavin Mak <gavinmak@google.com>
Change-Id: Ic87f34411bb38b91fcf49adb07601ae10244e828
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3881043
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
This removes backslash from log like
```
Python 2 scripts were run during Python 2 presubmits. Please see https://bugs.chromium.org/p/chromium/issues/detail?id=1313804#c61 for tips on resolving this.
scripts/common/env.py print from /usr/local/google/home/tikuta/chrome_infra/build \
/usr/local/google/home/tikuta/chrome_infra/build/recipes/unittests/recipe_test.py from /usr/local/google/home/tikuta/chrome_infra/build \
-u /usr/local/google/home/tikuta/chrome_infra/build/recipes/.recipe_deps/recipe_engine/recipe_engine/main.py --package /usr/local/google/home/tikuta/chrome_infra/build/infra/config/recipes.cfg test run from /usr/local/google/home/tikuta/chrome_infra/build \
-u /usr/local/google/home/tikuta/chrome_infra/build/recipes/.recipe_deps/recipe_engine/recipe_engine/main.py --package /usr/local/google/home/tikuta/chrome_infra/build/infra/config/recipes.cfg --proto-override /usr/local/google/home/tikuta/chrome_infra/build/recipes/.recipe_deps/_pb --log-level ERROR -O depot_too
ls=/usr/local/google/home/tikuta/chrome_infra/build/recipes/.recipe_deps/depot_tools -O chromiumos_proto=/usr/local/google/home/tikuta/chrome_infra/build/recipes/.recipe_deps/chromiumos_proto -O infra=/usr/local/google/home/tikuta/chrome_infra/build/recipes/.recipe_deps/infra -O recipe_engine=/usr/local/google/home
/tikuta/chrome_infra/build/recipes/.recipe_deps/recipe_engine -O chromeos=/usr/local/google/home/tikuta/chrome_infra/build/recipes/.recipe_deps/chromeos -O chromiumos_config=/usr/local/google/home/tikuta/chrome_infra/build/recipes/.recipe_deps/chromiumos_config test _runner --cov-file /tmp/tmpqNhYvj.recipe_test_cov
erage/thread-0.coverage --cover-module-imports from /usr/local/google/home/tikuta/chrome_infra/build \
```
I think this backslash is bit confusing there.
Change-Id: If5911e27808cfcde029d5ea2b34dda06847085ff
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3827922
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
When running presubmits you can tell if Python 2 presubmits are running,
but it's subtle, and it isn't obvious which presubmits are running under
Python 2.
This makes it obvious by always printing the path to PRESUBMIT.py
scripts that are run under Python 2. This change also fixes a bug in the
--verbose logic - it would print that a script was running before
evaluating _ShouldRunPresubmit.
Bug: 1352347
Change-Id: I45213acdb3b37f7be0713e5caba775841704817c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3826917
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
code-owners should have been enabled for most hosts that depot_tools
supports by now. Remove our own implementation and rely on code-owners.
Change-Id: Iaf0d3db65b2e5063b67d42b92188c4ec51d2cd9a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3783475
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Recent gerrit issues made it obvious that "git cl presubmit" relies on
gerrit much more than most people would expect (the expectation is zero
for many people). This makes presubmits flaky or much slower under poor
network conditions, and it means that the presubmit step may drastically
underestimate how long it takes to run because of a
cl.FetchDescription() that may occur outside of the timed portion of the
presubmits.
This change wraps more network-touching steps in try/except blocks, to
make them robust. It also gets them to check for the existence of a
PRESUBMIT_SKIP_NETWORK environment variable. And, it prints the elapsed
time to get the CL description if this is inordinately long.
Bug: 1350227
Change-Id: I7954fd50e928fd24975a4f61a316cb280542ebbd
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3813095
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
When investigating presubmits it is sometimes handy to get stacks in
order to understand where a presubmit is being called from. In one
recent investigation to see why CheckOwners was being called multiple
times the traceback just said:
File "c:\src\depot_tools\presubmit_support.py", line 1646, in _run_check_function
result = eval(function_name + '(*__args)', context)
File "<string>", line 1, in <module>
File "PRESUBMIT.py", line 251, in CheckChangeOnUpload
That's not very helpful because Chromium has 288 files named
PRESUBMIT.py. This change just uses the actual presubmit path as the
filename parameter to compile instead of a hard-coded string, which
means that "PRESUBMIT.py" is replaced with the full path in the
tracebacks.
Bug: 1309977
Change-Id: Icacfae4399444cb24e9cfb07475b90099e2e18d3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3776955
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Some presubmits can be take orders of magnitude longer than others, but
finding these presubmits is difficult if the time is printed at the end
of the output lines. Moving the time to the start, with enough alignment
for all plausible times, makes scanning the output easy.
Bug: 1309977
Change-Id: I9ba1a599e9825801e1128844a39c7ccafe046f91
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3744130
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Headers currently have the format: `** Presubmit ${name} **`. Make these
more helpful by including the number of total errors there are.
Bug: 1341987
Change-Id: Ib41a133c31568d4264d73c14350f0bbd9590356a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3756169
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
PRESUBMIT.py files should not have CRLF line endings, but if a git repo
is misconfigured this may happen and it is important for the presubmits
to run properly so that they can report on this issue.
In particular, the regex which detects USE_PYTHON3 = True did not work
with CRLF line endings.
It was supposed to handle these, and there is even a comment saying:
# Accept CRLF presubmit script.
However the line after that uses 'rU' as the file open mode in order to
try to get universal newlines, but FileRead ignores the file mode. So,
the replace method is now used instead.
Bug: angleproject:7475, angleproject:4905
Change-Id: I38321176feca089a5796b24756b371709d852243
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3739269
Reviewed-by: Eddie Hatfield <eddiehatfield@google.com>
Reviewed-by: Cody Northrop <cnorthrop@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Aravind Vasudevan
Gavin Mak
Rupert Wiser
Brian Ryner
Joanna Wang
Joanna Wang
Josip Sokcevic
Liviu Rau
Mike Frysinger
Yiwei Zhang
Bruce Dawson
Daniel Cheng
Robert Iannucci
Josip Sokcevic
Pavol Marko
Erik Staab
Takuto Ikuta