Commit Graph

288 Commits (364205c70ed16c00802b1c264e88d8e03a0b37ae)

Author SHA1 Message Date
Gavin Mak e6a623392d [depot_tools] Use logging.warning
logging.warn is deprecated and should be replaced with logging.warning.

Bug:1097402
Change-Id: I55ce007bec47608eeafcf3cd9ac2b90f60073115
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2575621
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
4 years ago
Scott Lee cc2fe9b767 [resultdb] use requests.Session in rdb_wrapper
requests.post() creates a new HTTP connection for each call.
It makes presubmit checks slower and timed out.
This CL updates rdb_wrapper to use requests.Session() to use
a connection pool for Sink requests.

Change-Id: Id02ecce85898ae15993d53df17473bb5dfb7e89f
Bug: 1145762
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2532895
Commit-Queue: Scott Lee <ddoman@chromium.org>
Reviewed-by: Chan Li <chanli@chromium.org>
Reviewed-by: Erik Staab <estaab@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
4 years ago
Edward Lesmes cf49cb8206 [presubmit][owners] Skip owners check when Owners-Override is +1
Add support for Owners-Override label to allow trusted bots, sheriffs,
and global owners to bypass owners checks.

Bug: 1093627
Change-Id: Ie217369a67e4060807c0a12f84430e30a7c57daa
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2530488
Reviewed-by: Yulan Lin <yulanlin@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes 02d4b82580 [presubmit][owners] Skip owners check when Bot-Review is +1
Add support for Bot-Review label to allow trusted bots to
bypass owners checks.

Bug: 1093627
Change-Id: Ie7398041e9e32fd87bb415b239b5ec0a4740e1ec
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2530516
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Yulan Lin <yulanlin@google.com>
5 years ago
Garrett Beaty acb807e458 Output a newline when there are warnings and no prompt.
Not outputting a newline when not prompting causes the command prompt to
not appear on a new line.

Change-Id: I791c829e1e0b37c9ad5d6bfdb246db4c779be020
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2380598
Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
5 years ago
Josip Sokcevic c6aa151180 Warn only if non-inclusive term is used
This fixes the issue where presubmit_canned_checks issues a warning if
optional arguments in CheckAuthorizedAuthor are not used at all.

This relands commit b09f2bb2f2 with some
additional changes.

R=apolito@google.com

Bug: 1098560
Change-Id: If323d90ab7d6bcca68ed89142ea67edc4be057d2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2373216
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Josip Sokcevic b09f2bb2f2 Revert "Warn when non-inclusive parameters are used"
This reverts commit 149a3b7581.

Reason for revert: warnings are issued if no optional parameters
are passed.

Original change's description:
> Warn when non-inclusive parameters are used
> 
> R=​apolito@google.com
> 
> Bug: 1098560
> Change-Id: Iabc2a5d89a6af1418c1228f3c09eacbbd799b5c7
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2355112
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>

TBR=apolito@google.com,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: 1098560
Change-Id: Ib0f188de90121c692f5c0d7271938ee312917fd1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2376042
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Josip Sokcevic 149a3b7581 Warn when non-inclusive parameters are used
R=apolito@google.com

Bug: 1098560
Change-Id: Iabc2a5d89a6af1418c1228f3c09eacbbd799b5c7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2355112
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Saagar Sanghavi 9981690ea6 New approach for timing lower-level checks
Use input_api.version and PRESUBMIT_VERSION to ensure newest versions.

Enable presubmit_support to time/ResultDB wrap each individual
lower-level check in PRESUBMIT.py. The approach taken here is to have
the PRESUBMIT_VERSION = 2.0.0 in PRESUBMIT.py
so that presubmit_support is aware of the new version of PRESUBMIT.py,
then check input_api.version in CheckChangeOnCommit/Upload to ensure
that presubmit_support itself is the new version.


Once these changes are landed, the associated changes in PRESUBMIT.py
(crrev.com/c/2350234) can landed as well.


Change-Id: If6d5e318547bc4f128ffe151387c4dc339ef5dc8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2349979
Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
5 years ago
Saagar Sanghavi 531d992f00 Changed naming convention
presubmit:gerrit_host/folder/to/repo:path/to/file/:CheckName
Example
presubmit:chromium-review.googlesource.com/chromium/src/:services/viz/

Bug: 1112667
Change-Id: I095a2c04e73c64d67fa1bb3dbf7e6dfd1d93fe26
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2335873
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>
5 years ago
Saagar Sanghavi 03b15131d3 [Backward Compatible] Timing and RDB for Lower-level checks
Allows presubmit_support to perform Timing and RDB reporting for each of
the lower-level "CheckXYZ" functions, provided that
ENABLE_NEW_PRESUBMIT_CHECKS is called in the PRESUBMIT.py
file that we are running. I am using the variable to avoid the breakage
caused by the similar previous CL (2332321) which was not backwards
compatible.
In addition, asks for "-realm" with ResultDB so it can maintain the
security boundary.

Bug: 1106943
Change-Id: Ia49e8884e2653bee7de7ed7d254452d55dd5c617
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2341828
Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Ben Pastene 8351dc1fb7 Revert "[Fixed from older branch] Timing/RDB for lower-level checks"
This reverts commit dba1d65a19.

Reason for revert: "Check* takes exactly 3 arguments (2 given)" presubmit errors
eg: https://ci.chromium.org/p/chromium/builders/try/chromium_presubmit/933062?
https://crbug.com/1113506

Original change's description:
> [Fixed from older branch] Timing/RDB for lower-level checks
> 
> presubmit: Report timing to rdb for individual checks
> Rather than using CheckChangeOnCommit() or CheckChangeOnUpload() as the
> sole entry points for PRESUBMIT.py files, this cl enables presubmit_support.py
> to use any CheckXYZ[OnCommit/Upload] function to serve as an entry point.
> 
> This way, we can perform timing and RDB reporting for each of the
> presubmit check functions and have check-specific data.
> 
> Bug: 1106943
> Change-Id: Ifdabd68c0904a6d70a828f12de157369c6c1571d
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2332321
> Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Dirk Pranke <dpranke@google.com>

TBR=dpranke@google.com,estaab@chromium.org,ehmaldonado@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,saagarsanghavi@google.com

Change-Id: Ibd80f70661d2196f903f0c6b79eb74eda6c94d44
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1106943, 1113506
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2340692
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
5 years ago
Saagar Sanghavi dba1d65a19 [Fixed from older branch] Timing/RDB for lower-level checks
presubmit: Report timing to rdb for individual checks
Rather than using CheckChangeOnCommit() or CheckChangeOnUpload() as the
sole entry points for PRESUBMIT.py files, this cl enables presubmit_support.py
to use any CheckXYZ[OnCommit/Upload] function to serve as an entry point.

This way, we can perform timing and RDB reporting for each of the
presubmit check functions and have check-specific data.

Bug: 1106943
Change-Id: Ifdabd68c0904a6d70a828f12de157369c6c1571d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2332321
Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
5 years ago
Saagar Sanghavi 98b332f2db Fixed relative path to work when presubmit run from any directory
Bug: 1111829
Change-Id: I75c616f4fa9c16c19e732e43ba740bb67f3eb827
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2327839
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>
5 years ago
Saagar Sanghavi 9949ab7a4b Preliminary changes to git cl and presubmit_support
Added an rdb wrapper and use it to time CheckChangeOn{Commit, Upload} from every PRESUBMIT.py file, then reports the results to ResultSink.

Bug: http://crbug.com/1107588

Change-Id: I9bc8a18ba615a76f3e0611eae5b2c4adc5924276
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2242097
Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Garrett Beaty <gbeaty@chromium.org>
5 years ago
local_bot 6402141d1f Rename blocklist/allowlist to skip_list/check_list
This change is backwards compatible.

R=dpranke@google.com

Bug: 1098560
Change-Id: Ic9447041a68b8c379f65900af11c89b006e3f806
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2285156
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
5 years ago
local_bot 37ce201d92 Add setters to legacy input_api.DEFAULT_*
Some PRESUBMIT.py scripts set DEFAULT_WHITE_LIST and DEFAULT_BLACK_LIST.
This patch adds support to override those defaults during transition
period.

R=apolito@google.com, ehmaldonado@chromium.org

Bug: 1098562
Change-Id: I5dfff253e61485d354a83f3f509d8a6d4d87dec6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2270717
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
local_bot 30f774ef87 Rename whitelist/blacklist to allowlist/blocklist
R=apolito@google.com, ehmaldonado@chromium.org

Bug: 1098562
Bug: 1098560
Change-Id: I45dc739f9e6d1a2e9bfcd9fa0587f4bb4a115c5f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2265255
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Aaron Gable 6e7ddb698b Remove agable from depot_tools OWNERS and TODOs
Change-Id: I0667af4562208a111ab3673f3fca6aab655f5f26
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2218962
Auto-Submit: Aaron Gable <agable@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Anthony Polito <apolito@google.com>
5 years ago
Bruce Dawson 7a0b07a8a5 Fix slash direction sensitivity in git cl owners
Slashes and backslashes are interchangeable on Windows in many contexts.
But not all. In particular, if you add a path to a dictionary and then
try looking it up with an equivalent path that has different slashes
then - not surprisingly - the lookup will fail. Therefore it is
important to always use slashes when populating _paths_to_owners. This
is now done, and documented.

Additionally, while fnmatch.fnmatch handles comparing slash separated
paths to backslash separated paths, fnmatch.translate does not. This is
arguably a bug in fnmatch.translate, or at least something that should
be documented. This bug is worked around by having _fnmatch sanitize
file names to always use slashes, and asserting that the patterns do
not contain backslashes.

With these changes this command:
  git cl owners --ignore-current --show-all chrome/browser/BUILD.gn
now correctly gives the same results as this command on Windows:
  git cl owners --ignore-current --show-all chrome\browser\BUILD.gn

The modified test fails without the other changes, passes with them.

Bug: 1009104
Change-Id: I416c7131281f00e352c1d2b85c30fcc463417fa5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1915410
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Erik Staab cca5c49d45 Add timestamp info to presubmit logs for easier debugging.
Change-Id: I80f53f1130ba1dd17c886b14a5737d30394604f0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2152004
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Erik Staab <estaab@chromium.org>
5 years ago
Edward Lemur ac5c55f6d7 Revert "Improve git cl split"
This reverts commit 684096347b.

Reason for revert:
Breaks git-cl split.
https://bugs.chromium.org/p/chromium/issues/detail?id=1054888

Original change's description:
> Improve git cl split
>
> This CL changes the behavior of `git cl split` to split the change
> by the size of the resulting CLs. For now, this is based on the number
> of bytes changed, and not by the number of changed lines. Depending
> on the shape of change, this may still produce more CLs than expected
> (and possibly more than before).
>
> A future change will switch the split to be based on the number
> of affected lines, and also introduce a mode to base the split
> on the number of affected files.
>
> Bug: 998922
> Change-Id: I49f868972a61b89b426ef9e2ceedc733eacb4350
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1778744
> Commit-Queue: Yannic Bonenberger <yannic.bonenberger@gmail.com>
> Reviewed-by: Dirk Pranke <dpranke@chromium.org>

TBR=fdoray@chromium.org,dpranke@chromium.org,yannic.bonenberger@gmail.com,infra-scoped@luci-project-accounts.iam.gserviceaccount.com

Bug: 998922
Change-Id: I466e1245d5f1c934443d850c48c42d3e694e71c6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2080205
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lemur 7552630e0d git-cl: Run PostUpload hooks via CLI.
Bug: 1042324
Change-Id: Iae90e632e9c1b2c798575ab940d82a145b47204a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2070681
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
5 years ago
Edward Lemur 6eb1d32f5d presubmit_support: Refactor and add tests for DoPostUploadExecuter.
Remove input and output stream, that always pointed to sys.stdin
and sys.stdout.
Add tests for DoPostUploadExecuter.

Bug: 1042324
Change-Id: Ib5d092706a242ae97ec877af5dbceef6db7cbb73
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2079411
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lemur 227d510857 git-cl: Invoke presubmit checks via subprocess execution instead of via module.
PRESUBMIT.py scripts are executed in presubmit_support.py using exec().
Since PRESUBMIT.py scripts might not be yet compatible with python 3, we
have to execute presubmit_support.py using python 2.

git_cl.py imports presubmit_support.py, and executes presubmit checks using
presubmit_support as a module. This forces git_cl.py to be executed using
python 2 to maintain compatibility for PRESUBMIT.py scripts.

This change allows git_cl.py to be executed using python 3, while
presubmit_support.py is executed using python 2.

Similar changes for post-submit hooks and git-cl try masters will follow.

Bug: 1042324
Change-Id: Ic3bb1c2985459baf6aa04d0cc65017a1c2578153
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2068945
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lemur 1dc66e16b3 presubmit: Add should_continue and more_cc to presubmit json output.
Also, rename --all-files to --all_files to be consistent with other
options, and remove reviewers, since it's never set in presubmit_support.

Bug: 1042324
Change-Id: I34fa3162c56e39bcdd52ad1e19058357de934f1c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2067351
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lemur 7f6dec0169 git-cl: Simplify Change object construction.
The `author` parameter is never set.
The `local_description` parameter was only set with the
purpose of getting the local description, so separate it into its
own method and call that instead.
Use the absolute root when getting the list of modified files instead.

Bug: 1042324
Change-Id: I8505172edaacee0656dda98ebbd5f8ebfacb3cef
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2038050
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lemur 50984a64fa Reland "presubmit_support: Add --all-files options."
This is a reland of 1c8026d2f1

Original change's description:
> presubmit_support: Add --all-files options.
> 
> Will be set when --all is passed to git-cl presubmit to execute
> presubmit checks over all files on a repo.
> 
> Bug: 1042324
> Change-Id: I2ebb7c3ec633f9d3a3774314799aa056bc2919c8
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2003071
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

Bug: 1042324
Change-Id: I452d98972f9b2c6d0a212029bac5aa6af57dce40
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2038055
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes 44ea3ffa40 Revert "presubmit_support: Add --all-files options."
This reverts commit 1c8026d2f1.

Reason for revert:
Description should be set from gerrit before parsing the changes.

Original change's description:
> presubmit_support: Add --all-files options.
> 
> Will be set when --all is passed to git-cl presubmit to execute
> presubmit checks over all files on a repo.
> 
> Bug: 1042324
> Change-Id: I2ebb7c3ec633f9d3a3774314799aa056bc2919c8
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2003071
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

TBR=ehmaldonado@chromium.org,apolito@google.com,infra-scoped@luci-project-accounts.iam.gserviceaccount.com

Change-Id: I8c455a269cdd6eb3dea9379faf351a04644bd23f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1042324, 1048885
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2038052
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lemur 016a0876ea presubmit_support: Pass a gerrit object instead of a cl object to PostUploadHook.
Pass a gerrit object instead of a git-cl ChangeList object to
PostUploadHook to remove dependency of presubmit_support in git-cl.

Requires https://skia-review.googlesource.com/c/skia/+/265209

Bug: 1042324
Change-Id: I4b20b75deb6a3d8d4caf46ca3580883699dbf707
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2008287
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lemur 1c8026d2f1 presubmit_support: Add --all-files options.
Will be set when --all is passed to git-cl presubmit to execute
presubmit checks over all files on a repo.

Bug: 1042324
Change-Id: I2ebb7c3ec633f9d3a3774314799aa056bc2919c8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2003071
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lemur e4d329cc00 presubmit_support: Add support for setting the change description.
Needed by Skia's PostUploadHooks, so that presubmit_support
doesn't depend on git-cl classes.

Bug: 1042324
Change-Id: I02475e346437b26fa8d0d2021ea62754077a2b5b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2007262
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lemur 69bb8bef75 presubmit_support: Support getting and adding footers.
Needed by Skia's PostUploadHooks, so that presubmit_support doesn't
depend on git-cl internal classes.

Bug: 1042324
Change-Id: If8f4e6ea035197667cd9705a4274590c4d11511e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2007670
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Yannic Bonenberger 684096347b Improve git cl split
This CL changes the behavior of `git cl split` to split the change
by the size of the resulting CLs. For now, this is based on the number
of bytes changed, and not by the number of changed lines. Depending
on the shape of change, this may still produce more CLs than expected
(and possibly more than before).

A future change will switch the split to be based on the number
of affected lines, and also introduce a mode to base the split
on the number of affected files.

Bug: 998922
Change-Id: I49f868972a61b89b426ef9e2ceedc733eacb4350
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1778744
Commit-Queue: Yannic Bonenberger <yannic.bonenberger@gmail.com>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
5 years ago
Edward Lemur b24ca5ac74 presubmit_support: Initialize Gerrit object when gerrit URL is given.
Only gerrit_url is needed to initialize gerrit_obj.
This will be needed for git cl to call presubmit support
as a script.

Bug: 1042324
Change-Id: I79d78fb67465f989755bf8c8ceb48d665464fa05
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2005050
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lemur a5799e3bb4 presubmit_support: Use single quotes and convert to ArgumentParser.
Change-Id: I54f0fe3a2257166cba2a69659e5128e4ac9310bc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2001707
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lemur ecc27074d2 presubmit_support: Add timeout to presubmit tests.
Change-Id: I7a434d4420573c6e5fc014455b011984f4d681b3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1952159
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lemur 7cf9438ca9 presubmit: Print executed command along with test name.
Useful when runnig the same test both on Python 2 and Python 3.

Change-Id: I6074e4cbd64318985cf438a2d491182b7b9ccc7b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1913259
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Edward Lemur 336e51f685 depot_tools: Execute tests using bot's vpython instead of depot_tools' vpython
On Windows, scripts on the current directory take precedence over PATH, so
that when testing depot_tools on Windows, calling `vpython.bat` will
execute the copy of vpython of the depot_tools under test instead of the
one in the bot.
As a workaround, we run the tests from the parent directory instead.

Change-Id: I5187943b6cf7499d50b3f1a2ec906a5e8fd84835
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1910448
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Edward Lemur b983024505 presubmit_support: Fix tests on Python 3.
The problems on crbug.com/1017367 should have been fixed by crrev.com/c/1880013

Bug: 1009814
Change-Id: Ia98304ca983b4e7e293fedb9df1fee589de58ba9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1887714
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Edward Lemur b964662972 presubmit_support: Set python_executable to vpython.
PRESUBMIT.py scripts might execute commands directly instead of using
CallCommand, so that the conversion to vpython is bypassed, resulting in
errors.

See e.g.
https://cs.chromium.org/chromium/src/third_party/blink/web_tests/PRESUBMIT.py?l=79

Bug: 1017410
Change-Id: If1943fe46cd2fc1e37b89aee14865429f884332b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1879493
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Edward Lemur de9e3cabe6 presubmit: Make ThreadPool surface exceptions on CallCommand.
Exceptions other than OSError are not surfaced.
This caused errors like this to be printed, but not block presubmit,
allowing bugs to sneak in.

Exception in thread Thread-8:
Traceback (most recent call last):
  File "C:\b\s\w\ir\cipd_bin_packages\cpython\bin\Lib\threading.py", line 801, in __bootstrap_inner
    self.run()
  File "C:\b\s\w\ir\cipd_bin_packages\cpython\bin\Lib\threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "C:\b\s\w\ir\kitchen-checkout\depot_tools\presubmit_support.py", line 199, in _WorkerFn
    result = self.CallCommand(test)
  File "C:\b\s\w\ir\kitchen-checkout\depot_tools\presubmit_support.py", line 170, in CallCommand
    p = subprocess.Popen(cmd, **test.kwargs)
  File "C:\b\s\w\ir\kitchen-checkout\depot_tools\subprocess2.py", line 143, in __init__
    super(Popen, self).__init__(args, **kwargs)
  File "C:\b\s\w\ir\cipd_bin_packages\cpython\bin\Lib\subprocess.py", line 390, in __init__
    errread, errwrite)
  File "C:\b\s\w\ir\cipd_bin_packages\cpython\bin\Lib\subprocess.py", line 640, in _execute_child
    startupinfo)
TypeError: environment can only contain strings

https://logs.chromium.org/logs/infra/buildbucket/cr-buildbucket.appspot.com/8898840708364523888/+/steps/presubmit/0/stdout

Change-Id: I34e65d8c0050eed7ed26fd782e0a5dc8616f30f7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1877051
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
6 years ago
Edward Lemur d2a5a4c043 Revert "presubmit_support: Fix tests on Python 3."
This reverts commit a834f39e21.

Reason for revert:
https://bugs.chromium.org/p/chromium/issues/detail?id=1017367

Original change's description:
> presubmit_support: Fix tests on Python 3.
>
> Bug: 1009814
> Change-Id: I2c2a835fc4fa0a7fc9e68dfff680fb0beacb491e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1869535
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

TBR=ehmaldonado@chromium.org,apolito@google.com

Bug: 1009814
Change-Id: I48907caeca9b9b32e4e3a7cb5a636280d017a4c3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1877350
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Edward Lesmes 401b25c878 presubmit: Convert literals to strings before setting env variables.
On windows on Python 2, only strings can be environment variables, but
literals are unicode, so we need to convert to strings before setting
env vars.

Change-Id: I40e8e6ae4f5497d56efa568153eb65392dbd85e6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1875108
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Edward Lemur a834f39e21 presubmit_support: Fix tests on Python 3.
Bug: 1009814
Change-Id: I2c2a835fc4fa0a7fc9e68dfff680fb0beacb491e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1869535
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Edward Lemur 16af35606f presubmit: Remove unused imports.
Some of the modules imported by presubmit_support are unused, and some of those (cPickle, cStringIO) are not available on Python3, so remove them.
Also expose urllib2 via urllib_request and urllib_error to improve compatibility with Python3.

Bug: 1009814
Change-Id: I9f758df4270cc039df092e814c981a7a87b2c350
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1866170
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Marc-Antoine Ruel 8e57b4bc55 python3 improvements
Ran:
  vi $(git grep --name-only iteritems | grep -v third_party)
  vi $(git grep --name-only itervalues | grep -v third_party)
  vi $(git grep --name-only 'print ' | grep -v third_party)

and edited the files quickly with adhoc macros. Then ran in recipes/:
  ./recipes.py test train

There was only a small subset of files that had been updated to use
six.iteritems() and six.itervalues(). Since the dataset size that is
being used in gclient is small (pretty much always below 200 items),
it's better to just switch to .items() right away and take the temporary
performance hit, so that we don't need to come back to rewrite the code.

Recipe-Nontrivial-Roll: build
Bug: 984182
Change-Id: I5faf11486b66b0d73c9098ab0f2ce1b15a45c53e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1854900
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Marc-Antoine Ruel <maruel@chromium.org>
6 years ago
Dan Beam 6295404c95 Support "Fixed:" syntax in BugsFromDescription()
Bug: monorail:4470
Change-Id: I2a1dcc1fe00d29f9d058152c15c75f8f904c354a
Recipe-Nontrivial-Roll: chromiumos
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1836601
Auto-Submit: Dan Beam <dbeam@chromium.org>
Commit-Queue: Dan Beam <dbeam@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
6 years ago
Edward Lemur 9a5bb612e6 git-cl: Make Ctrl+C work again.
Bug: 961724
Change-Id: I4fc8f86c02fb53310c5967284594ba1fbbbcbafc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1826054
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
6 years ago
Edward Lemur 2d6b67c590 depot_tools: Fix bug when running both python2 and python3 tests.
Don't overwrite passed kwargs. Make a copy instead.
Otherwise, subsequent calls with the same kwargs will have
stdin set to subprocess.PIPE.

Bug: 984182
Change-Id: I358ffa1951e8b42486e0ac3a0d3d587a93c6dc4e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1769405
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
6 years ago