Commit Graph

2034 Commits (d22bf607218d50bf20444b0610f62b265f829dbc)

Author SHA1 Message Date
Gavin Mak d22bf60721 Add number of presubmit errors/warnings in header
Headers currently have the format: `** Presubmit ${name} **`. Make these
more helpful by including the number of total errors there are.

Bug: 1341987
Change-Id: Ib41a133c31568d4264d73c14350f0bbd9590356a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3756169
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
3 years ago
Joanna Wang 5a7c824fee [no-sync] Add method to check for diffs between the current checkout and another commit.
Bug: 1339472

Change-Id: If97d2fad7e5c61ec907e15a00db225b653a62bef
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3738365
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
3 years ago
Joanna Wang 18af7efdd8 [no-sync] Update ParseDepsFile to process deps and hooks appropriately with should_sync.
Bug: 1339472
Change-Id: Iea077cdb655105d27431ff4f677d93eec121bc31
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3738361
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
3 years ago
Joanna Wang 6628661eba [no-sync] Add a skip_sync_revisions and process it before running deps.
Bug: 1339472
Change-Id: I429489f7deea035c47e27e32ada858fb8a827643
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3735487
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
3 years ago
manukh 566e9d0177 [git-cl] If set, use `options.title` for initial uploads.
Change-Id: Ia57b7a7b769d352e05820af0e40a52fcab471ddb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3736424
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: manuk hovanesian <manukh@chromium.org>
3 years ago
Bruce Dawson 8254f06ba7 Mitigate Python 3 multiprocessing bug on Windows
The multiprocessing module on Windows has a bug where if you ask for
more than 60 child processes then it will hang. This is related to the
MAXIMUM_WAIT_OBJECTS (64) limit of WaitForMultipleObjects. Other sources
have listed the multiprocessing limit as being 61, or have said that the
the maximum number of objects that can be waited on is actually 63, but
those details don't really matter.

The original fix for this class of issues was crrev.com/c/2785844. This
change extends those fixes to depot_tools, which was missed last year.
This change also updates how PyLint is called by further limiting the
number of jobs to the number of files being processed divided by 10.
This is because there is a significant cost to creating PyLint
subprocesses - each takes about 0.5 s on my test machine. So there needs
to be enough parallelism to justify this.

Patches for PyLint and a bug for cpython are planned.

This will stop PyLint from hanging during presubmits on many-core
machines. The command used to reproduce the hangs and validate the fix
was:

  git cl presubmit -v --force --files "chrome/test/mini_installer/*.py"

Prior to this change this command would use (on my many-core test
machine) 96 processes and would hang. How it uses just two processes
because there are only 16 files to analyze.

Output before:
  Pylint (16 files using ['--disable=cyclic-import'] on 96 cores)
Output after:
  Pylint (16 files using ['--disable=cyclic-import'] on 2 processes)

This is actually not quite true because the hang would prevent the
old message from being displayed.

Bug: 1190269, 1336854
Change-Id: Ie82baf91df4364a92eb664a00cf9daf167e0a548
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3711282
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
3 years ago
Joanna Wang a9467d8216 Remove duplicate sparse-checkout call.
Bug:1330629
Change-Id: Ieccf72c75dda7cbec473fcfa07d25fb59f8eb7c5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3689728
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
3 years ago
Aleksey Khoroshilov 35ef5ada1d Run presubmit checks without GerritAccessor if GERRIT_HOST is not set.
When a repository doesn't work with Gerrit, it doesn't make sense to
create GerritAccessor and pass it to presubmit checks.
gerrit.host is stored into git config when codereview.settings file contains GERRIT_HOST item.

Change-Id: I9740950caf85a1da9a6e4ae12e3612625cf0cec5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3687235
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Auto-Submit: Aleksey Khoroshilov <akhoroshilov@brave.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
3 years ago
Joanna Wang 1a977bdc2d Update gclient cloning to work with cog.
Bug:1330629
Change-Id: I522255528e36d4806eada70c22afee1277a6778d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3668429
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
3 years ago
Aravind Vasudevan 5965d3e6fb Reland "Update fetch protocol using --protocol-override flag"
This is a reland of commit 6817010e83

Additional Changes:
This reland fixes https://crbug.com/1330995 by stopping gclient from accidentally updating the scheme when one is not present.

Original change's description:
> Update fetch protocol using --protocol-override flag
>
> This CL updates gclient sync to use the protocol of the URL specified in the solutions for cloning all the child dependencies of it.
>
> Bug: chrome-operations:170
> Change-Id: I33588059788b677fbae8c3b434100af5c7979a67
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3631600
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>

Bug: 1330995
Change-Id: I1447a5e884e41d671d8556c35193f1635f2f6936
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3684112
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
3 years ago
Stephanie Kim 700aee769d Override cipd install mode for runhooks
Currently cipd packages listed in src/DEPS are being created and
installed in symlink mode (see bug for more info). This CL adds `$OverrideInstallMode copy` to the ensure files for cipd packages installed during gclient runhooks. `$OverrideInstallMode` was
implemented here, https://chromium-review.googlesource.com/c/infra/luci/luci-go/+/3124202,
so that every cipd package can be installed directly to the installation
target, instead of being symlinked to an executable inside the cache/builder/.cipd dir.

The added `$OverrideInstallMode copy` can be removed once all cipd
packages themselves or the default mode is updated to be created in
copy mode.

Bug: 1329641
Change-Id: I68dc718e20f193c384d005c9b6b87603753943d4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3673698
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
3 years ago
Aravind Vasudevan fe3a70a926 Revert "Update fetch protocol using --protocol-override flag"
This reverts commit 6817010e83.

Reason for revert: Breaks gclient sync for flutter

Original change's description:
> Update fetch protocol using --protocol-override flag
>
> This CL updates gclient sync to use the protocol of the URL specified in the solutions for cloning all the child dependencies of it.
>
> Bug: chrome-operations:170
> Change-Id: I33588059788b677fbae8c3b434100af5c7979a67
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3631600
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>

Bug: chrome-operations:170
Change-Id: I76a49030f48fa266eeea336307e7153b22ded4e9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3684110
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
3 years ago
Aravind Vasudevan 6817010e83 Update fetch protocol using --protocol-override flag
This CL updates gclient sync to use the protocol of the URL specified in the solutions for cloning all the child dependencies of it.

Bug: chrome-operations:170
Change-Id: I33588059788b677fbae8c3b434100af5c7979a67
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3631600
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
3 years ago
Bruce Dawson 10a8286824 Don't halt presubmits on exceptions
Some presubmit failures result in uncaught exceptions which historically
have halted presubmit runs. This behavior makes it more difficult to
find all failures, measure presubmit times, or detect other behavior
over time. This is particularly problematic when running "git cl
presubmit --all".

As an example, presubmit --all was broken and would not complete from
when crrev.com/3642633 landed until crrev.com/c/3657600. The minimal
repro was this presubmit command:

    git cl presubmit --files ui\accessibility\ax_mode.h

This command, or presubmit --all, triggered this error, which halts the
presubmits:

Evaluation of CheckChangeOnCommit failed: [Errno 2] No such file or directory: 'chrome/browser/resources/accessibility/accessibility.js'
Traceback (most recent call last):
  File "presubmit_support.py", line 2038, in <module>
    sys.exit(main())
  File "presubmit_support.py", line 2015, in main
    return DoPresubmitChecks(
  File "presubmit_support.py", line 1738, in DoPresubmitChecks
    results += executer.ExecPresubmitScript(presubmit_script, filename)
  File "presubmit_support.py", line 1602, in ExecPresubmitScript
    self._run_check_function(function_name, context, sink,
  File "presubmit_support.py", line 1640, in _run_check_function
    six.reraise(e_type, e_value, e_tb)
  File "C:\Users\brucedawson\.vpython-root\e726d2\lib\site-packages\six.py", line 686, in reraise
    raise value
  File "presubmit_support.py", line 1630, in _run_check_function
    result = eval(function_name + '(*__args)', context)
  File "<string>", line 1, in <module>
  File "PRESUBMIT.py", line 314, in CheckChangeOnCommit
    errs.extend(CheckModesMatch(input_api, output_api))
  File "PRESUBMIT.py", line 283, in CheckModesMatch
    ax_modes_in_js = GetAccessibilityModesFromFile(
  File "PRESUBMIT.py", line 245, in GetAccessibilityModesFromFile
    for line in open(fullpath).readlines():
FileNotFoundError: [Errno 2] No such file or directory: 'chrome/browser/resources/accessibility/accessibility.js'

With this change the exception is handled and presubmits continue after
printing this message:

Evaluation of CheckChangeOnCommit failed: [Errno 2] No such file or directory: 'chrome/browser/resources/accessibility/accessibility.js', Traceback (most recent call last):
  File "presubmit_support.py", line 1630, in _run_check_function
    result = eval(function_name + '(*__args)', context)
  File "<string>", line 1, in <module>
  File "PRESUBMIT.py", line 314, in CheckChangeOnCommit
    errs.extend(CheckModesMatch(input_api, output_api))
  File "PRESUBMIT.py", line 283, in CheckModesMatch
    ax_modes_in_js = GetAccessibilityModesFromFile(
  File "PRESUBMIT.py", line 245, in GetAccessibilityModesFromFile
    for line in open(fullpath).readlines():
FileNotFoundError: [Errno 2] No such file or directory: 'chrome/browser/resources/accessibility/accessibility.js'

That is, the crucial information regarding the failure is printed, but
subsequent presubmits are not skipped.

Bug: 1309977
Change-Id: I7cfeda85c830e6c8e567c0df3c50f27af1dbe835
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3653978
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
3 years ago
Bruce Dawson 09c0c073ea Optimize presubmit --all with --no_diffs option
Some of the expensive checks when running presubmit --all, such as
CheckStableMojomChanges (~300 s) and CheckAddedDepsHaveTargetApprovals
(~200 s) only look at diffs and are therefore guaranteed to be NOPs when
running presubmit --all or --files=. Passing along the no_diffs state
lets these expensive checks be skipped, thus allowing for faster
iteration times.

Initial testing suggests that (with some supporting changes in the
Chromium repo) this reduces "presubmit --all" times by about ten
minutes, or a bit more than 10%, and additional improvements may be
possible.

Special handling for the no-diffs case also offers a simple way to avoid
presubmit failures that happen whenever all files are flagged as being
changed.

Finally, and perhaps most importantly for having a presubmit --all bot,
when --no_diffs is passed we can treat errors like "Issue wasn't
uploaded" and "Add a description to the CL" as messages, thus making it
possible to have zero presubmit errors when run on origin/main.

Bug: 1320937, 1322936
Change-Id: I0d09dd4aae8fdaa48c8b2f89337441cf96dcff72
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3628368
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
3 years ago
Bruce Dawson 76fe1a701b Print presubmit messages in a consistent order
Presubmit message types were printed in an inconsistent order, generally
determined by which type was encountered first. This unpredictability
can be confusing when comparing runs. This change enforces a consistent
order, in increasing order of severity. ERRORS always go last so that
they will be the most visible when running presubmits locally.

This also adds a "There were Python %d presubmit errors." message to
the end to make it easier to tell when there were presubmit errors.
This is particularly useful when manually running presubmits.

Bug: 1309977
Change-Id: Ib2b73c7625789bad5b21ae12abf238b746cd11e1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3668724
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
3 years ago
Josip Sokcevic 632bbc0cb1 Skip Python 2 presubmit step when unneeded
All of the PRESUBMIT.py files in the Chromium repo are running under
Python 3. However "git cl presubmit" also works with other repos where
some PRESUBMIT.py scripts still run under Python 2. This means that
the Python 2 presubmit commit checks step cannot simply be disabled.
That meant that Chromium was paying up to a one-minute cost just to
setup for and look for Python 2 scripts that it doesn't run.

This change runs the Python 3 PRESUBMIT.py scripts first, and keeps
track of whether any were skipped. If none were skipped then the
Python 2 PRESUBMIT.py stage can be skipped.

Note that the child scripts of PRESUBMIT.py scripts may still be run
under Python 2, but that is orthogonal to this change.

Bug: 1313804
Change-Id: Ib65838223f232f1e78058d6a08ea15a89f442310
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3614453
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
3 years ago
Joanna Wang 50c631e318 Skip 'ls' when downloading from gs.
Bug: 959167
Change-Id: I160658264dac71a9cff280cf3ba606d583d8f2b3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3646593
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
3 years ago
Bruce Dawson 31bfd51995 Actually add a trailing slash to dir_with_slash
AffectedFiles carefully added a slash to dir_with_slash, but then called
normpath - which trims trailing path separators. This meant that
ui/views/PRESUBMIT.py would analyze files in ui\views_content_client,
leading to spooky action at a distance. This was noticed when this
command triggered a presubmit error:

    git cl presubmit "--files=ui/views_content_client/*.cc;ui/views/readme.md"

when running presubmit on either file individually found nothing.

This change was tested by getting CheckChangeLintsClean to print the
files that it was asked to analyze, making the behavior quite obvious.

This bug appears to have existed for about thirteen years, but only
triggers in rare cases, and even then the incorrect behavior was almost
impossible to notice.

One of the tests was inadvertently testing the broken AffectedFiles
behavior and another seemed to be requiring it to handle '.'
correctly which I'm not sure we want to support.

Bug: 1309977
Change-Id: Ibdc39981d69664b03448acb228d4ab05b49436f6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3632034
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
3 years ago
dpapad 443d9135cc Include TypeScript .ts files in DEFAULT_FILES_TO_CHECK.
These were previously erroneously skipped resulting in missing
violations.

Fixed: 1322170
Change-Id: I62d27f60bc1761f1f8809c6a7db2ca5f4e05e120
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3626289
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Auto-Submit: Demetrios Papadopoulos <dpapad@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Demetrios Papadopoulos <dpapad@chromium.org>
3 years ago
Aravind Vasudevan fb8cf9cc78 Add --protocol-override to fetch.py
This CL adds a non-functional `--protocol` flag to fetch.py. This will be used in the follow-up CL to update fetching protocol.

Bug: 1322156
Change-Id: I7eeebb9993face20bb671d6942ee23fd04802d8b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3609453
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
3 years ago
Takuto Ikuta 381db68adc autoninja: add simple test
This is to prevent revert like https://crrev.com/c/3607513

Bug: 1317620
Change-Id: I6ab7aba5f92719bd573d22d90358f58e48aeb10c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3607514
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
3 years ago
Joanna Wang 583ca66091 [depot_tools] Give user option to clear cl issue for merged changes.
Test:
Merged: https://chromium-review.googlesource.com/c/infra/gerrit-plugins/buildbucket/+/3600858

Created new CL in same branch:
https://chromium-review.googlesource.com/c/infra/gerrit-plugins/buildbucket/+/3609588

Bug: 1312511

Change-Id: I20ccfc978fd84c4c68f71007cf636fc1ee8e3d4d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3609115
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
3 years ago
Bruce Dawson db8622bc6b Ensure _PresubmitResult._message is text
It is easy to get type confusion and end up passing a list as the
message parameter to _PresubmitResult. This error will not be detected
until the end of the run - perhaps hours later - when all evidence of
where the list came from is lost.

This change ensures that the message parameter is a string. If it is not
then the exception that is thrown should allow quick identification of
the problematic code.

This also fixes a presubmit unit test that passed None as the message.
We could support that but I don't think that we should.

Bug: 1309977
Change-Id: Ifb1d5100d47922b0ebd8bb834caa6fbba690b43c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3566436
Reviewed-by: Jochen Eisinger <jochen@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
3 years ago
Josip Sokcevic 017544dc14 Add git cl presubmit --files
Chromium's presubmits contain a lot of latent errors - presubmit errors
that will trigger the next time a file is modified, but have been
waiting for years. Flushing those out with "git cl presubmit --all"
works poorly because it exercises all presubmits simultaneously, which
is too slow and triggers too many failures. Adding a --files option lets
small areas of the tree or specific file types to be exercised in a
controlled manner.

Bug: 1311697
Change-Id: I36ec6a759a80000d6ed4a8cc218ece327d45f8d6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3559174
Auto-Submit: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
3 years ago
Aravind Vasudevan 075cd76fe6 Update argument parsing to use argparse
This CL updates argument parsing mechanism from using a custom implementation + optparse to argparse. It is intended to clear tech debt for future changes.

Change-Id: I06ba5d3c532638a7970be960e02ebbafbea9dcb0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3541479
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
3 years ago
Josip Sokcevic 0399e1762c Add a basic depot_tools version information
depot_tools has no versioning. It's hard to know if reported issues were
caused due to outdated depot_tools or actual unresolved bug.

This CL adds basic information about depot_tools version and it's
included in presubmit failure.

R=aravindvasudev@google.com, gavinmak@google.com

Change-Id: If8577c0826063693a7278a57a0cce629d4b1325f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3541061
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
3 years ago
Ravi Mistry c848a4ed33 Fix host and repo parsing for download-topics
Previous code was incorrectly assuming that the repo name always matched the host name.

Bug: chromium:1298922
Change-Id: Ic90ed585503d721f4d771b4f390a04c7f5b26622
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3507359
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Ravi Mistry <rmistry@chromium.org>
3 years ago
Louis Romero 4ca9e1c783 Remove extraneous newlines from presubmit output
Prior to this CL, we have:

```
Running Python 2 presubmit upload checks ...

Python 2 presubmit checks passed.
Running Python 3 presubmit upload checks ...

Presubmit checks took 10.2s to calculate.

Python 3 presubmit checks passed.
 ios/chrome/browser/... | 20 +++++++++-----------
...
```

The first two checks are harder to parse with the newlines. This CL removes
them:

```
Running Python 2 presubmit upload checks ...
Python 2 presubmit checks passed.

Running Python 3 presubmit upload checks ...
Presubmit checks took 10.2s to calculate.
Python 3 presubmit checks passed.

 ios/chrome/browser/... | 20 +++++++++-----------
...
```

Change-Id: Iaf7ac4b203c401f60d74884fd9bdd3ba4ee12c20
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3512873
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Louis Romero <lpromero@chromium.org>
3 years ago
Ravi Mistry ecda782811 Reland "Reland "Add support for Gerrit topics in gclient syncs""
This is a reland of fc9a40e3c6

Hopefully the cause of the 2nd revert was fixed in https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3489327

Original change's description:
> Reland "Add support for Gerrit topics in gclient syncs"
>
> This is a reland of 0f13273f1f
>
> Hopefully the cause of the original revert was fixed in https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3480835
>
> Original change's description:
> > Add support for Gerrit topics in gclient syncs
> >
> >
> > If the new flag "--download-topics" is specified with a "--patch-ref" then:
> > * Finds the topic of the Gerrit change.
> > * Finds all open changes in the same repo as the Gerrit change.
> > * Cherrypicks all changes locally.
> >
> > This functionality can be used by developers and bots to apply all changes with the same topic in the checkout to be tested at the same time (similar to how Android's TreeHugger handles topics).
> >
> >
> > Tested by:
> >
> > * Running the new unit test with `python gclient_scm_test.py GerritChangesTest.testDownloadsTopics` from the `tests/` directory.
> >
> > * Running an end-to-end test with `DEPOT_TOOLS_UPDATE=0 gclient sync --patch-ref "skia@d831da5b8ac2d257c5b0cf2ec6645a148f05e662:refs/changes/17/505217/2" --download-topics` in a skia checkout.
> >
> >
> > Bug: chromium:1298922
> > Change-Id: Ieace5e27fbc9c5d0ea90a037bf80a95062c1b164
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3444003
> > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> > Commit-Queue: Ravi Mistry <rmistry@chromium.org>
>
> Bug: chromium:1298922
> Change-Id: I80747d797234bba06c17ef5c5e85b310281922c4
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3484976
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Ravi Mistry <rmistry@chromium.org>

Bug: chromium:1298922
Change-Id: I21d7251bafff808b1144d6e522fa9f384f4541bf
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3490488
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Ravi Mistry <rmistry@chromium.org>
3 years ago
Josip Sokcevic 368f980b6d Revert "Reland "Add support for Gerrit topics in gclient syncs""
This reverts commit fc9a40e3c6.

Reason for revert: at least one tryjob if failing:
Gerrit Plugins Tester

Original change's description:
> Reland "Add support for Gerrit topics in gclient syncs"
>
> This is a reland of 0f13273f1f
>
> Hopefully the cause of the original revert was fixed in https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3480835
>
> Original change's description:
> > Add support for Gerrit topics in gclient syncs
> >
> >
> > If the new flag "--download-topics" is specified with a "--patch-ref" then:
> > * Finds the topic of the Gerrit change.
> > * Finds all open changes in the same repo as the Gerrit change.
> > * Cherrypicks all changes locally.
> >
> > This functionality can be used by developers and bots to apply all changes with the same topic in the checkout to be tested at the same time (similar to how Android's TreeHugger handles topics).
> >
> >
> > Tested by:
> >
> > * Running the new unit test with `python gclient_scm_test.py GerritChangesTest.testDownloadsTopics` from the `tests/` directory.
> >
> > * Running an end-to-end test with `DEPOT_TOOLS_UPDATE=0 gclient sync --patch-ref "skia@d831da5b8ac2d257c5b0cf2ec6645a148f05e662:refs/changes/17/505217/2" --download-topics` in a skia checkout.
> >
> >
> > Bug: chromium:1298922
> > Change-Id: Ieace5e27fbc9c5d0ea90a037bf80a95062c1b164
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3444003
> > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> > Commit-Queue: Ravi Mistry <rmistry@chromium.org>
>
> Bug: chromium:1298922
> Change-Id: I80747d797234bba06c17ef5c5e85b310281922c4
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3484976
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Ravi Mistry <rmistry@chromium.org>

Bug: chromium:1298922
Change-Id: I845321941157ab55d026488b7ce59787ba5e57f2
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3488245
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
3 years ago
Ravi Mistry fc9a40e3c6 Reland "Add support for Gerrit topics in gclient syncs"
This is a reland of 0f13273f1f

Hopefully the cause of the original revert was fixed in https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3480835

Original change's description:
> Add support for Gerrit topics in gclient syncs
>
>
> If the new flag "--download-topics" is specified with a "--patch-ref" then:
> * Finds the topic of the Gerrit change.
> * Finds all open changes in the same repo as the Gerrit change.
> * Cherrypicks all changes locally.
>
> This functionality can be used by developers and bots to apply all changes with the same topic in the checkout to be tested at the same time (similar to how Android's TreeHugger handles topics).
>
>
> Tested by:
>
> * Running the new unit test with `python gclient_scm_test.py GerritChangesTest.testDownloadsTopics` from the `tests/` directory.
>
> * Running an end-to-end test with `DEPOT_TOOLS_UPDATE=0 gclient sync --patch-ref "skia@d831da5b8ac2d257c5b0cf2ec6645a148f05e662:refs/changes/17/505217/2" --download-topics` in a skia checkout.
>
>
> Bug: chromium:1298922
> Change-Id: Ieace5e27fbc9c5d0ea90a037bf80a95062c1b164
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3444003
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Ravi Mistry <rmistry@chromium.org>

Bug: chromium:1298922
Change-Id: I80747d797234bba06c17ef5c5e85b310281922c4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3484976
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Ravi Mistry <rmistry@chromium.org>
3 years ago
Josip Sokcevic 63343b07f3 Revert "Add support for Gerrit topics in gclient syncs"
This reverts commit 0f13273f1f.

Reason for revert: breaks codesearch recipes and autorollers

crbug.com/1298961

Original change's description:
> Add support for Gerrit topics in gclient syncs
>
>
> If the new flag "--download-topics" is specified with a "--patch-ref" then:
> * Finds the topic of the Gerrit change.
> * Finds all open changes in the same repo as the Gerrit change.
> * Cherrypicks all changes locally.
>
> This functionality can be used by developers and bots to apply all changes with the same topic in the checkout to be tested at the same time (similar to how Android's TreeHugger handles topics).
>
>
> Tested by:
>
> * Running the new unit test with `python gclient_scm_test.py GerritChangesTest.testDownloadsTopics` from the `tests/` directory.
>
> * Running an end-to-end test with `DEPOT_TOOLS_UPDATE=0 gclient sync --patch-ref "skia@d831da5b8ac2d257c5b0cf2ec6645a148f05e662:refs/changes/17/505217/2" --download-topics` in a skia checkout.
>
>
> Bug: chromium:1298922
> Change-Id: Ieace5e27fbc9c5d0ea90a037bf80a95062c1b164
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3444003
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Ravi Mistry <rmistry@chromium.org>

Bug: chromium:1298922
Bug: chromium:1298961
Change-Id: I88c56cd68372bad09b612de7de1a45f9a0c6c681
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3474793
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
3 years ago
Ravi Mistry 0f13273f1f Add support for Gerrit topics in gclient syncs
If the new flag "--download-topics" is specified with a "--patch-ref" then:
* Finds the topic of the Gerrit change.
* Finds all open changes in the same repo as the Gerrit change.
* Cherrypicks all changes locally.

This functionality can be used by developers and bots to apply all changes with the same topic in the checkout to be tested at the same time (similar to how Android's TreeHugger handles topics).


Tested by:

* Running the new unit test with `python gclient_scm_test.py GerritChangesTest.testDownloadsTopics` from the `tests/` directory.

* Running an end-to-end test with `DEPOT_TOOLS_UPDATE=0 gclient sync --patch-ref "skia@d831da5b8ac2d257c5b0cf2ec6645a148f05e662:refs/changes/17/505217/2" --download-topics` in a skia checkout.


Bug: chromium:1298922
Change-Id: Ieace5e27fbc9c5d0ea90a037bf80a95062c1b164
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3444003
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Ravi Mistry <rmistry@chromium.org>
3 years ago
Josip Sokcevic e293d3dce9 Support py3 in post upload presubmit hooks
Currently, post upload presubmit hooks are exlusively executed with py2,
regardless of USE_PYTHON3 magic variable. This change adds py3 support
in the same fasion as regular presubmit hooks.

R=aravindvasudev@google.com

Fixed: 1297712
Change-Id: Ib464f8563e4135a63fc48692d27c8692fe1f630b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3469285
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
3 years ago
Josip Sokcevic a59e9ea222 Revert "Remove unused functionality in bot_update.py"
This reverts commit 8b64ddec2a.

Reason for revert: it appears we need this, since .DEPS.git
specifies custom dependencies needed for tryjob.

Original change's description:
> Remove unused functionality in bot_update.py
>
> .DEPS.git has been removed in 2014 and no longer supported. It was
> deleted in https://codereview.chromium.org/497313003 (August 2014). Last
> entry in chromium/src is from tag 44.0.2403.47 in June 2015.
>
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited
> Recipe-Nontrivial-Roll: chrome_release
> Recipe-Nontrivial-Roll: chromiumos
> Recipe-Nontrivial-Roll: infra
> Change-Id: I3ffb8813af867df959098c02d399c8d745894aae
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3439573
> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>

Change-Id: I36bcd352a984b4b762096197a384e86347418290
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3448707
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
3 years ago
Josip Sokcevic 8b64ddec2a Remove unused functionality in bot_update.py
.DEPS.git has been removed in 2014 and no longer supported. It was
deleted in https://codereview.chromium.org/497313003 (August 2014). Last
entry in chromium/src is from tag 44.0.2403.47 in June 2015.

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Change-Id: I3ffb8813af867df959098c02d399c8d745894aae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3439573
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
3 years ago
Aravind Vasudevan c5f0cbb865 Use pylint 2.7 for depot_tools
This includes a few fixes for specific errors, and disables several new
warnings introduced in this version, in order to allow for an incremental migration.

Bug:1262286
Change-Id: I4b8f8fc521386419a3121bbb07edc8ac83170a94
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3413679
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
3 years ago
Josip Sokcevic 42c5bbbc96 Revert "Use pylint 2.7 for depot_tools"
This reverts commit 22bf605bb6.

Reason for revert: breaks gclient sync

Original change's description:
> Use pylint 2.7 for depot_tools
>
> This includes a few fixes for specific errors, and disables several new
> warnings introduced in this version, in order to allow for an incremental migration.
>
> Bug:1262286
> Change-Id: Ie97d686748c9c952e87718a65f401c5f6f80a5c9
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3400616
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>

Bug: 1262286
Change-Id: Ieb946073c7886c7bf056ce843a5a48e82becf7a5
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3413672
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
3 years ago
Aravind Vasudevan 22bf605bb6 Use pylint 2.7 for depot_tools
This includes a few fixes for specific errors, and disables several new
warnings introduced in this version, in order to allow for an incremental migration.

Bug:1262286
Change-Id: Ie97d686748c9c952e87718a65f401c5f6f80a5c9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3400616
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
3 years ago
Gavin Mak e9ea0d5c0d Fix depot_tools tests when running without py2
Currently, running git cl presubmit fails without python 2 installed.
This CL replaces instances of 'python' in fetch_test.py with the generic
sys.executable and fixes a small error testRebaseUpdate due to an extra
period.

Bug: 1289839
Change-Id: Ib5802545af70fe0886448397fa76058bd2036527
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3407864
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
3 years ago
Takuto Ikuta 3dc0172a6b fix ninjalog_uploader.py
This is to fix https://crbug.com/1288639#c5

Bug: 1288639
Change-Id: Ife224fb5d5fe4faa4a78785828d4fa488fda3c6e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3403475
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
3 years ago
Takuto Ikuta a657331e90 ninjalog_uploader: use goma_auth to detect googler
Bug: 1288639
Change-Id: I447e2f66603ffb8d68599dcf22023fd7857dc4fd
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3400398
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
3 years ago
Andrew Grieve cb340f5b7b Fix git_common_test.py: git checkout -tb -> git checkout -t -b
git now takes an optional parameter for -t

Bug: 1288010
Change-Id: I0b7b212229f8df846a2d80a1632387982fd7358e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3395441
Auto-Submit: Andrew Grieve <agrieve@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Dirk Pranke <dpranke@google.com>
3 years ago
Aravind Vasudevan aae6725ff0 Fix recursedeps to be overridden using custom_deps in windows
R=gavinmak@google.com
Change-Id: Ibae42b9533b0da526c3a534cd327806611216327
Bug: 1215885
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3335580
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
3 years ago
Alex Turner 30ae637731 Accept URLs missing schemes in git cl patch
Currently, calling a command like "git cl patch crrev.com/c/1234567"
will fail with "git cl: error: Invalid issue ID or URL." as no scheme is
supplied (i.e. "git cl patch https://crrev.com/c/1234567" is considered
valid).

This cl removes the requirement for supplying a scheme. Instead, the
script attempts to add "https://" where it would've thrown an error
previously. If the resulting string is valid, it is used. Otherwise, an
error will still be thrown.

Bug: 1223200
Change-Id: I24ee4df48b5f5d434f3abe270f5c3e793c347cc8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2983443
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Alex Turner <alexmt@chromium.org>
3 years ago
Takuto Ikuta 26bfecdd07 include filename in LUCI config validation
Fixed: 1254615
Change-Id: Ic6d5350c7a89a597c1126ec05235a3e63eb2ad09
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3312231
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Anthony Polito <apolito@google.com>
Reviewed-by: Anthony Polito <apolito@google.com>
3 years ago
Josip Sokcevic a29f589a15 Make fix_encoding_test py3 compatible
And run it in presubmit checks

R=gavinmak@google.com

Change-Id: If2a8848f6ce85c466f45d9bf0f4fca6b4d75312f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3296677
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
3 years ago
Josip Sokcevic b6c1ed40d5 Make presubmit_unittest py3 compatible
And run it in presubmit checks

R=apolito@google.com

Change-Id: I1521fe4dd9fd5fe391459a00a91f03399b15b36a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3292211
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Anthony Polito <apolito@google.com>
4 years ago
Josip Sokcevic 6635baff5b Make CannedChecksUnittest py3 compatible
R=apolito@google.com

Change-Id: I24610ca05536b4ac6eed2c0a33ef411650b46dd0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3292089
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago