Flag will be used to test out the formatter, and eventually be removed
and made default (or removed and abandon google-java-format)
Bug: 1462204
Change-Id: I3dc9a77fcabc7513674f5db5eab6979a97d2b315
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4864924
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Andrew Grieve <agrieve@chromium.org>
Did you know that args.gn files can have import statements and
conditionals? I did not, but apparently some developers make use of both
of these.
Supporting import statements is not too hard, so this change adds this
support. Supporting conditionals is possible, but risks turning
autoninja into a turing complete language which is more than I think we
want to do.
This doesn't use the similar code in tools/mb/mb.py because that
code is complex, and relies on the script location to find the src
directory.
This change also updates two of the existing test conditionals that
were not quite sufficient - ninja/autoninja default to num-cores
plus 2 so > cpu_count() is actually not sufficient to prove
anything.
Bug: 1482404
Change-Id: I0539d8068af59d11927cbdad260278a24ab912e6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4864898
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
This CL introduces bash autocompletion for some specific git commands
contributed by depot_tools (git new-branch and git reparent-branch).
Change-Id: I8208ef7d7d974765c468d01fa94c4ed334b01bb3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4859465
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Dominic Battre <battre@chromium.org>
No behavior change. This is to make it easier to add new formatters.
Bug: 1462204
Change-Id: Ifc9c46ad60fe5024f5dfb0cf781ff468b2cc1044
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4863139
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Andrew Grieve <agrieve@chromium.org>
This CL makes autoninja raise the number of allowed open file
descriptors to its hard limit, instead of a random value.
Also, independently of whether this operation was successful or not, the
ninja j value is capped at 80% of the current value of allowed open file
descriptors. This approximately matches the previous value of 200 for
the default value of 256 in macOS, but it's also more future-proof.
Lastly, this changes are also applied on Linux.
Change-Id: Idf2cd08384fe9f2bc699293d7062122590284dba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4852717
Commit-Queue: Henrique Ferreiro <hferreiro@igalia.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
If there are dirty submodules, git commit -a will commit those which is
not desired. Turns out we don't need to use -a, since reset --soft
stages all changes anyways and we can just use git commit. Note that
there is a check prior to commit to ensure we are not creating an empty
commit.
R=jojwang
Fixed: 1478668
Change-Id: Iaa1ff8e638b7431511e6e194ad59e3c4adb39deb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4858836
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
siso doesn't generate .ninja_log files so post_build_ninja_summary.py
doesn't work, but it _does_ generate siso_metrics.json files which can
be used to generate similar reports to post_build_ninja_summary.py.
Therefore, when a siso_metrics.json file is detected the siso metrics
summary command is invoked, thus preserving (more or less) the old
behavior.
Bug: b/293657720
Change-Id: I084402ca4dca9895b502ab336fa7b45b770f4768
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4854377
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
This CL changes what is considered valid versioning info. Instead of
both Date and Revision being required if Version was unknown, now only
one of Date or Revision has to be specified.
Bug: b:277147404
Change-Id: Iedb06e2d55f0cd0ef0a2931013a2a52b15befd75
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4852699
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
This reverts commit cea35a3f77.
Reason for revert: b/298657697 is unrelated to this
Original change's description:
> Revert "Remove unnecessary notify=None from git cl upload."
>
> This reverts commit 7688e78450.
>
> Reason for revert: afaict there is a gerrit notifications bug b/297928626 unrelated to this change but i'm reverting this anyway and relanding when the gerrit bug is resolved.
>
> Original change's description:
> > Remove unnecessary notify=None from git cl upload.
> >
> > This is a NOOP change for all cases except for "Publish comments on push" which gets fixed with this CL.
> > Our hosts have the notify on each patchset turned off.
> >
> > Bug: 1472724
> > Change-Id: I3672c383b1e4ca1f6243c9b9d2c906473f5037d9
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4797981
> > Reviewed-by: Gavin Mak <gavinmak@google.com>
> > Commit-Queue: Joanna Wang <jojwang@chromium.org>
>
> Bug: 1472724
> Change-Id: I3de1a25fe84a21a2adb8b4bbddb460dd45530418
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4817767
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Joanna Wang <jojwang@chromium.org>
Bug: 1472724
Change-Id: I63cdb99111ae89eb6edd5172d827f84b314c015e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4852217
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
python3 is the only supported version of python in depot_tools.
Changes include:
* updating python version requirements
* removing third_party/six
Bug: 1475402
Change-Id: Ib86231413fcac494ceb27d1783e53b9d86533c26
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4845292
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
These are currently produced by Siso, until b/298594790 is addressed.
Before:
$ post_build_ninja_summary.py -C out/fastbuild-siso-reclient
Traceback (most recent call last):
File "/usr/local/google/home/philwo/depot_tools/post_build_ninja_summary.py", line 366, in <module>
sys.exit(main())
^^^^^^
File "/usr/local/google/home/philwo/depot_tools/post_build_ninja_summary.py", line 356, in main
entries = ReadTargets(log, False)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/google/home/philwo/depot_tools/post_build_ninja_summary.py", line 123, in ReadTargets
assert header == '# ninja log v5\n', \
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: unrecognized ninja log version ''
After:
$ post_build_ninja_summary.py -C out/fastbuild-siso-reclient
<nothing>
Bug: b/298594790
Fixed: b/297349353
Change-Id: I10d4613e7386707276003fe0fd05cb5b0914be46
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4846349
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Philipp Wollermann <philwo@google.com>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
This is already covered in the PDD for googlers. We are already gathering hostnames in cloud monitoring mertrics
Bug: b/294945709
Change-Id: Idfb5b8d3b7cf64edd60092f6bd49a3f9f1950d15
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4848094
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Ben Segall <bentekkie@google.com>
This will help analyze startup metrics
Bug: b/294945709
Change-Id: Ia397dbebef7cc30b49c614dda51e56f482b6145b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4847325
Commit-Queue: Ben Segall <bentekkie@google.com>
Auto-Submit: Ben Segall <bentekkie@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
This is a reland of https://crrev.com/c/4719827
- Fixed the issue for cipd wrapper. Now all environment variables should
be perserved when invoking cipd: https://crrev.com/c/4669637
- Isolate python when creating virtualenv: https://crrev.com/c/4780372
1. virtualenv field in the spec is ignored.
2. --vpython-tool removed support for delete and help subcommands.
3. --vpython-tool installed removed support for naming venv.
4. removed support for -vpython-interpreter.
5. removed support for searching interpreters in host PATH.
6. python 2.7 is available only if the binary is invoked as `vpython`.
7. fixed a bug that passes invalid vpython arguments to the script,
which may be silently ignored.
8. python_version in the vpython_spec must specify a minor version
(python_version: "3" is not valid anymore).
9. vpython now requires the cipd binary to be present in PATH, which is
true already when using Swarming or depot_tools.
Bug: 1415212
Change-Id: I05fddbb10c7c9bcfa6f5390710fab28fe8d89131
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4844251
Commit-Queue: Chenlin Fan <fancl@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
I suspect the reason of INFRA_FAILURE in https://crbug.com/1477933#c10
is that cipd is busy with downloading on the packages (it's also
possible that cipd is stucked on one or more packages) and since the
log-level is error, it won't ouptut anything until it completes.
However, from bot_update PoV, gclient is considered as a stale process
because it won't write anything to stdout in 20min so it decide to kill
the process. This CL fix the issue by changing the log-level to info
instead.
R=sokcevic
Bug: 1477933
Change-Id: I2fa05bcfe7f63debeda3371e755b5e7c6d011f12
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4844365
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Leave the recipes/ code at 2 space to match the rest of the recipes
project in other repos.
Reformatted using:
files=( $(
git ls-tree -r --name-only HEAD | \
grep -Ev -e '^(third_party|recipes)/' | \
grep '\.py$';
git grep -l '#!/usr/bin/env.*python' | grep -v '\.py$'
) )
parallel ./yapf -i -- "${files[@]}"
~/chromiumos/chromite/contrib/reflow_overlong_comments "${files[@]}"
The files that still had strings that were too long were manually
reformatted because they were easy and only a few issues.
autoninja.py
clang_format.py
download_from_google_storage.py
fix_encoding.py
gclient_utils.py
git_cache.py
git_common.py
git_map_branches.py
git_reparent_branch.py
gn.py
my_activity.py
owners_finder.py
presubmit_canned_checks.py
reclient_helper.py
reclientreport.py
roll_dep.py
rustfmt.py
siso.py
split_cl.py
subcommand.py
subprocess2.py
swift_format.py
upload_to_google_storage.py
These files still had lines (strings) that were too long, so the pylint
warnings were suppressed with a TODO.
auth.py
gclient.py
gclient_eval.py
gclient_paths.py
gclient_scm.py
gerrit_util.py
git_cl.py
presubmit_canned_checks.py
presubmit_support.py
scm.py
Change-Id: Ia6535c4f2c48d46b589ec1e791dde6c6b2ea858f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4836379
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Reformat this dir by itself to help merging with conflicts with other CLs.
Reformatted using:
parallel ./yapf -i -- tests/*.py
~/chromiumos/chromite/contrib/reflow_overlong_comments tests/*.py
These files still had lines (strings) that were too long, so the pylint
warnings were suppressed with a TODO.
tests/bot_update_coverage_test.py
tests/cipd_bootstrap_test.py
tests/gclient_eval_unittest.py
tests/gclient_git_smoketest.py
tests/gclient_scm_test.py
tests/gclient_smoketest.py
tests/gclient_test.py
tests/gclient_transitions_smoketest.py
tests/gclient_utils_test.py
tests/git_cl_test.py
tests/git_hyper_blame_test.py
tests/git_rebase_update_test.py
tests/lockfile_test.py
tests/metrics_test.py
tests/presubmit_canned_checks_test.py
tests/presubmit_unittest.py
tests/roll_dep_test.py
Change-Id: I8fed04b4ba81d54b8f45da612213aad27a9e1a2c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4842592
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Reformat this dir by itself to help merging with conflicts with other CLs.
Reformatted using:
parallel ./yapf -i -- testing_support/*.py
~/chromiumos/chromite/contrib/reflow_overlong_comments testing_support/*.py
The files that still had strings that were too long were manually
reformatted.
testing_support/coverage_utils.py
testing_support/fake_repos.py
testing_support/git_test_utils.py
testing_support/presubmit_canned_checks_test_mocks.py
Change-Id: I4726a4bbd279a70bcf65d0987fcff0ff9a231386
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4842593
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Reformat this dir by itself to help merging with conflicts with other CLs.
Reformatted using:
parallel ./yapf -i -- fetch_configs/*
~/chromiumos/chromite/contrib/reflow_overlong_comments fetch_configs/*
The 2 files that still had strings that were too long were manually
reformatted.
fetch_configs/config_util.py
fetch_configs/devtools-internal.py
Change-Id: Ic8aaf41fa462342743a39166da98c4e915fadd44
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4844310
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
python3 is the only supported version of python in depot_tools.
Remove py2 support from files including:
* cpplint.py
* subprocess2.py
* many tests and testing_support files
Bug: 1475402
Change-Id: I67a98188bc13c4dc119e6158a37bd236bfd6ea70
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4824474
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Scott Lee <ddoman@chromium.org>
Upstream reformatted. We want it. We get it.
Change-Id: I64450e60dc0c1bb012720b586e68b48456ca20ed
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4842913
Commit-Queue: Sergey Frolov <sfrolov@google.com>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Sergey Frolov <sfrolov@google.com>
All files in metadata/ are new, so they should follow the PEP-8 style.
Change-Id: I5d8424536c3d7b703e6b8087e0e2d70c06a1549c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4834909
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Rachael Newitt <renewitt@google.com>
Adds script metadata/scan.py which can be used to search for and
validate Chromium dependency metadata files, given a repository
root directory.
Bug: b:277147404
Change-Id: Ibde0eeb7babe0b1e3f9c7f887bece629d390974a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4823596
Commit-Queue: Anne Redulla <aredulla@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
This is a reland of commit e6f40ea034
Original change's description:
> Check if DEPS git is not in git submodules
>
> If a new git dependency is added to DEPS file, presubmit check should
> fail if there's no corresponding git submodule entry if git_dependencies
> is set to SYNC.
>
> R=jojwang@google.com
>
> Bug: 1476115
> Change-Id: I0fdebb036c129c2f97524b86ee4d70c07e5b0091
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4818792
> Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
R=jojwang@google.com
Bug: 1476115
Change-Id: I4cd6e541aabb5d8be883e15e5693c4ad9085bcad
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4825584
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Most of the changes are straightforward typo fixes.
Two more significant changes:
- the entire ChangeDescription class was moved earlier in the file so
that it is defined before any type annotations that reference it.
- annotations that use list[...] have been changed to use List[...].
While this is supported >= python3.9, it seems some parts of the
Python toolchain (e.g. Python formatting) do not support this yet.
Change-Id: Ifb60f10e2424f79a19229328d19d9fd1d0e87dea
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4824618
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Daniel Cheng <dcheng@chromium.org>
python3 is the only supported version of python in depot_tools.
Bug: 1475402
Change-Id: Ic925af3a79311b11a953f1d0d4602754488a9de3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4824480
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
python3 is the only supported version of python in depot_tools.
Bug: 1475402
Change-Id: I71db631b5556525dd4932134679c70cacd205a90
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4824616
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
This CL adds a new function `CheckChromiumDependencyMetadata` in
`presubmit_canned_checks.py`. It can be used to check that files satisfy
the format defined by `README.chromium.template`
(https://chromium.googlesource.com/chromium/src/+/main/third_party/README.chromium.template).
The code for metadata validation can be found in `//metadata`. Note that
all metadata validation issues will be returned as warnings only for
now, while the quality of metadata is being uplifted.
Bug: b:277147404
Change-Id: Iacf1b3a11219ab752549f6dc6e882c93c0fbe780
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4812578
Commit-Queue: Anne Redulla <aredulla@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Occasionally, users can get a branch into a state where `git cl upload`
takes a long time because it is operating on an unexpectedly large diff.
One way where this can happen is when the local view of origin/main is
behind and someone force-patches in a CL based on a newer origin/main:
git cl upload will happily consider all the origin/main commits it
does not have in its local view of origin/main to simply be part of the
diff.
As this is a rather frustrating user experience, make it a bit easier to
realize when this sort of thing is happening by summarizing the number
of commits `git cl upload` is processing.
One alternative would be to stats about the diff; however, calculating
the number of commits is considerably faster than calculating the actual
diff. A quick local test shows that calculating the diff for 10k commits
takes nearly 20 seconds, while calculating the number of commits takes a
150 milliseconds.
Change-Id: I0e8706f164f6bf669f36f4792401589644be38b5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4819796
Commit-Queue: Daniel Cheng <dcheng@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
This was added many years ago to support migration of in-flight CLs
when we reformatted Blink C++ code. This should no longer be needed.
Bug: 574611
Change-Id: If60e89562cef2231b3e9c3e176b81d34250f6c4a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4823890
Commit-Queue: Daniel Cheng <dcheng@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
recipe-roller
Yiwei Zhang
Andrew Grieve
Bruce Dawson
Dominic Battre
Ben Segall
Henrique Ferreiro
Josip Sokcevic
Joanna Wang
Chan Li
Anne Redulla
Gavin Mak
Philipp Wollermann
Chenlin Fan
Mike Frysinger
Yue She
Aravind Vasudevan
Daniel Cheng