git_cache populate can get interrupted midway (e.g. by LUCI CV sending a
signal that build is no longer needed). When that happens, a git mirror
may be in a state where some commits are available, but cloning such
repositry results in an empty repsitory.
This change ensures that the initial fetch operation finished
successfully.
R=ddoman@google.com
Bug: 1517944
Change-Id: I1ee860860877dbfff7a444b45fe4515fe26b248c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5219032
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Scott Lee <ddoman@chromium.org>
+ Change scm GetAllFiles back to returning all files
+ should be safe to do since it's only used by presubmit code
+ add ignore_submodules option to CaptureStatus
+ It's used by other parts of depot_tools code
+ Add AffectedSubmodules to return submodules only
+ AffectedFiles still only returns non-submodule files.
Bug: 1475770
Change-Id: I457ad96099fb20eff4cc1a4fc84f59e7ae707abe
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5216005
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
The previous binary was so old that it does not work on recent
versions of macOS.
Note that `cipd selfupdate-roll` requires the cipd package to exist
at the given tag for all platforms that the package has ever been
built for. Therefore, I'm using a revision after the latest platform
was added to the builders.
Change-Id: I865c31a5e3823ffc344bcb6585a29935ef6214a6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5191588
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Auto-Submit: Brian Ryner <bryner@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
The cache won't be used if ResolveCommit throws an exception or when
non-sha1 is used.
This optimization saves about 7% of overall gclient sync on
chromium/src.
R=gavinmak@google.com
Change-Id: I9c36d937dc7b8553818888e2e9c4eecd029a978c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5147496
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Assigning os.environ the result of os.environ.copy() replaces it
with a standard dict. This breaks reflection into the actual
environment, as well as features such as case-insensitivity on
Windows, leading to undefined standard library behavior.
R=bryner, gavinmak
Bug: 1511316
Change-Id: I63d026387104ae92669256567854f8bf9d2397a2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5133887
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Brian Ryner <bryner@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
This is to remove 1~3 seconds from no-op build using autoninja.
Bug: b/309720176
Change-Id: I2c15a43517d5a99eae794015c18c884a3f3fdef4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5092847
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Philipp Wollermann <philwo@chromium.org>
Reviewed-by: Scott Lee <ddoman@chromium.org>
This change updates the `_CheckClean()` fn to use `git status` instead
of `git update-index` and `git diff-index`. The `_CheckClean()` fn is
run during every update and this change reduces the subprocess calls
made by it by half.
Change-Id: Ie9a23b0bce748bec4cac88df09655569e88e4841
Bug: 1501984
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5076224
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
From my recent go/gclient-sync-no-ops analysis, I noticed that
HostArch() is repeated called within get_builtin_vars() fn. The
`platform.architecture()` call made in arm-based systems and some x64 is
expensive and adds to gclient sync's runtime when repeatedly called.
When caching HostArch(), I noticed 22.4% speed-up in incremental no-op
gclient sync when run on arm-based machine.
Change-Id: I962c4fb0879d2931268f5bec1678ef782c56e7f2
Bug: 5076224
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5076226
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
This disallows Googler to use non-google account with reclient and
siso from corp machine.
Bug: b/309720176
Change-Id: I8077eff8bbf47f579462e1fefa3609a5d492a013
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5035266
Reviewed-by: Philipp Wollermann <philwo@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
The code rotating the old build logs expects to delete directories
(since it uses rmtree) but tries to delete all items returned by
os.listdir(...) which may includes symbolic links or regular files.
Change it to only consider the old build logs directories instead.
Based on crrev/5028323, with fix to unit test.
Fixed: b/310900283
Change-Id: I4c618e6618c0193331c063028ebf02d8c4e7baee
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5038467
Auto-Submit: Michael Savigny <msavigny@google.com>
Reviewed-by: Ramy Medhat <abdelaal@google.com>
Commit-Queue: Michael Savigny <msavigny@google.com>
The package is deprecated and will be removed in py 3.12.
R=ddoman@google.com
Bug: 1500415
Change-Id: Ia130c556ad3f2135fac4a9b1d9bb4ecc088cfe9d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5030818
Reviewed-by: Scott Lee <ddoman@chromium.org>
Reviewed-by: Jonathan Nieder <jrn@google.com>
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Scott Lee <ddoman@chromium.org>
Commit-Queue: Jonathan Nieder <jrn@google.com>
.git suffix is not well supported by the systems that utilize
submodules. Since some tools expect DEPS to have .git suffix, it's not
possible to simply update DEPS without breaking the tools.
This patch updates `gclient gitmodules` to omit .git suffix and trailing
slashes. However, if `.gitmodule` already exists, we are expecting the
tool to generate the same result if DEPS didn't change.
R=aravindvasudev@google.com
Bug: b/303296048
Change-Id: I30497b9d5f4e9681e16b4a913151de0f4a7c9545
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5008942
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
On unsupported platforms, luci-auth is not downloaded from CIPD and
hence the scripts relying on gsutil.py fail. Given we aren't using
luci-auth in unsupported platforms anyway, skip wrapping gsutil with
luci-auth on unsupported platforms.
Curently, only "aix" is added to the unsupported platforms list
eventhough we have other unsupported platforms too. This list will be
expanded if necessary.
R=yiwzhang
Change-Id: I70f6fceddc672c76bbea9a108a75b84662e26459
Fixed: 1467752
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4989412
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Instead of printing a command-line, we just directly call into the respective main functions from Python. This saves spawning another interpreter and prevents things that can go wrong from having to quote, unquote, split and tunnel arguments through shells.
Part of my bigger auto{ninja,siso} refactoring.
Tested:
- Handling of the ^^ suffix on Windows still works correctly.
- Handling of error codes - i.e.; making sure
"autoninja base_unittests && base_unittests.exe" behaves properly
in the success/failure case.
- Make sure the command prompt title is reliably reset on exit.
I tested autoninja with all combinations of these:
- Host platform: Linux, macOS, Windows
- Remote GN args: <none>, use_goma=true, use_remoteexec=true
- Siso GN args: <none>, use_siso=true
- Targets: base, ../../base/types/expected_macros_unittest.cc^ (on Linux) and ../../base/types/expected_macros_unittest.cc^^ (on Windows)
R=brucedawson@chromium.org, jwata@google.com, tikuta@chromium.org
Bug: b/293657720
Change-Id: I275a775fdc5abb6555f79d4beab76cd0914d4bd6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4924185
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Philipp Wollermann <philwo@chromium.org>
In `CheckChangedLUCIConfigs`, it will call luci-config to list all
config sets. Change that call to v2.
Change-Id: I7a00d157fa631174b2124f0920026fa6b1fea04c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4900076
Commit-Queue: Yuanjun Huang <yuanjunh@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Several clients of this module call its methods repeatedly. Cache them
to speed them up.
E.g. "autosiso" calls FindGclientRoot() 5 times before building
E.g. "git cl format" calls it 5 times in one of my CLs.
Bug: None
Change-Id: I70995c2def9689d46a89896640cc748d9629df3b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4904697
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Make auth be able to generate id_token. Some services on Cloud Run will
need it (e.g. luci-config v2).
Bug: 1487020
Change-Id: Icfe95002f93ee552b99ab2694c7b777e2322484b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4899437
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Yuanjun Huang <yuanjunh@google.com>
We cannot distinguish if this is a moved third-party file. So just
do warnings for a license header from a third-party company.
Bug: 1462922
Change-Id: I78394dcccb9028bce6c535dfce625364dd29157b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4895337
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Maggie Chen <magchen@chromium.org>
This reverts commit 36d937d16d.
Reason for revert: Not sure the best way to notify you about this, but suspect this is the root cause of https://chat.google.com/room/AAAAXGgrwSo/hZDeSAsoQ4k
Verified via rolling back depot_tools
Original change's description:
> Clear existing Change-Ids from description if issue is 0
>
> git cl upload may not create a new change even if issue is 0 because
> of a leftover Change-Id footer in the commit message.
>
> Bug: 1484251
> Change-Id: Ibee0542e0f5a2cbf930b1882892fbb7640054b69
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4880971
> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> Commit-Queue: Gavin Mak <gavinmak@google.com>
Bug: 1484251
Change-Id: Ic3230fd7432f57bf9f3d0266b2e2c0df6a7e9bae
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4893199
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
git cl upload may not create a new change even if issue is 0 because
of a leftover Change-Id footer in the commit message.
Bug: 1484251
Change-Id: Ibee0542e0f5a2cbf930b1882892fbb7640054b69
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4880971
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
All __future__ imports (unicode_literals, print_function) are already
mandatory in py3. Also remove an outdated py2 comment in
presubmit_canned_checks.py
Bug: 1475402
Change-Id: I27cf6a8268f6dd1081f22af782c4c29a975376ae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4867135
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
With submodules, users can accidentally stage and commit gitlink
changes. Add a new gclient command to install a pre-commit hook to
automatically drop gitlink changes that don't correspond to a DEPS
change.
Dropping gitlinks can be bypassed by setting
SKIP_GITLINK_PRECOMMIT=1.
Bug: 1481266
Change-Id: Idd8b273e7d8e37d52627964e8ed6004d068b6b7a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4863221
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
This option can lead to high `git cl upload` latency. We never want to
use this option for gclient managed repositories anyways since changes
need to go through code review.
Note: the default value is off, but it will be set to on if
`submodule.recurse` is set to true.
R=jojwang@google.com
Bug: b/299969165
Change-Id: I2173a2d96d478bf96bfe6e78ef0c5b21d4ac8015
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4874177
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Aravind Vasudevan
Gavin Mak
Joanna Wang
Josip Sokcevic
Brian Ryner
Helmut Januschka
Ben Pastene
Takuto Ikuta
Michael Savigny
Yiwei Zhang
Ben Segall
Philipp Wollermann
Yuanjun Huang
Andrew Grieve
Maggie Chen
Eli Trexler