For a ProvidedDiffChange, the AllFiles method naively returns all files
with rglob("*"). The returned list includes files in nested submodules.
This does not match the behavior of GitChange's AllFiles which uses
git ls-files to find all files.
Implement a new SCM that stops iterating recursively when it
sees a submodule from .gitmodules in the repo root.
Bug: b/323243527
Change-Id: I170d0f1bc4a838acea04779dee3df7fca0bce359
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5648616
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Previously EnsureCanUploadPatchset only had a working implementation
for the CookiesAuthenticator, relying on being able to parse the
user name out of the .gitcookies file.
Additionally, the previous implementation assumed that you would
always authenticate as your primary Gerrit account OR you had
a matching `user.email` gitconfig entry, even though neither of these
is a strict requirement for the upload to work.
The new implementation still short-circuits if issue_owner matches
the configured user.email, but other than this it just asks Gerrit
what the full list of linked emails is for the currently authenticated
account.
The new approach is not only correct, but will now work for all auth
schemes in exactly the same way.
When the accounts do mismatch, you will now see output like:
```
WARNING: Change 5590262 is owned by iannucci@chromium.org, but Gerrit knows you as:
* user@example.org
* other.user@example.com
* primary@real.example.com (preferred)
Uploading may fail due to lack of permissions.
```
R=ayatane@chromium.org, yiwzhang@google.com
Bug: 336351842
Change-Id: I89c1b121c9110e00d1348884aaf025fc783542d0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5590262
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
This will allow us to implement EnsureAuthenticated for other
Authenticator implementations.
R=ayatane@chromium.org
Bug: 336351842
Change-Id: I5ff4efd45eb5baa6baa048842510ec6e18e1b8fa
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5588510
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
I noticed that the Authenticator is resolved maybe 5 or 6 times per
git-cl invocation. This should lead to more consistent behavior and
will likely be a bit faster, especially for SSOAuthenticator and
LuciAuthAuthenticator which involve subprocess invocations.
R=ayatane@chromium.org
Bug: 336351842
Change-Id: Id6c2873a6960a171305560acb98afe2c4f397295
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5589865
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Add a layer of abstraction/isolation for general organization.
Also, this logic needs to be used in Git setup too, not just Gerrit
authentication.
Bug: b/348024314
Change-Id: Ie1310a9b8e71c05c72a4b987dcbff76b70c67945
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5645906
Commit-Queue: Allen Li <ayatane@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Unfortunately, the depot_tools presubmit builders are incredibly slow
which make the subprocess based tests fail flakily. I've marked them
all as `skip` with an optional way to run them locally.
R=ayatane, yiwzhang
Bug: b/335483238
Change-Id: I407aed3a1ed01563a0a80973b679aca405b9cde9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5641259
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
An valid empty diff can be passed into _diffs_to_change_files if the
modified file's contents is same as upstream. In this case,
_diffs_to_change_files gets an IndexError.
Bug: b/336555565
Change-Id: I848e6016a1e8089473ff8a72d2e0142fbabfda9f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5635166
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Previous comment was... lacking. Actually, wrong. It's doing the
exact opposite of lazy loading. Maybe they were trying to use AI code
gen before it existed?
Bug: b/348024314
Change-Id: I068d43fe1abf757cb8b109890a14ca0cefe3fec0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5641090
Commit-Queue: Allen Li <ayatane@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
These checks will be needed in git_cl.py and probably other places.
Make it easy to change the default to opt-in as some point (and for
deletion later).
Bug: b/348024314
Change-Id: I768986e47ec2e86604268d74ca2b632458263bba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5641084
Commit-Queue: Allen Li <ayatane@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
This is a reland of commit 561772c448
Original change's description:
> [presubmit checks] Check if files are written to a dep dir
>
> No files should be written to a directory that's used by CIPD or GCS, as
> defined in DEPS. Git already doesn't allow files to be written to a
> directory that's a gitlink.
>
> R=jojwang@google.com
>
> Bug: 343199633
> Change-Id: I8d3414eac728580eaf9ac7e337bb22bca3989e4e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5633187
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Bug: 343199633
Change-Id: Ifc3f6c4df328cdd215ceb7d0333f68223e9a1ccb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5643922
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Previous comments on SSOAuthenticator were incorrect (the
git-remote-sso process does NOT need to persist). Remove comments
which were inconsistent with the code, and add an explainer on why
the _launch_sso_helper method is written the way it is.
R=ayatane, yiwzhang
Bug: b/335483238
Change-Id: I6b318ef36d7c4f757cd0b979fdc90c01c7a5b529
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5641089
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Adds a few very basic tests.
Fixes bug where, when using python3.8, the cookie jar would not
correctly parse the cookie file.
R=ayatane
Bug: b/335483238
Change-Id: If44eea00d67cb2716df460ef0af93811e351f764
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5637936
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
This reverts commit 561772c448.
Reason for revert: `KeyError: 'deps'` breaking some builds
Original change's description:
> [presubmit checks] Check if files are written to a dep dir
>
> No files should be written to a directory that's used by CIPD or GCS, as
> defined in DEPS. Git already doesn't allow files to be written to a
> directory that's a gitlink.
>
> R=jojwang@google.com
>
> Bug: 343199633
> Change-Id: I8d3414eac728580eaf9ac7e337bb22bca3989e4e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5633187
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Bug: 343199633
Change-Id: I26b8e72320260e394a72aee1f7e67dfc2b3e2987
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5641258
Auto-Submit: Gavin Mak <gavinmak@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
No files should be written to a directory that's used by CIPD or GCS, as
defined in DEPS. Git already doesn't allow files to be written to a
directory that's a gitlink.
R=jojwang@google.com
Bug: 343199633
Change-Id: I8d3414eac728580eaf9ac7e337bb22bca3989e4e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5633187
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
This new option allows programatically providing new
description when uploading a new patchset.
Bug: 346174899
Change-Id: I6d7e361b7da0fd06675c04df4fd2f68c7954dd40
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5620640
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
We are relocating siso binary from third_party/siso/siso to
third_party/siso/cipd/siso to make it compatible with Cog, which
requires cipd packages to be placed in directories with no Git content.
Bug: 346837573
Change-Id: If40e6033e341dcedfee5670f07d68c14bd923f7b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5625091
Reviewed-by: Philipp Wollermann <philwo@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
This is achieved by duplicating vpython spec in tests directory and
changing the python version to 3.8. The reason for this change is that
even though depot_tools support 3.11, the entire infra is still on
3.8 so any new 3.8+ feature used in depot_tools may break infra test and
it has already happened couple of times.
Bug: 346784198
Change-Id: I7dbd0d926da21854abf47af2c2f6edfeec2c5c4d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5625090
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Running a command line longer that 32k on Windows lead to issues (as
usual). We run git diff in multiple scenarios on a list of changed
files. This list is not partitioned and may lead to a command line
much longer than OS can allow.
This change replaces BuildGitDiffCmd with RunGitDiffCmd that splits
the file list internally and runs git diff multiple times if required.
Change-Id: Icfd8efc7520105b8d98125e90bb96c2bb9bd3a0a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5613673
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Aleksei Khoroshilov <akhoroshilov@brave.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
This reverts commit 02091c6148.
Reason for revert:
https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket/8745365240091840049/+/u/Commit_log_for_uploaded_revision/stdout
Original change's description:
> [gerrit_util] Add opt-in SSOAuthenticator.
>
> This adds a new authenticator implementation to gerrit_util which
> uses the Google corp `git-remote-sso` protocol helper to get
> secure tunneled access to the internal hosted Google git services.
>
> It is possible to opt in on Google-managed devices by adding the
> following git configuration:
>
> [url "sso://chromium/"]
> insteadOf = https://chromium.googlesource.com/
> insteadOf = http://chromium.googlesource.com/
> [depot-tools]
> useNewAuthStack = 1
>
> This CL was uploaded using the new authenticator with no
> .gitcookies file present on disk.
>
> R=ayatane@chromium.org, ayatane@google.com, ddoman@chromium.org
>
> Bug: 336351842
> Change-Id: Ia3eccb77ebf1cdc863513ac095bf65ede6adda43
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5586799
> Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Allen Li <ayatane@chromium.org>
> Reviewed-by: Scott Lee <ddoman@chromium.org>
Bug: 336351842
Change-Id: I412d560955ff78169a83b527d2fe2dd0a2c5d50d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5624014
Owners-Override: Patti Lor <patricialor@chromium.org>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Patti Lor <patricialor@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
This reverts commit f871d80a7e.
Reason for revert:
This is to revert https://crrev.com/c/5624014.
Original change's description:
> [gerrit_util] Add dogfoodable luci-auth Authenticator.
>
> Inspired by https://chromium-review.googlesource.com/c/5577744.
>
> This implementation allows toggling the entire new-auth-stack with
> the git configuration:
>
> [depot-tools]
> useNewAuthStack = 1
>
> Additionally, you can set:
>
> [depot-tools]
> newAuthSkipSSO = 1
>
> To intentionally skip SSOAuthenticator for now while doing local
> evaluation of these auth methods.
>
> This CL was uploaded without gitcookies using the new luci-auth
> Authenticator.
>
> Subsequent CLs will adjust creds-check and EnsureAuthenticated to
> work correctly with the new auth stack.
>
> R=ayatane@google.com
>
> Bug: 336351842, 336652327
> Change-Id: I0eb6d82ca106ddd114b74f63d8cda4c5a7b70c86
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5590324
> Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
> Reviewed-by: Scott Lee <ddoman@chromium.org>
> Commit-Queue: Allen Li <ayatane@chromium.org>
Bug: 336351842, 336652327
Change-Id: I7c947760a096f48bdac3d640f71e40ad10fe6f3e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5624015
Owners-Override: Patti Lor <patricialor@chromium.org>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Patti Lor <patricialor@chromium.org>
Reviewed-by: Patti Lor <patricialor@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Or $XDG_CONFIG_HOME/.config/depot_tools if set
A followup CL can add support for win/macos.
Bug: b/345092320
Change-Id: I877baa4d7fd912b42cfcd88ad0aa347b700a89f2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5604606
Commit-Queue: Richard Wang <richardwa@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
This reverts commit 165fdee7e6.
Reason for revert: broke build
Original change's description:
> [cipd] Add git-credential-luci
>
> This can be used for OAuth to Gerrit (in lieu of gitcookies).
>
> Bug: b/335483238
> Change-Id: Ief933ca0003b9f85e14aad1596b66b134311ea39
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5620638
> Reviewed-by: Scott Lee <ddoman@chromium.org>
> Commit-Queue: Allen Li <ayatane@chromium.org>
Bug: b/335483238
Change-Id: I95bf6d85313e06afcd6045f93898153ef0118b3b
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5620639
Owners-Override: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Auto-Submit: Allen Li <ayatane@chromium.org>
The chromium recipe (and only the chromium recipe?) currently reads this
prop to trigger a clobber build. But support for that in the recipe is
being removed in https://crrev.com/c/5620860. So this removes the
ability to set that arg via `git cl try`. Across all chrome/chromium
builders, that arg was only passed in six builds in the past year. So
it's very unlikely anyone depends on it, or will notice it's gone.
Bug: 346363294
Change-Id: Icf05c9a2fd6f2e9239c735e4c5507aee9a26e0e5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5620634
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Inspired by https://chromium-review.googlesource.com/c/5577744.
This implementation allows toggling the entire new-auth-stack with
the git configuration:
[depot-tools]
useNewAuthStack = 1
Additionally, you can set:
[depot-tools]
newAuthSkipSSO = 1
To intentionally skip SSOAuthenticator for now while doing local
evaluation of these auth methods.
This CL was uploaded without gitcookies using the new luci-auth
Authenticator.
Subsequent CLs will adjust creds-check and EnsureAuthenticated to
work correctly with the new auth stack.
R=ayatane@google.com
Bug: 336351842, 336652327
Change-Id: I0eb6d82ca106ddd114b74f63d8cda4c5a7b70c86
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5590324
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Allen Li <ayatane@chromium.org>
Those 2 commands have been broken since the introduction of gcs dep as
it tries to run a git command aginst path to a gcs dep. Since nobody
has complained so far, it's safe to assume the command is never used
by anyone today, therefore, it should be safe to delete.
Bug: 345486495
Change-Id: Ia27e5e8486f790234bd66e890e995f1d5c14f9c5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5608374
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
This adds a new authenticator implementation to gerrit_util which
uses the Google corp `git-remote-sso` protocol helper to get
secure tunneled access to the internal hosted Google git services.
It is possible to opt in on Google-managed devices by adding the
following git configuration:
[url "sso://chromium/"]
insteadOf = https://chromium.googlesource.com/
insteadOf = http://chromium.googlesource.com/
[depot-tools]
useNewAuthStack = 1
This CL was uploaded using the new authenticator with no
.gitcookies file present on disk.
R=ayatane@chromium.org, ayatane@google.com, ddoman@chromium.org
Bug: 336351842
Change-Id: Ia3eccb77ebf1cdc863513ac095bf65ede6adda43
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5586799
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Allen Li <ayatane@chromium.org>
Reviewed-by: Scott Lee <ddoman@chromium.org>
Gavin Mak
Allen Li
Allen Li
Robert Iannucci
recipe-roller
mlcui
Josip Sokcevic
Yiwei Zhang
Takuto Ikuta
Chenlin Fan
Aleksey Khoroshilov
Richard Wang
Ben Pastene