Commit Graph

211 Commits (caa77da9568fb7c48c9db679cf9dc0ae20080585)

Author SHA1 Message Date
Gavin Mak d4c6d40d50 Reduce RebaseChange max tries from 6 to 2
On failure, gerrit_util always retries HTTP requests the maximum
number of times. This doesn't always make sense, e.g. for RebaseChange
which gets 409 on a merge conflict and can't be retried into
succeeding.

Bug: b/341792235
Change-Id: I6f9e212c5b0365236a99768f056bab2eb60cddc6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5773566
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
9 months ago
Gavin Mak ec800aa077 Add "git cl cherry-pick" for cherry picking a chain of CLs via Gerrit
Users who want to upload multiple cherry picks usually run "git
cherry-pick" locally, multiple times. Gerrit does not recognize
these changes as cherry picks and neither do other services that
query cherry pick info from Gerrit, e.g. rubber stamper.

For Gerrit to identify a change as a true cherry pick, you need to
use their Cherry Pick Revision REST API endpoint. This new command
uses it to create a chain of cherry pick CLs recognized by Gerrit.

Bug: b/341792235
Change-Id: I4ba75da3901f6ea68c1debd65820e802da681798
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5756161
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
9 months ago
Allen Li 29838db8fe [gerrit_util] Add type annotations
Bug: b/351071334
Change-Id: I9e31f14120953e89f6cef3759f595f6aca79cd33
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5754253
Commit-Queue: Allen Li <ayatane@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
9 months ago
Robert Iannucci 863573b665 [gerrit_util] Automatically re-warm SSO cookies.
In the event that `git-remote-sso` doesn't yield the expected
cookies, run `git ls-remote` on chromium/All-Projects.

This operation is fairly quick and robust - this fallback code
should only execute at most 1/day.

R=ayatane, yiwzhang

Change-Id: I5290f47e56341e9c4fd3b295ae117650f592dc7c
Bug: 342644760
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5741848
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
9 months ago
Gavin Mak 7a0c603984 Add CherryPickCommit to gerrit_util
This adds a helper function for this cherrypick endpoint:
https://gerrit-review.googlesource.com/Documentation/rest-api-projects.html#cherry-pick-commit

Compared to gerrit_util.CherryPick, this doesn't require the caller to
provide a change id.

Bug: b/341792235
Change-Id: I874321d182464af45837194d5cef9d650361b8c4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5731033
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
9 months ago
Allen Li 397bf12548 [git_cl] Default to SSO with missing email
This is a safer default if SSO is available

Bug: b/351071334
Change-Id: I2d6b3b5c0fbe3fb7b9783de3d7548be7f14d7391
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5723448
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Allen Li <ayatane@chromium.org>
9 months ago
Allen Li 91937bf196 [git_cl] Parametrize email in ShouldUseSSO
Moves the dependency on Git+cwd up the call stack

Bug: b/351071334
Change-Id: Ia313f9d4720ee10398b757217c333118e9fc7341
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5723091
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Allen Li <ayatane@chromium.org>
9 months ago
Allen Li 4986d4a74b [git_cl] Parametrize cwd in ShouldUseSSO
Remove implicit/global assumptions which could affect tests.

Bug: b/351071334
Change-Id: Ie266228f404b768cb539fdc17dddbbb13693e939
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5723208
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Allen Li <ayatane@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
9 months ago
Robert Iannucci 779f70fd7c [gerrit_util] Move Authenticator to be private in gerrit_util.
This should help give additional confidence while refactoring in
gerrit_util.

R=ayatane, yiwzhang

Change-Id: I03927e072e62f6109571ab699f90db7c51ccc6c0
Bug: b/335483238
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5665455
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
9 months ago
Daniel Rubery 9e9fc99280 Fallback to looking up CL issue from triplet_id
In nosquash mode, `git cl` is pretty challenging to use since we don't
have issue numbers. All three parts of the triplet id are readily
available though. This CL adds a fallback to `git cl` which looks up the
issue number by getting the change details by triplet_id.

Change-Id: I0839fe75bcb4bc8d60ff36b4da26dc0e419a1493
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5633186
Commit-Queue: Daniel Rubery <drubery@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
10 months ago
Allen Li 21b86f4b1c [gerrit_util] Fix typo
Bug: b/351039944
Change-Id: If906ddfdae10b6ba9051eced2b1e7a2c24d54aad
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5677486
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Allen Li <ayatane@chromium.org>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
10 months ago
Allen Li 3c93933ce9 [gerrit_util] Add note about workaround
Bug: b/349433100
Change-Id: I287499a339e1f5d3eab9c5a63068ad68ecd67a11
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5677617
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Allen Li <ayatane@chromium.org>
10 months ago
Allen Li ca091f0d16 [gerrit_util] Update docstring for new param
Change-Id: Ifb6106ad0063183ee0a0377bb3564c4fc99d88ef
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5673779
Auto-Submit: Allen Li <ayatane@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Allen Li <ayatane@chromium.org>
10 months ago
Allen Li ea366e35f0 [gerrit_util] Use shared SSO logic for SSOAuthenticator
Bug: b/348024314
Change-Id: I1bc9aa297360421fad3fd11b157be3c439f944c8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5665454
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Allen Li <ayatane@chromium.org>
10 months ago
Gavin Mak 280b823136 Link to the new password URL when authentication fails
Instead of just telling users their credentials are missing, link them to the new password URL so they can fix it.

Change-Id: I9559868ae948e6d18cef1205083aee0ad5c2489a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5665862
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Auto-Submit: Gavin Mak <gavinmak@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
10 months ago
Allen Li f4e8e13e8b [gerrit_util] Add linked account detection for SSO
Bug: b/348024314
Change-Id: Ic982de2892769870805407c6a00856943133dd62
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5651293
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Allen Li <ayatane@chromium.org>
10 months ago
Allen Li 459e1a54a5 [gerrit_util] Add ChainedAuthenticator
Because we need to dynamically determine whether to use SSO.

Bug: b/348024314
Change-Id: I5ac768f1e0c20254b4cfd4815270ee4e2b9a5544
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5660884
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Allen Li <ayatane@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
10 months ago
Allen Li 334a3acddb [gerrit_util] Fix typo
Bug: b/348024314
Change-Id: I6419ad006535d2763b190d2e4f354816e3fd1c8f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5651991
Commit-Queue: Allen Li <ayatane@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
10 months ago
Allen Li 24e9380bc5 [gerrit_util] Add explicit authenticator param
We need this to figure out which authenticator to use, so we need to
override the initial "bootstrap" authenticator.

Bug: b/348024314
Change-Id: I52c5b1db83bc4e2a0a1ec2a07155d352b593cde4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5651288
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Allen Li <ayatane@chromium.org>
10 months ago
Allen Li 3a5d348a0b [gerrit_util] Add ShouldUseSSO helper
Bug: b/348024314
Change-Id: I5b061bc6afeffbc84aee12250732e96a0ad0f32b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5651287
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Allen Li <ayatane@chromium.org>
10 months ago
Allen Li bb38aa2f8e [gerrit_util] Use SSO only if configured with appropriate email
So you can use chromium.org or other accounts in some repos

Bug: b/348024314
Change-Id: Ice6b6d9e6e827a606a2ce7f3b127d4660df1aedf
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5641255
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Allen Li <ayatane@chromium.org>
10 months ago
Robert Iannucci 005e60ceda [git_cl] Make EnsureCanUploadPatchset work for all auth methods.
Previously EnsureCanUploadPatchset only had a working implementation
for the CookiesAuthenticator, relying on being able to parse the
user name out of the .gitcookies file.

Additionally, the previous implementation assumed that you would
always authenticate as your primary Gerrit account OR you had
a matching `user.email` gitconfig entry, even though neither of these
is a strict requirement for the upload to work.

The new implementation still short-circuits if issue_owner matches
the configured user.email, but other than this it just asks Gerrit
what the full list of linked emails is for the currently authenticated
account.

The new approach is not only correct, but will now work for all auth
schemes in exactly the same way.

When the accounts do mismatch, you will now see output like:

```
WARNING: Change 5590262 is owned by iannucci@chromium.org, but Gerrit knows you as:
  * user@example.org
  * other.user@example.com
  * primary@real.example.com (preferred)
Uploading may fail due to lack of permissions.
```

R=ayatane@chromium.org, yiwzhang@google.com

Bug: 336351842
Change-Id: I89c1b121c9110e00d1348884aaf025fc783542d0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5590262
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
10 months ago
mlcui 6592795601 gerrit_util: Fix NameError in GceAuthenticator._get
Bug: None
Change-Id: I6a6a210069da0f77ebbed4c6ef530864cb2d8109
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5644850
Reviewed-by: Gavin Mak <gavinmak@google.com>
Auto-Submit: Michael Cui <mlcui@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
10 months ago
Robert Iannucci 6daca50597 [git_cl] Refactor EnsureAuthenticated logic.
This will allow us to implement EnsureAuthenticated for other
Authenticator implementations.

R=ayatane@chromium.org

Bug: 336351842
Change-Id: I5ff4efd45eb5baa6baa048842510ec6e18e1b8fa
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5588510
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
10 months ago
Robert Iannucci 137bb69871 [gerrit_util] Cache resolved Authenticator as a class variable.
I noticed that the Authenticator is resolved maybe 5 or 6 times per
git-cl invocation. This should lead to more consistent behavior and
will likely be a bit faster, especially for SSOAuthenticator and
LuciAuthAuthenticator which involve subprocess invocations.

R=ayatane@chromium.org

Bug: 336351842
Change-Id: Id6c2873a6960a171305560acb98afe2c4f397295
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5589865
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
10 months ago
Allen Li bdf64705c3 [gerrit_util] Factor out SSOHelper
Add a layer of abstraction/isolation for general organization.

Also, this logic needs to be used in Git setup too, not just Gerrit
authentication.

Bug: b/348024314
Change-Id: Ie1310a9b8e71c05c72a4b987dcbff76b70c67945
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5645906
Commit-Queue: Allen Li <ayatane@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
10 months ago
Robert Iannucci c4c3d5326e [gerrit_util] Add some additional tests and fixes for SSOAuthenticator.
Unfortunately, the depot_tools presubmit builders are incredibly slow
which make the subprocess based tests fail flakily. I've marked them
all as `skip` with an optional way to run them locally.

R=ayatane, yiwzhang

Bug: b/335483238
Change-Id: I407aed3a1ed01563a0a80973b679aca405b9cde9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5641259
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
10 months ago
Allen Li fa1bba51d7 [gerrit_util] Add newauth.py helper
These checks will be needed in git_cl.py and probably other places.

Make it easy to change the default to opt-in as some point (and for
deletion later).

Bug: b/348024314
Change-Id: I768986e47ec2e86604268d74ca2b632458263bba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5641084
Commit-Queue: Allen Li <ayatane@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
10 months ago
Robert Iannucci 6833e00918 [gerrit_util] Revise comments on SSOAuthenticator.
Previous comments on SSOAuthenticator were incorrect (the
git-remote-sso process does NOT need to persist). Remove comments
which were inconsistent with the code, and add an explainer on why
the _launch_sso_helper method is written the way it is.

R=ayatane, yiwzhang

Bug: b/335483238
Change-Id: I6b318ef36d7c4f757cd0b979fdc90c01c7a5b529
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5641089
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
10 months ago
Robert Iannucci 1f4f982beb [git-cl] SSOAuthenticator: Add tests and fix bugs.
Adds a few very basic tests.

Fixes bug where, when using python3.8, the cookie jar would not
correctly parse the cookie file.

R=ayatane

Bug: b/335483238
Change-Id: If44eea00d67cb2716df460ef0af93811e351f764
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5637936
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
10 months ago
Robert Iannucci 88a72320ed [gerrit_util] Add dogfoodable luci-auth Authenticator (reland)
Inspired by https://chromium-review.googlesource.com/c/5577744.

This implementation allows toggling the entire new-auth-stack with
the git configuration:

    [depot-tools]
        useNewAuthStack = 1

Additionally, you can set:

    [depot-tools]
        newAuthSkipSSO = 1

To intentionally skip SSOAuthenticator for now while doing local
evaluation of these auth methods.

This CL was uploaded without gitcookies using the new luci-auth
Authenticator.

Subsequent CLs will adjust creds-check and EnsureAuthenticated to
work correctly with the new auth stack.

Originally landed in
https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5590324

Reverted because of the preceding CL
https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5586799

Bug: 336351842, 336652327
Change-Id: I72df6689ec1ea5429fecc86e5752366fcda7465a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5627835
Reviewed-by: Scott Lee <ddoman@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Allen Li <ayatane@chromium.org>
11 months ago
Robert Iannucci 5f585948d4 [gerrit_util] Add opt-in SSOAuthenticator (reland)
This adds a new authenticator implementation to gerrit_util which
uses the Google corp `git-remote-sso` protocol helper to get
secure tunneled access to the internal hosted Google git services.

It is possible to opt in on Google-managed devices by adding the
following git configuration:

    [url "sso://chromium/"]
        insteadOf = https://chromium.googlesource.com/
        insteadOf = http://chromium.googlesource.com/
    [depot-tools]
      useNewAuthStack = 1

This CL was uploaded using the new authenticator with no
.gitcookies file present on disk.

This was originally landed in
https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5586799

It was reverted because it caused the error:

    AttributeError: module 'functools' has no attribute 'cache'

https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket/8745365240091840049/+/u/Commit_log_for_uploaded_revision/stdout

So we changed functools.cache -> functools.lru_cache

Bug: 336351842
Change-Id: I0dcb04ada8281a60792ae08777b01ecf116f2a8f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5627834
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Allen Li <ayatane@chromium.org>
11 months ago
Takuto Ikuta e757e08663 Revert "[gerrit_util] Add opt-in SSOAuthenticator."
This reverts commit 02091c6148.

Reason for revert:
https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket/8745365240091840049/+/u/Commit_log_for_uploaded_revision/stdout

Original change's description:
> [gerrit_util] Add opt-in SSOAuthenticator.
>
> This adds a new authenticator implementation to gerrit_util which
> uses the Google corp `git-remote-sso` protocol helper to get
> secure tunneled access to the internal hosted Google git services.
>
> It is possible to opt in on Google-managed devices by adding the
> following git configuration:
>
>     [url "sso://chromium/"]
>         insteadOf = https://chromium.googlesource.com/
>         insteadOf = http://chromium.googlesource.com/
>     [depot-tools]
>       useNewAuthStack = 1
>
> This CL was uploaded using the new authenticator with no
> .gitcookies file present on disk.
>
> R=ayatane@chromium.org, ayatane@google.com, ddoman@chromium.org
>
> Bug: 336351842
> Change-Id: Ia3eccb77ebf1cdc863513ac095bf65ede6adda43
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5586799
> Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Allen Li <ayatane@chromium.org>
> Reviewed-by: Scott Lee <ddoman@chromium.org>

Bug: 336351842
Change-Id: I412d560955ff78169a83b527d2fe2dd0a2c5d50d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5624014
Owners-Override: Patti Lor <patricialor@chromium.org>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Patti Lor <patricialor@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
11 months ago
Takuto Ikuta 58cb6562e3 Revert "[gerrit_util] Add dogfoodable luci-auth Authenticator."
This reverts commit f871d80a7e.

Reason for revert:
This is to revert https://crrev.com/c/5624014.

Original change's description:
> [gerrit_util] Add dogfoodable luci-auth Authenticator.
>
> Inspired by https://chromium-review.googlesource.com/c/5577744.
>
> This implementation allows toggling the entire new-auth-stack with
> the git configuration:
>
>     [depot-tools]
>         useNewAuthStack = 1
>
> Additionally, you can set:
>
>     [depot-tools]
>         newAuthSkipSSO = 1
>
> To intentionally skip SSOAuthenticator for now while doing local
> evaluation of these auth methods.
>
> This CL was uploaded without gitcookies using the new luci-auth
> Authenticator.
>
> Subsequent CLs will adjust creds-check and EnsureAuthenticated to
> work correctly with the new auth stack.
>
> R=ayatane@google.com
>
> Bug: 336351842, 336652327
> Change-Id: I0eb6d82ca106ddd114b74f63d8cda4c5a7b70c86
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5590324
> Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
> Reviewed-by: Scott Lee <ddoman@chromium.org>
> Commit-Queue: Allen Li <ayatane@chromium.org>

Bug: 336351842, 336652327
Change-Id: I7c947760a096f48bdac3d640f71e40ad10fe6f3e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5624015
Owners-Override: Patti Lor <patricialor@chromium.org>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Patti Lor <patricialor@chromium.org>
Reviewed-by: Patti Lor <patricialor@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
11 months ago
Robert Iannucci f871d80a7e [gerrit_util] Add dogfoodable luci-auth Authenticator.
Inspired by https://chromium-review.googlesource.com/c/5577744.

This implementation allows toggling the entire new-auth-stack with
the git configuration:

    [depot-tools]
        useNewAuthStack = 1

Additionally, you can set:

    [depot-tools]
        newAuthSkipSSO = 1

To intentionally skip SSOAuthenticator for now while doing local
evaluation of these auth methods.

This CL was uploaded without gitcookies using the new luci-auth
Authenticator.

Subsequent CLs will adjust creds-check and EnsureAuthenticated to
work correctly with the new auth stack.

R=ayatane@google.com

Bug: 336351842, 336652327
Change-Id: I0eb6d82ca106ddd114b74f63d8cda4c5a7b70c86
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5590324
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Allen Li <ayatane@chromium.org>
11 months ago
Robert Iannucci 02091c6148 [gerrit_util] Add opt-in SSOAuthenticator.
This adds a new authenticator implementation to gerrit_util which
uses the Google corp `git-remote-sso` protocol helper to get
secure tunneled access to the internal hosted Google git services.

It is possible to opt in on Google-managed devices by adding the
following git configuration:

    [url "sso://chromium/"]
        insteadOf = https://chromium.googlesource.com/
        insteadOf = http://chromium.googlesource.com/
    [depot-tools]
      useNewAuthStack = 1

This CL was uploaded using the new authenticator with no
.gitcookies file present on disk.

R=ayatane@chromium.org, ayatane@google.com, ddoman@chromium.org

Bug: 336351842
Change-Id: Ia3eccb77ebf1cdc863513ac095bf65ede6adda43
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5586799
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Allen Li <ayatane@chromium.org>
Reviewed-by: Scott Lee <ddoman@chromium.org>
11 months ago
Robert Iannucci ad3ad021d9 [gerrit_util] Invert Authenticator model.
Instead of having CreateHttpConn manipulate the request, with the
Authenticator only able to provide the Authorization header value,
the Authenticator now gets the ability to manipulate the entire
HttpConn object.

This will be used for a new Authenticator method which needs to
include a proxy, cookies, and also manipulate the target request
URI, in addition to providing an Authorization header value.

R=ayatane@chromium.org, yiwzhang@google.com

Bug: 336351842
Change-Id: Ia7d0bbfbb907d8ab6c6d12d000f514fa7afc7245
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5585665
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Allen Li <ayatane@chromium.org>
11 months ago
Robert Iannucci 840e538154 [gerrit_util] Add stronger type annotations around CreateHttpConn.
This also fixes a potential bug where ReadHttpJsonResponse could
improperly return None if the server had a completely empty reply.
ReadHttpJsonResponse will now return an empty dictionary in this
case (which is the assumption that most of the callsites are
making).

R=yiwzhang@google.com

Bug: 336351842
Change-Id: I0aa88e233563a0685b6c0f32ea77ad3e094b9cbc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5585184
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
11 months ago
Gregory NISBET 3963cc6399 [gerrit_util.py] Remove unnecessary imports from gerrit_util.py
Remove the libraries that have no uses according to gd in vim.

BUG=None
TEST=upload this CL

Change-Id: I7fe6bbd54c1a4f00a3d0e3f04be0f02609734e46
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5578897
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gregory Nisbet <gregorynisbet@google.com>
11 months ago
Robert Iannucci 6427b94bc2 [git_cl] Refactor EnsureAuthenticated.
This will allow us to invert the main Authenticator API.

R=yiwzhang@google.com

Bug: 336351842
Change-Id: Iebe06f68373b99685809849bcd96371e7634d6ef
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5582406
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
11 months ago
Robert Iannucci c57b7ed364 [gerrit_util] Change Authenticator API to return proxy info.
This will be used with an upcoming SSOAuthenticator implementation
which will need to proxy all http requests for Googlers.

R=ayatane, gavinmak@google.com

Bug: 336351842
Change-Id: If8cbb8db51fce198e704f109232868421130b40c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5582100
Commit-Queue: Gavin Mak <gavinmak@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
11 months ago
Robert Iannucci a85dcffff4 [git_cl] Refactor away a use of isinstance.
Previously when composing a debugging trace, it would use an
isinstance check to special-case inclusion of a gitcookies file.

This CL refactors this to be part of the Authenticator API, instead,
which means that we will always get some sort of authenticator log
file in a trace, even if it's empty.

This also provides an affordance to add debugging information for
the other authenticator types later.

R=ddoman@chromium.org, gavinmak@google.com

Bug: 336351842
Change-Id: Idd6f45ea60b089f9b2391b5527c5281f67421043
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5571497
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
11 months ago
Robert Iannucci 3ea849f363 [git_cl.py] Remove all usage and messages around netrc.
netrc was already legacy in 2017:
fe30f18fc5

The removed gerrit-init.sh script was written in 2013 and no longer
works.

R=yiwzhang@google.com

Bug: 336351842
Change-Id: Ia8e845899973eb5ead66aee2f54078c09d80ef28
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5564197
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
11 months ago
Fei Shao b4fbce4794 gerrit_util: Expand user path from http.cookiefile config
This fixes an issue where "~" in user http.cookiefile config doesn't get
expanded and causes failures when "git cl" commands tries to read the
credentials.

Expand the path returned by scm.GIT.GetConfig() but not just the default
value.

Bug: b/341619336
Change-Id: Ib89d5b86710a0a6bb6ca669639eeec6adc75689d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5547345
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Fei Shao <fshao@chromium.org>
11 months ago
Josip Sokcevic ab914fad1e Improve cloudtop error message on git cl failure
R=gavinmak@google.com

Fixed: 280621923
Change-Id: I856dc1eb4f3fcd65906196b66f358b1dd413012e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5547690
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
11 months ago
Josh Berenhaus 03ee6d950d Fix use of deprecated LOGGER.warn in gerrit test
Change-Id: Ibe60a15f80ffb6072094c53c7b80fcc9419fc52f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5380997
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Auto-Submit: Josh Berenhaus <joshber@meta.com>
1 year ago
Aravind Vasudevan 9f9bab25da Reland "Update gclient to use git config caching"
This reverts commit 3569608028.

Reason for revert: This includes a fix for crbug.com/324358728.

The rebase-update command has logic which tries to specifically set a key to an empty string and this has been intentionally set this way[1]. The new SetConfig implementation does treats empty string as None and hence tries to unset the config, resulting in error code 5. The patchset 2 fixes this bug and adds a test to ensure SetConfig can set an empty string to be backward compatible.

[1] https://codereview.chromium.org/228353003

Original change's description:
> Revert "Update gclient to use git config caching"
>
> This reverts commit 3edda8d185.
>
> Reason for revert: Breaks rebase-update; crbug.com/324358728
>
> Original change's description:
> > Update gclient to use git config caching
> >
> > This change updates all the modules used by gclient to use `scm.GIT` for git config calls over directly invoking the subprocess.
> >
> > This change currently doesn't modify git_cache since the config reads and writes within it are done on bare repository. A follow-up CL will update git_cache.
> >
> > A follow-up CL will also update git_cl and git_map_branches since they have shown performance improvements too: https://crrev.com/c/4697786.
> >
> > Benchmarking
> > ============
> > With chromium/src as the baseline super project, this change reduces about 380 git config calls out of 507 total calls on cache hits during no-op. The below numbers are benchmarked with `update_depot_tools` turned off.
> >
> > Windows Benchmark
> > =================
> > Baseline (gpaste/6360045736951808): ~1min 12 sec.
> > With Caching (gpaste/6480065209040896): ~1min 3sec.
> > ~12.5% decrease in gclient sync noop runtime.
> >
> > Linux Benchmark
> > ===============
> > Baseline (gpaste/4730436763254784): ~3.739 sec.
> > With Caching (gpaste/4849870978940928): ~3.534 sec.
> > ~5.5% decrease in gclient sync noop runtime.
> >
> > Bug: 1501984
> > Change-Id: Ib48df2d26a0c742a9b555a1e2ed6366221c7db17
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5252498
> > Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
> > Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
>
> Bug: 1501984
> Change-Id: I4a603238d9ed43edafc8e574493800670520a1d9
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5279198
> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>

Bug: 1501984
Change-Id: I405abc16c2ef6f0689031c82c61af71aad302122
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5280779
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
1 year ago
Aravind Vasudevan 3569608028 Revert "Update gclient to use git config caching"
This reverts commit 3edda8d185.

Reason for revert: Breaks rebase-update; crbug.com/324358728

Original change's description:
> Update gclient to use git config caching
>
> This change updates all the modules used by gclient to use `scm.GIT` for git config calls over directly invoking the subprocess.
>
> This change currently doesn't modify git_cache since the config reads and writes within it are done on bare repository. A follow-up CL will update git_cache.
>
> A follow-up CL will also update git_cl and git_map_branches since they have shown performance improvements too: https://crrev.com/c/4697786.
>
> Benchmarking
> ============
> With chromium/src as the baseline super project, this change reduces about 380 git config calls out of 507 total calls on cache hits during no-op. The below numbers are benchmarked with `update_depot_tools` turned off.
>
> Windows Benchmark
> =================
> Baseline (gpaste/6360045736951808): ~1min 12 sec.
> With Caching (gpaste/6480065209040896): ~1min 3sec.
> ~12.5% decrease in gclient sync noop runtime.
>
> Linux Benchmark
> ===============
> Baseline (gpaste/4730436763254784): ~3.739 sec.
> With Caching (gpaste/4849870978940928): ~3.534 sec.
> ~5.5% decrease in gclient sync noop runtime.
>
> Bug: 1501984
> Change-Id: Ib48df2d26a0c742a9b555a1e2ed6366221c7db17
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5252498
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>

Bug: 1501984
Change-Id: I4a603238d9ed43edafc8e574493800670520a1d9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5279198
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
1 year ago
Aravind Vasudevan 3edda8d185 Update gclient to use git config caching
This change updates all the modules used by gclient to use `scm.GIT` for git config calls over directly invoking the subprocess.

This change currently doesn't modify git_cache since the config reads and writes within it are done on bare repository. A follow-up CL will update git_cache.

A follow-up CL will also update git_cl and git_map_branches since they have shown performance improvements too: https://crrev.com/c/4697786.

Benchmarking
============
With chromium/src as the baseline super project, this change reduces about 380 git config calls out of 507 total calls on cache hits during no-op. The below numbers are benchmarked with `update_depot_tools` turned off.

Windows Benchmark
=================
Baseline (gpaste/6360045736951808): ~1min 12 sec.
With Caching (gpaste/6480065209040896): ~1min 3sec.
~12.5% decrease in gclient sync noop runtime.

Linux Benchmark
===============
Baseline (gpaste/4730436763254784): ~3.739 sec.
With Caching (gpaste/4849870978940928): ~3.534 sec.
~5.5% decrease in gclient sync noop runtime.

Bug: 1501984
Change-Id: Ib48df2d26a0c742a9b555a1e2ed6366221c7db17
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5252498
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
1 year ago
Josip Sokcevic 0cba8dd68e [owners] Limit number of concurrent Gerrit connections
R=gavinmak@google.com

Fixed: 319469729
Change-Id: I955fa17c80cee25bc2163a8bca0234cdbb659f62
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5186760
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
1 year ago