Commit Graph

2350 Commits (a1c9f5ecefd1ba3b66660bab97732ea640dee5f1)

Author SHA1 Message Date
Allen Li a1c9f5ecef [scm_mock] Fix types
Change-Id: I3d1dba2d8c3adc5ef4cf475514174b1d3f817010
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5773567
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Allen Li <ayatane@chromium.org>
9 months ago
Josip Sokcevic bed17582bf Add a script to copy GCS packages from one DEPS file to another
This script will be used to roll downstream GCS packages.

Bug: 358435510
Change-Id: I7ea1229eb4e8c4c590cad336573f9d24662f5730
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5765858
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
9 months ago
Gavin Mak ec800aa077 Add "git cl cherry-pick" for cherry picking a chain of CLs via Gerrit
Users who want to upload multiple cherry picks usually run "git
cherry-pick" locally, multiple times. Gerrit does not recognize
these changes as cherry picks and neither do other services that
query cherry pick info from Gerrit, e.g. rubber stamper.

For Gerrit to identify a change as a true cherry pick, you need to
use their Cherry Pick Revision REST API endpoint. This new command
uses it to create a chain of cherry pick CLs recognized by Gerrit.

Bug: b/341792235
Change-Id: I4ba75da3901f6ea68c1debd65820e802da681798
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5756161
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
9 months ago
Gavin Mak f52dad9747 Handle binaries in presubmit_diff
presubmit_diff always tries to decode files from Gitiles with UTF-8
but if the file is binary, it gets a UnicodeDecodeError. Don't
assume that a file is text and just write the bytes directly.

Bug: b/358175830
Change-Id: Iaa259051b4157737397933e9d994c00e9da5836c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5771938
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
9 months ago
Robert Iannucci a3fb9bad66 [scm] Mock global git config scope globally.
This will be useful for writing tests which rely on shared 'global'
config between different working directories.

This also adds support for mocking 'system' (global, immutable) and
'workspace' (local, mutable). The workspace scope I think is a bit iffy
though, given how `git cl` is actually built - currently scm.GIT doesn't
really know about clone vs. workspace, and afaik no config adjustements
actually apply to the workspace scope.

Adds tests for mocked git config implementation, including bug fixes
to the current implementation revealed by the tests.

R=ayatane, yiwzhang

Change-Id: Ia56d2a81d8df6ae75d9f8d0497be0d67bdc03651
Bug: 355505750
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5759163
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
9 months ago
Robert Iannucci 126b52e8e5 [scm.py] Fix error with missing_ok in GitConfig.
Previously there was an error where you could have:

  global: {"key": ["something"]}
  local: {}

And called SetConfig("key", None, missing_ok=False) - this would end
up raising CalledProcessError(returncode=5) because the caching
layer would see that "key" was not missing, then call into the
storage layer which would do `git config --local ...` which was
missing "key" (and therefore fail).

R=ayatane, yiwzhang

Change-Id: Idc9dc63a1b1c0a4a203c403dbb3c2ea521099b26
Bug: 342644760
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5744333
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Allen Li <ayatane@chromium.org>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
9 months ago
Gavin Mak 99198b3f70 Skip tests not supported in non-git env if in non-git env
Some depot_tools commands now fail explicitly on cog. Skip their
non-supported tests too if on cog.

Bug: b/333744051
Change-Id: Ifa4116d1f8951d4cc28826570c4ea06773a71502
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5742101
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
9 months ago
Allen Li 03434d28f4 [git_auth] Add git_auth tests
Bug: b/351024645
Change-Id: Ie8808de63d09c4a89187f81a5782e09fe73c8b0f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5698748
Commit-Queue: Allen Li <ayatane@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
9 months ago
Junji Watanabe d3d4c0ac53 autoninja: Use AUTONINJA_BUILD_ID for RBE_invocation_id as is.
To use build id/invocation id consistently, reclient_helper will set AUTONINJA_BUILD_ID as RBE_invocation_id as is.
The host name information will be added as a metrics label.

Bug: 355127782
Change-Id: I5609ef38d897b19029795adf1b288898986c7c70
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5737676
Auto-Submit: Junji Watanabe <jwata@google.com>
Reviewed-by: Michael Savigny <msavigny@google.com>
Commit-Queue: Michael Savigny <msavigny@google.com>
9 months ago
Robert Iannucci a6502426b5 [git_cl] Remove uses of RunGit which interact with git config.
The previous interactions were incorrect w.r.t. scm.GIT's config
cache.

This moves depot_tools towards having one fewer git wrapper
(converging towards scm.GIT).

R=ayatane, yiwzhang@google.com

Change-Id: I507628b53f6a87a1eecbbe3e1e27c1eb6af3b878
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5648617
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
9 months ago
Junji Watanabe b12257963c autoninja: Replace ninjalog_uploader_wrapper.py and reclient_metrics.py with build_telemetry.py
This CL unifies the opt-in/opt-out handling for build telemetry collections about Reclient and Ninjalog.

The user consent message will be displayed only once at the beginning of a build.

```
❯ autoninja -C out/deterministic-andorid-dbg base
*** NOTICE ***
Google-internal telemetry (including build logs, username, and hostname) is collected on corp machines to diagnose performance and fix build issues. This reminder will be shown 9 more times. See http://go/chrome-build-telemetry for details. Hide this notice or opt out by running: build_telemetry [opt-in] [opt-out]
*** END NOTICE ***

Proxy started successfully.
...
```

Bug: 345113094
Change-Id: Ie5886287c4bd20262be0ff247508ac3869441eb1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5669094
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Michael Savigny <msavigny@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
9 months ago
Allen Li 397bf12548 [git_cl] Default to SSO with missing email
This is a safer default if SSO is available

Bug: b/351071334
Change-Id: I2d6b3b5c0fbe3fb7b9783de3d7548be7f14d7391
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5723448
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Allen Li <ayatane@chromium.org>
9 months ago
Allen Li 9a92cd9b69 [git_cl] Fix ShouldUseSSO tests
They weren't testing what they were supposed to be testing and just
happened to pass.

Bug: b/351071334
Change-Id: I2cdd4fe3dc4b09509c73042e81c78949da5f9ac0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5723254
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Allen Li <ayatane@chromium.org>
9 months ago
Allen Li 91937bf196 [git_cl] Parametrize email in ShouldUseSSO
Moves the dependency on Git+cwd up the call stack

Bug: b/351071334
Change-Id: Ia313f9d4720ee10398b757217c333118e9fc7341
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5723091
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Allen Li <ayatane@chromium.org>
9 months ago
Allen Li 4986d4a74b [git_cl] Parametrize cwd in ShouldUseSSO
Remove implicit/global assumptions which could affect tests.

Bug: b/351071334
Change-Id: Ie266228f404b768cb539fdc17dddbbb13693e939
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5723208
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Allen Li <ayatane@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
9 months ago
Scott Lee 1c81122f69 [depot_tools] retry git config if it lock-fails
If git config set is executed, it writes the new content into
a temp file (but called lock-file) and then replace .gitconfig
with it.

However, if it cannot create the lock file, it returns an error.
With this CL, if git config fails with the lock file error,
depot_tools will retry it at most 5 times with 0.2s interval.

It's found that there are applications, such as vscode extensions,
executes `git config set` frequently, and those could often cause
unexpected interruptions to ongoing `git rebase-updates`
by the lock failure.

Bug: 351950514
Change-Id: I985af0d8b7458dbf47cd6baa857dc5adccf15031
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5705561
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Scott Lee <ddoman@chromium.org>
9 months ago
Robert Iannucci 14ddf6e8ba [scm] Refactor git config state to be fully mock-able.
Previously little bits of the scm.GIT were mocked (like SetConfig,
and GetConfig, but not the other config related methods).

This changes things so that the git config state is a class whose
logic is shared between prod and test, with a 'real' and 'test'
implementation which know how to load and save configuration at
a low level.

R=yiwzhang

Change-Id: I1dc11b550908862607ea539de7fa60fbce30e700
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5660891
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Allen Li <ayatane@chromium.org>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
9 months ago
Robert Iannucci 779f70fd7c [gerrit_util] Move Authenticator to be private in gerrit_util.
This should help give additional confidence while refactoring in
gerrit_util.

R=ayatane, yiwzhang

Change-Id: I03927e072e62f6109571ab699f90db7c51ccc6c0
Bug: b/335483238
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5665455
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
10 months ago
Junji Watanabe 01e29c8b5d autoninja: Implement build_telemetry utils
This will replace the user consent logics in ninjalog_uploader_wrapper.py and reclient_metrics.py

See also https://crrev.com/c/5669094 for how this will be used.

Bug: 345113094
Change-Id: Iffc4975c152ba63f7577c09e5254c499fe8973c0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5671052
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Auto-Submit: Junji Watanabe <jwata@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
10 months ago
Junji Watanabe 1387a8c958 autoninja: Refactor the code calling Reclient
- Remove reclient_helper.py to avoid calling it directly.
- Cleans up autoninja.py

I just want to do this cleanup before merging the user notice messages
for logs/metrics collection.

Bug: 345113094
Change-Id: I0c76426b3cb387eae6dede031727c107e57d5d1a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5668282
Reviewed-by: Philipp Wollermann <philwo@google.com>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Ben Segall <bentekkie@google.com>
Auto-Submit: Junji Watanabe <jwata@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
10 months ago
Daniel Rubery 9e9fc99280 Fallback to looking up CL issue from triplet_id
In nosquash mode, `git cl` is pretty challenging to use since we don't
have issue numbers. All three parts of the triplet id are readily
available though. This CL adds a fallback to `git cl` which looks up the
issue number by getting the change details by triplet_id.

Change-Id: I0839fe75bcb4bc8d60ff36b4da26dc0e419a1493
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5633186
Commit-Queue: Daniel Rubery <drubery@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
10 months ago
Fumitoshi Ukai d7d8221581 gn_helper: ignore spaces on gn lines
Change-Id: I2f676411a5282c938fc1d095acbbde0bbe6efde3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5668281
Commit-Queue: Junji Watanabe <jwata@google.com>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
10 months ago
Patrick Monette 367c6f720d Add git squash-branch-tree
This script allows squashing a branch and all of its downstream
branches in one command.

This is useful for squashing stacked branches that have a lot of
commits each just before using `git rebase-update`, as this is
highly likely to run into a lot of rebase conflicts, repeatedly.

This should only be used if the user is okay with losing their
git commit history.

Change-Id: Id28a941d898def73f785d791499c7ee0784e0136
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5659262
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Patrick Monette <pmonette@chromium.org>
10 months ago
Gavin Mak ea802a77b7 Don't show ".gitcookies have credentials" if no creds
Running `git cl creds-check` without any credentials will show:

```
Your .gitcookies have credentials for these hosts:
No Git/Gerrit credentials found
```

The first line of the output should not assume that there are hosts. Don't show this line if credentials are not found.

Change-Id: Ie6eba3518f4b270eef5fbe0c51b4494762a3aef4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5663779
Reviewed-by: Scott Lee <ddoman@chromium.org>
Auto-Submit: Gavin Mak <gavinmak@google.com>
Commit-Queue: Scott Lee <ddoman@chromium.org>
10 months ago
Fumitoshi Ukai 356ef0324e ninja: error if trying to build for use_remoteexec=true
Change-Id: Ia32dd3cba1d1874401c6614f792f212b2cfa60a1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5660200
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Junji Watanabe <jwata@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
10 months ago
Allen Li f4e8e13e8b [gerrit_util] Add linked account detection for SSO
Bug: b/348024314
Change-Id: Ic982de2892769870805407c6a00856943133dd62
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5651293
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Allen Li <ayatane@chromium.org>
10 months ago
Gavin Mak 8ac3425344 Create scm.DIFF.GetAllFiles
For a ProvidedDiffChange, the AllFiles method naively returns all files
with rglob("*"). The returned list includes files in nested submodules.
This does not match the behavior of GitChange's AllFiles which uses
git ls-files to find all files.

Implement a new SCM that stops iterating recursively when it
sees a submodule from .gitmodules in the repo root.

Bug: b/323243527
Change-Id: I170d0f1bc4a838acea04779dee3df7fca0bce359
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5648616
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
10 months ago
Allen Li bb38aa2f8e [gerrit_util] Use SSO only if configured with appropriate email
So you can use chromium.org or other accounts in some repos

Bug: b/348024314
Change-Id: Ice6b6d9e6e827a606a2ce7f3b127d4660df1aedf
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5641255
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Allen Li <ayatane@chromium.org>
10 months ago
Robert Iannucci 137bb69871 [gerrit_util] Cache resolved Authenticator as a class variable.
I noticed that the Authenticator is resolved maybe 5 or 6 times per
git-cl invocation. This should lead to more consistent behavior and
will likely be a bit faster, especially for SSOAuthenticator and
LuciAuthAuthenticator which involve subprocess invocations.

R=ayatane@chromium.org

Bug: 336351842
Change-Id: Id6c2873a6960a171305560acb98afe2c4f397295
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5589865
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
10 months ago
Allen Li bdf64705c3 [gerrit_util] Factor out SSOHelper
Add a layer of abstraction/isolation for general organization.

Also, this logic needs to be used in Git setup too, not just Gerrit
authentication.

Bug: b/348024314
Change-Id: Ie1310a9b8e71c05c72a4b987dcbff76b70c67945
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5645906
Commit-Queue: Allen Li <ayatane@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
10 months ago
Robert Iannucci c4c3d5326e [gerrit_util] Add some additional tests and fixes for SSOAuthenticator.
Unfortunately, the depot_tools presubmit builders are incredibly slow
which make the subprocess based tests fail flakily. I've marked them
all as `skip` with an optional way to run them locally.

R=ayatane, yiwzhang

Bug: b/335483238
Change-Id: I407aed3a1ed01563a0a80973b679aca405b9cde9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5641259
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
10 months ago
Gavin Mak 47a841a204 Handle empty diff in _diffs_to_change_files
An valid empty diff can be passed into _diffs_to_change_files if the
modified file's contents is same as upstream. In this case,
_diffs_to_change_files gets an IndexError.

Bug: b/336555565
Change-Id: I848e6016a1e8089473ff8a72d2e0142fbabfda9f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5635166
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
10 months ago
Robert Iannucci e5490cc303 [gerrit_util] Fix unstopped mocks.
R=ayatane, yiwzhang

Bug: b/335483238
Change-Id: I9df8c754f25de2ad51cac78eecc2ede08eff5fa1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5646325
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Allen Li <ayatane@chromium.org>
10 months ago
Josip Sokcevic 2517f89cdf Reland "[presubmit checks] Check if files are written to a dep dir"
This is a reland of commit 561772c448

Original change's description:
> [presubmit checks] Check if files are written to a dep dir
>
> No files should be written to a directory that's used by CIPD or GCS, as
> defined in DEPS. Git already doesn't allow files to be written to a
> directory that's a gitlink.
>
> R=jojwang@google.com
>
> Bug: 343199633
> Change-Id: I8d3414eac728580eaf9ac7e337bb22bca3989e4e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5633187
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>

Bug: 343199633
Change-Id: Ifc3f6c4df328cdd215ceb7d0333f68223e9a1ccb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5643922
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
10 months ago
Robert Iannucci 1f4f982beb [git-cl] SSOAuthenticator: Add tests and fix bugs.
Adds a few very basic tests.

Fixes bug where, when using python3.8, the cookie jar would not
correctly parse the cookie file.

R=ayatane

Bug: b/335483238
Change-Id: If44eea00d67cb2716df460ef0af93811e351f764
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5637936
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
10 months ago
Gavin Mak 4d36c419f3 Revert "[presubmit checks] Check if files are written to a dep dir"
This reverts commit 561772c448.

Reason for revert: `KeyError: 'deps'` breaking some builds

Original change's description:
> [presubmit checks] Check if files are written to a dep dir
>
> No files should be written to a directory that's used by CIPD or GCS, as
> defined in DEPS. Git already doesn't allow files to be written to a
> directory that's a gitlink.
>
> R=jojwang@google.com
>
> Bug: 343199633
> Change-Id: I8d3414eac728580eaf9ac7e337bb22bca3989e4e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5633187
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>

Bug: 343199633
Change-Id: I26b8e72320260e394a72aee1f7e67dfc2b3e2987
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5641258
Auto-Submit: Gavin Mak <gavinmak@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
10 months ago
Josip Sokcevic 561772c448 [presubmit checks] Check if files are written to a dep dir
No files should be written to a directory that's used by CIPD or GCS, as
defined in DEPS. Git already doesn't allow files to be written to a
directory that's a gitlink.

R=jojwang@google.com

Bug: 343199633
Change-Id: I8d3414eac728580eaf9ac7e337bb22bca3989e4e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5633187
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
10 months ago
Yiwei Zhang 66df2a3ec7 git cl upload: introduce commit-description option
This new option allows programatically providing new
description when uploading a new patchset.

Bug: 346174899
Change-Id: I6d7e361b7da0fd06675c04df4fd2f68c7954dd40
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5620640
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
11 months ago
Yiwei Zhang fdfc24e36a run tests using python 3.8
This is achieved by duplicating vpython spec in tests directory and
changing the python version to 3.8. The reason for this change is that
even though depot_tools support 3.11, the entire infra is still on
3.8 so any new 3.8+ feature used in depot_tools may break infra test and
it has already happened couple of times.

Bug: 346784198
Change-Id: I7dbd0d926da21854abf47af2c2f6edfeec2c5c4d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5625090
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
11 months ago
Richard Wang 6f180c0a23 Use $HOME/.config/depot_tools on linux for .cfg files
Or $XDG_CONFIG_HOME/.config/depot_tools if set

A followup CL can add support for win/macos.

Bug: b/345092320
Change-Id: I877baa4d7fd912b42cfcd88ad0aa347b700a89f2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5604606
Commit-Queue: Richard Wang <richardwa@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
11 months ago
Robert Iannucci ad3ad021d9 [gerrit_util] Invert Authenticator model.
Instead of having CreateHttpConn manipulate the request, with the
Authenticator only able to provide the Authorization header value,
the Authenticator now gets the ability to manipulate the entire
HttpConn object.

This will be used for a new Authenticator method which needs to
include a proxy, cookies, and also manipulate the target request
URI, in addition to providing an Authorization header value.

R=ayatane@chromium.org, yiwzhang@google.com

Bug: 336351842
Change-Id: Ia7d0bbfbb907d8ab6c6d12d000f514fa7afc7245
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5585665
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Allen Li <ayatane@chromium.org>
11 months ago
Robert Iannucci 840e538154 [gerrit_util] Add stronger type annotations around CreateHttpConn.
This also fixes a potential bug where ReadHttpJsonResponse could
improperly return None if the server had a completely empty reply.
ReadHttpJsonResponse will now return an empty dictionary in this
case (which is the assumption that most of the callsites are
making).

R=yiwzhang@google.com

Bug: 336351842
Change-Id: I0aa88e233563a0685b6c0f32ea77ad3e094b9cbc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5585184
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
11 months ago
Robert Iannucci 6427b94bc2 [git_cl] Refactor EnsureAuthenticated.
This will allow us to invert the main Authenticator API.

R=yiwzhang@google.com

Bug: 336351842
Change-Id: Iebe06f68373b99685809849bcd96371e7634d6ef
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5582406
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
11 months ago
Robert Iannucci c57b7ed364 [gerrit_util] Change Authenticator API to return proxy info.
This will be used with an upcoming SSOAuthenticator implementation
which will need to proxy all http requests for Googlers.

R=ayatane, gavinmak@google.com

Bug: 336351842
Change-Id: If8cbb8db51fce198e704f109232868421130b40c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5582100
Commit-Queue: Gavin Mak <gavinmak@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
11 months ago
Junji Watanabe 58c6521691 Add per-file autoninja_test.py=file://BUILD_OWNERS to tests/OWNERS
Change-Id: I0f2edc79ba52296d21a8303608c3231cb87bae4f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5568703
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Auto-Submit: Junji Watanabe <jwata@google.com>
11 months ago
Robert Iannucci a85dcffff4 [git_cl] Refactor away a use of isinstance.
Previously when composing a debugging trace, it would use an
isinstance check to special-case inclusion of a gitcookies file.

This CL refactors this to be part of the Authenticator API, instead,
which means that we will always get some sort of authenticator log
file in a trace, even if it's empty.

This also provides an affordance to add debugging information for
the other authenticator types later.

R=ddoman@chromium.org, gavinmak@google.com

Bug: 336351842
Change-Id: Idd6f45ea60b089f9b2391b5527c5281f67421043
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5571497
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
11 months ago
Junji Watanabe 450c36420d autoninja: Remove autosiso
autosiso has been deprecated. Nobody should be using it now.

Bug: 342038479
Change-Id: I3b22b9fc6db5b4693e954818c5aed58a3e4b4a0d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5567813
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Auto-Submit: Junji Watanabe <jwata@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Junji Watanabe <jwata@google.com>
11 months ago
Robert Iannucci 3ea849f363 [git_cl.py] Remove all usage and messages around netrc.
netrc was already legacy in 2017:
fe30f18fc5

The removed gerrit-init.sh script was written in 2013 and no longer
works.

R=yiwzhang@google.com

Bug: 336351842
Change-Id: Ia8e845899973eb5ead66aee2f54078c09d80ef28
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5564197
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
11 months ago
Robert Iannucci 2f779d3061 [git_cl] Minor type annotation and initialization cleanup.
Noticed that the comments around lazily initialization on settings
mentioned that it does logging or something like this, but in fact
the __init__ method is purely vanilla.

R=gavinmak@google.com

Bug: 336351842
Change-Id: I9d8120001690b2bbd52ac2172d346ce95e2dddd3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5565083
Reviewed-by: Gavin Mak <gavinmak@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
11 months ago
Bruce Dawson 062ecac69f Use git_common to call git
In preparation for some Windows optimizations to how git_common calls
git it is important to use git_common more widely, specifically from
scm.py and gclient_scm.py. This change updates scm.py and gclient_scm.py
and updates the associated tests:

Test command lines used when updating the tests include:
vpython3 tests/gclient_scm_test.py ManagedGitWrapperTestCaseMock.testUpdateConflict

vpython3 tests/gclient_scm_test.py GerritChangesTest.testRecoversAfterPatchFailure

vpython3 tests/gerrit_util_test.py CookiesAuthenticatorTest.testGetGitcookiesPath

Bug: 332982922
Change-Id: I7aacb110b2888c164259815385cd77e26942adc7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5478509
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
11 months ago