Instead of having CreateHttpConn manipulate the request, with the
Authenticator only able to provide the Authorization header value,
the Authenticator now gets the ability to manipulate the entire
HttpConn object.
This will be used for a new Authenticator method which needs to
include a proxy, cookies, and also manipulate the target request
URI, in addition to providing an Authorization header value.
R=ayatane@chromium.org, yiwzhang@google.com
Bug: 336351842
Change-Id: Ia7d0bbfbb907d8ab6c6d12d000f514fa7afc7245
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5585665
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Allen Li <ayatane@chromium.org>
This also fixes a potential bug where ReadHttpJsonResponse could
improperly return None if the server had a completely empty reply.
ReadHttpJsonResponse will now return an empty dictionary in this
case (which is the assumption that most of the callsites are
making).
R=yiwzhang@google.com
Bug: 336351842
Change-Id: I0aa88e233563a0685b6c0f32ea77ad3e094b9cbc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5585184
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Remove the libraries that have no uses according to gd in vim.
BUG=None
TEST=upload this CL
Change-Id: I7fe6bbd54c1a4f00a3d0e3f04be0f02609734e46
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5578897
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gregory Nisbet <gregorynisbet@google.com>
[1] was originally added by an [ancient CL] in 2013.
It runs `git config ...` and checks if if the output is not "False".
It has two problems.
1) `git config` returns False only if the value was set as a text value,
AND the value is read as a text value. If "False" is set with --bool,
the value is "false", even if it's read as a text value.
from https://git-scm.com/docs/git-config
bool: canonicalize values as either "true" or "false".
In other words, the existing check would work only if someone sets
it as a text value with "False".
2) the inline comment, describing the intention, is confusing.
"Skip url auto-correction if gclient-auto-fix-url is set"
It's not clear whether it's set to any value? false? true?
Nevertheless, what the code actually does is to auto-correct the URL,
unless the config is set to False, which would unlikely be true, due
to (1). To make it work, someone needs to set it with "False" as
a text value, but "False" is not even a valid string value for
the corresponding legit boolean value that git config --bool defines.
Therefore, depot_tools had auto-corrected the URL, if the config is
- set to "False" as a boolean value
- set to true, false, bla, foo, bar, or any value
other than "False" as a text value.
OR
- unset
That's how the code had behaved from 2013 to 2024.
In Feb 2024, https://crrev.com/c/5280779 changed the behaviour.
It replaces the dubious check with GetConfigBool(), which may sound
correct by itself.
i.e., auto-correct if "remote.origin.gclient-auto-fix-url" is set to
"true".
However, it doesn't match the existing behaviour, which
auto-correct the URL regardless of the config. Depot tools had the
behaviour for 11 years. Fixing it is probably not the best decision.
This CL doesn't exactly recover the existing behaviour. Instead,
it removes the check. As a result, even if someone sets the config
with "False" as a text value, depot_tools will auto-correct the URL.
[1]: 3b9212b7ee:gclient_scm.py;l=764-768
[ancient CL]: https://chromiumcodereview.appspot.com/15325003
Bug: 342445995
Change-Id: Idc62f803f2ef1beb11a93a58432867ce16226da7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5572721
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Scott Lee <ddoman@chromium.org>
need to split input_args[1] even if it doesn't contains " ",
e.g. when input_args[1] == ""
Bug: b/343172216
Change-Id: I32d9346063534ea2b4f891fd596c7ec02ab10dbf
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5583415
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Richard Wang <richardwa@google.com>
Reviewed-by: Richard Wang <richardwa@google.com>
This will be used with an upcoming SSOAuthenticator implementation
which will need to proxy all http requests for Googlers.
R=ayatane, gavinmak@google.com
Bug: 336351842
Change-Id: If8cbb8db51fce198e704f109232868421130b40c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5582100
Commit-Queue: Gavin Mak <gavinmak@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
This is a reland of commit 7d95eb2eb0
Additional changes:
* introduce deletion blocklist
Original change's description:
> [gclient] Delete GCS output_dir on download
>
> GCS output_directory needs to be cleaned up prior to extracting new
> content. At the same time, it can't be cleaned up by individual GCS
> fetcher as there can be multiple objects in the same output_dir.
>
> This change adds a top level support to delete old GCS output_dir. If
> any of objects need to be downloaded, the output_dir will be completely
> deleted and all objects will be redownloaded.
>
> R=jojwang@google.com
>
> Bug: 342522902, 338612245
> Change-Id: Icbe4f1238cac54d7390bbb9b6fc5f17c538cca62
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5570466
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
> Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
R=jojwang@google.com
Bug: 342522902, 338612245
Change-Id: Ib5335cddfd60fb4d7da54e16aacb71b11413108e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5581228
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Perhaps this is a linux convention but it was not obvious to me that
'y' would use the default.
Change-Id: I5aedcb365828015effa64d41153f6cb9c9318dfc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5580992
Commit-Queue: Peter McNeeley <petermcneeley@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Previously when composing a debugging trace, it would use an
isinstance check to special-case inclusion of a gitcookies file.
This CL refactors this to be part of the Authenticator API, instead,
which means that we will always get some sort of authenticator log
file in a trace, even if it's empty.
This also provides an affordance to add debugging information for
the other authenticator types later.
R=ddoman@chromium.org, gavinmak@google.com
Bug: 336351842
Change-Id: Idd6f45ea60b089f9b2391b5527c5281f67421043
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5571497
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
This reverts commit 7d95eb2eb0.
Reason for revert: Causing bot_update failures: https://issues.chromium.org/issues/342945505
Original change's description:
> [gclient] Delete GCS output_dir on download
>
> GCS output_directory needs to be cleaned up prior to extracting new
> content. At the same time, it can't be cleaned up by individual GCS
> fetcher as there can be multiple objects in the same output_dir.
>
> This change adds a top level support to delete old GCS output_dir. If
> any of objects need to be downloaded, the output_dir will be completely
> deleted and all objects will be redownloaded.
>
> R=jojwang@google.com
>
> Bug: 342522902, 338612245
> Change-Id: Icbe4f1238cac54d7390bbb9b6fc5f17c538cca62
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5570466
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
> Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Bug: 342522902, 338612245, 342945505
Change-Id: I93ed99d7311f15a7a24a03ccba8c310cc7e4c4d4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5570280
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: danakj <danakj@chromium.org>
Reviewed-by: Yann Dago <ydago@google.com>
Reviewed-by: Friedrich Horschig <fhorschig@chromium.org>
Auto-Submit: danakj <danakj@chromium.org>
Owners-Override: danakj <danakj@chromium.org>
This CL allows autoninja to use Siso's builtin RBE client instead of using Reclient.
Developers have to specify `use_remoteexec=true use_reclient=false` GN arg to use this mode.
This should be no-op for all developers unless `use_reclient=false` is set.
Bug: 342038479
Change-Id: I8a67f114c056be6ddb0cd26cccf0157e0b184c5a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5567811
Commit-Queue: Junji Watanabe <jwata@google.com>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
GCS output_directory needs to be cleaned up prior to extracting new
content. At the same time, it can't be cleaned up by individual GCS
fetcher as there can be multiple objects in the same output_dir.
This change adds a top level support to delete old GCS output_dir. If
any of objects need to be downloaded, the output_dir will be completely
deleted and all objects will be redownloaded.
R=jojwang@google.com
Bug: 342522902, 338612245
Change-Id: Icbe4f1238cac54d7390bbb9b6fc5f17c538cca62
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5570466
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
In order for git to be run on Windows and ultimately run git.exe it is
normally necessary to pass shell=True to Popen or else invoke git.bat,
or both. However this means invoking cmd.exe every time we want to
invoke git.exe, which can end up being over 1,500 invocations of cmd.exe
when running the "update" stage of gclient sync. This makes the "update"
stage take about 1.45x as long to run - it's a significant overhead.
Even with optimizations to the update stage of gclient sync in the NOP
case this repeated invocation of cmd.exe still costs about 0.8 s of a
4.9 s "update" stage.
This change teaches git_common how to find git.exe instead of just
git.bat by parsing the .exe name out of the .bat file.
Bug: 332982922
Change-Id: I4e052444276d55c0496144b9816fe365a95a56d6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5462405
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
netrc was already legacy in 2017:
fe30f18fc5
The removed gerrit-init.sh script was written in 2013 and no longer
works.
R=yiwzhang@google.com
Bug: 336351842
Change-Id: Ia8e845899973eb5ead66aee2f54078c09d80ef28
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5564197
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Noticed that the comments around lazily initialization on settings
mentioned that it does logging or something like this, but in fact
the __init__ method is purely vanilla.
R=gavinmak@google.com
Bug: 336351842
Change-Id: I9d8120001690b2bbd52ac2172d346ce95e2dddd3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5565083
Reviewed-by: Gavin Mak <gavinmak@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
In preparation for some Windows optimizations to how git_common calls
git it is important to use git_common more widely, specifically from
scm.py and gclient_scm.py. This change updates scm.py and gclient_scm.py
and updates the associated tests:
Test command lines used when updating the tests include:
vpython3 tests/gclient_scm_test.py ManagedGitWrapperTestCaseMock.testUpdateConflict
vpython3 tests/gclient_scm_test.py GerritChangesTest.testRecoversAfterPatchFailure
vpython3 tests/gerrit_util_test.py CookiesAuthenticatorTest.testGetGitcookiesPath
Bug: 332982922
Change-Id: I7aacb110b2888c164259815385cd77e26942adc7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5478509
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
b8fc6011e50cc017b6773ec4be99f7de6b468b0f cipkg: fix race condition when two packages waiting the execlusive lock simultaneously
89692b880afac19a3cd83922500bd3f474e9cb79 Add --no-periodic-update flag for virtualenv.
e986e645f627a0e9044002f81d1307c66084d8cf [packages] build for GOOS=openbsd GOARCH=ppc64,riscv64
3e19cd7bb78031ef91dc239ca5e570a32305d103 vpython: use realpath when invoking virtualenv
7f1070fd28b718864a306e95cad0b7a757eb0eca Revert "vpython: use realpath when invoking virtualenv"
988aaae74e27c6f8d191b31fcfa77340bfc64d0b vpython: use realpath when invoking virtualenv
f03b35a50ae5912b9031e4ac0dbd3924f92b5bff cipkg: rename root_step_name to step_name
097ceb2f05696dd96f465fa4df800ce008954765 cipkg: better handle execution error
47b5ad06e33f96931416639890ec16fe6db2e045 cipkg: add RootStepName and PackageExecutor
8c0507b3a1d4e4604d1e48d0adb3ace4ac18caf9 cipkg: fix mode override for embed fs
67ea392d049d6e4c238019a4fd8835a66253068f Include relevant build/packages/ changes in the vpython changelog.
a3fa998caff7d6e235febb4e262edfce9b5861e1 [vpython] Update version of virtualenv for python3.11
5055ac4c2310863a7f05584e971ec1abaec8b126 [vpython] Factor out pins for virtualenv package.
Bug: 335552303
Change-Id: I673665e0ef235af2687f1d9727bd337956e336e5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5556721
Auto-Submit: Chenlin Fan <fancl@chromium.org>
Reviewed-by: Brian Ryner <bryner@google.com>
Commit-Queue: Chenlin Fan <fancl@chromium.org>
This fixes an issue where "~" in user http.cookiefile config doesn't get
expanded and causes failures when "git cl" commands tries to read the
credentials.
Expand the path returned by scm.GIT.GetConfig() but not just the default
value.
Bug: b/341619336
Change-Id: Ib89d5b86710a0a6bb6ca669639eeec6adc75689d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5547345
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Fei Shao <fshao@chromium.org>
Joanna Wang
Robert Iannucci
recipe-roller
Gregory NISBET
Josip Sokcevic
Scott Lee
Fumitoshi Ukai
Rob Mohr
Junji Watanabe
Peter McNeeley
danakj
Bruce Dawson
Chenlin Fan
nqmtuan
Fei Shao
Takuto Ikuta