This is a reland of commit 6817010e83
Additional Changes:
This reland fixes https://crbug.com/1330995 by stopping gclient from accidentally updating the scheme when one is not present.
Original change's description:
> Update fetch protocol using --protocol-override flag
>
> This CL updates gclient sync to use the protocol of the URL specified in the solutions for cloning all the child dependencies of it.
>
> Bug: chrome-operations:170
> Change-Id: I33588059788b677fbae8c3b434100af5c7979a67
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3631600
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Bug: 1330995
Change-Id: I1447a5e884e41d671d8556c35193f1635f2f6936
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3684112
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
This reverts commit 6817010e83.
Reason for revert: Breaks gclient sync for flutter
Original change's description:
> Update fetch protocol using --protocol-override flag
>
> This CL updates gclient sync to use the protocol of the URL specified in the solutions for cloning all the child dependencies of it.
>
> Bug: chrome-operations:170
> Change-Id: I33588059788b677fbae8c3b434100af5c7979a67
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3631600
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Bug: chrome-operations:170
Change-Id: I76a49030f48fa266eeea336307e7153b22ded4e9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3684110
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
This CL updates gclient sync to use the protocol of the URL specified in the solutions for cloning all the child dependencies of it.
Bug: chrome-operations:170
Change-Id: I33588059788b677fbae8c3b434100af5c7979a67
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3631600
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
This CL adds a non-functional `--protocol` flag to fetch.py. This will be used in the follow-up CL to update fetching protocol.
Bug: 1322156
Change-Id: I7eeebb9993face20bb671d6942ee23fd04802d8b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3609453
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
This CL updates argument parsing mechanism from using a custom implementation + optparse to argparse. It is intended to clear tech debt for future changes.
Change-Id: I06ba5d3c532638a7970be960e02ebbafbea9dcb0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3541479
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
This includes a few fixes for specific errors, and disables several new
warnings introduced in this version, in order to allow for an incremental migration.
Bug:1262286
Change-Id: I4b8f8fc521386419a3121bbb07edc8ac83170a94
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3413679
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
This reverts commit 22bf605bb6.
Reason for revert: breaks gclient sync
Original change's description:
> Use pylint 2.7 for depot_tools
>
> This includes a few fixes for specific errors, and disables several new
> warnings introduced in this version, in order to allow for an incremental migration.
>
> Bug:1262286
> Change-Id: Ie97d686748c9c952e87718a65f401c5f6f80a5c9
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3400616
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Bug: 1262286
Change-Id: Ieb946073c7886c7bf056ce843a5a48e82becf7a5
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3413672
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
This includes a few fixes for specific errors, and disables several new
warnings introduced in this version, in order to allow for an incremental migration.
Bug:1262286
Change-Id: Ie97d686748c9c952e87718a65f401c5f6f80a5c9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3400616
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Running `fetch v8` with Python 3.5 produces the following error:
TypeError: the JSON object must be str, not 'bytes'
Adding `.decode("utf-8")` makes it compatible with both versions.
Change-Id: Ib0699b61b24f191559c30f1e7ca8d2c919803d03
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2154108
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
In Python 3 the output of a subprocess is bytes, so they need to be decoded to be used as a string.
This previously caused the following error if there was an existing checkout during a fetch:
Traceback (most recent call last):
File "C:\Google\depot_tools\\fetch.py", line 318, in <module>
sys.exit(main())
File "C:\Google\depot_tools\\fetch.py", line 313, in main
return run(options, spec, root)
File "C:\Google\depot_tools\\fetch.py", line 299, in run
if not options.force and checkout.exists():
File "C:\Google\depot_tools\\fetch.py", line 98, in exists
return (os.path.exists(os.path.join(gclient_root, b'.gclient')) or
File "C:\Program Files\Python38\lib\ntpath.py", line 109, in join
genericpath._check_arg_types('join', path, *paths)
File "C:\Program Files\Python38\lib\genericpath.py", line 151, in _check_arg_types
raise TypeError("Can't mix strings and bytes in path components") from None
TypeError: Can't mix strings and bytes in path components
Bug: 939847
Change-Id: Ibf5b4923268595ba439586d688894f92696ecbb7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1701403
Auto-Submit: Raul Tambre <raul@tambre.ee>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Raul Tambre <raul@tambre.ee>
Bash and its descendants allow having $PATH entries containing literal tildes,
like:
...:~/depot_tools:...
If the user has their shell configured this way, their shell will be able to
invoke `fetch` and other depot_tools commands, but depot_tools will not in turn
be able to invoke subcommands via popen because popen does not have this
bash-specific behavior when not in shell mode.
This change has gclient detect this configuration problem and error out with a
descriptive error message. Not doing so leads to a puzzling condition where
gclient complains about being unable to find a program that (from the user's
perspective) is in their $PATH and invokable interactively.
It also has 'fetch' suppress its own failure stack trace if it runs gclient and
gclient fails; the 'fetch' stack trace pushes the messages from gclient (which
are actually diagnostic) off the screen.
Bug: chromium:952865
Change-Id: Ibba4d2fccee405aa68392ce141493f1de21ec018
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1573204
Commit-Queue: Elly Fong-Jones <ellyjones@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
The scripts still work with Python 2.
There are no intended behaviour changes.
Bug: 939847
Change-Id: Icada60c5b2cf351d62aead26b7364fcef2c2a3e5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1524486
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Raul Tambre <raul@tambre.ee>
After the initial gclient sync, "fetch --no-history" was adding a
fetchspec to the config for the main repository that causes all tags to
be downloaded in future. This means that after fetching a repository
with no history, the very next git pull (or other remote update
operation) will download all the tags, including all the history
referred to by those tags, leaving you with an almost-complete checkout
of the top-level repository that takes up much more space.
Fix this by skipping the "git config" command that requests the tags
when using no-history.
Change-Id: Ia31baa6a90df014ebc1e8c2ed6e05de3ad27d415
Reviewed-on: https://chromium-review.googlesource.com/909653
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Richard Coles <torne@chromium.org>
Add an option to not store stdout when running commands unless it's needed.
R=agable@chromium.org
Bug: 722686
Change-Id: I402c83767097d53e588ba3e8bca89291712a572f
Reviewed-on: https://chromium-review.googlesource.com/888584
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Several tools, including the "git" recipe module, hard-code a
checkout-relative "git.bat" path. Git is a feature that is provided by
the system, both to tooling and recipes:
1) For users, "depot_tools" must be on PATH, and during setup it will
have installed "git.bat", ensuring that Git tooling is available in
PATH.
2) For bots, the system is responsible for providing "git.bat" on PATH.
This is typically done at "/b/depot_tools/git.bat", which is sync'd
through the "update_scripts" step.
By formally treating Git as a system resource, we absolve Windows bots
and users from manually installing a depot_tools-local Git, bringing
them in line with other platforms.
BUG=chromium:590806
TEST=local
Change-Id: I93e89855cdd330a2ba7a8cfb8117a1789d1ab54e
Reviewed-on: https://chromium-review.googlesource.com/568694
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
This was breaking presubmit because the linked gae_ts_mon subdir was absent.
R=andybons@chromium.org, dsansome@chromium.org, dnj@chromium.org
BUG=608490
Review-Url: https://codereview.chromium.org/1959193002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@300511 0039d316-1c4b-4281-b951-d872f2087c98
We are making depot_tools a proper (chromium) recipe package, which assumes
that recipes are located in recipes/. So I need to move these other kinds of
recipes out of the way.
BUG=564920
R=dpranke@chromium.org, iannucci@chromium.org
Review URL: https://codereview.chromium.org/1494793002 .
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@297837 0039d316-1c4b-4281-b951-d872f2087c98
This reverts commit 78624b6839.
The reason is that install-build-deps.sh isn't meant to be run on mac
or windows, and this implementation ran it everywhere.
TBR=dpranke@chromium.org
BUG=229270
Review URL: https://codereview.chromium.org/1186893003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@295686 0039d316-1c4b-4281-b951-d872f2087c98
Handle KeyboardInterrupt gracefully rather the printing a
backtrace. Most users of these tools don't expect a
backtrace when then hit Ctrl-C.
Also, fix a few other inconsistencies found in the python
startup code of these different scripts:
- always call main function 'main' (rather than 'Main')
- always return 0 from main function
- if main takes args never include argv[0]
Review URL: https://codereview.chromium.org/955993006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294250 0039d316-1c4b-4281-b951-d872f2087c98
Original CL: https://codereview.chromium.org/440263002/
Many people* have complained on chromium-dev about the long times
required to perform a full fetch over a DSL. This seems to be mostly
due to the huge size of chromium's history (~9 GB). On the other side,
not everybody is interested in downloading the full git history of
the projects. The size of git packs required to fetch a working HEAD
is one order of magnitude smaller (1.5 GB).
This change makes it possible to perform a shallow fetch (in a way
which is consistent with DEPS, leveraging git templates on clone),
reducing fetch times by 80% for those not interested in the history.
* See:
[chromium-dev] "fetch chromium" keeps hanging/getting stuck on Windows 7
[chromium-dev] Initial checkout with git taking long
[chromium-dev] Trying to get latest source code fails when fetching
[chromium-dev] Gclient sync takes too long
BUG=228996
TBR=iannucci@chromium.org,szager@chromium.org,wtc@chromium.org
Review URL: https://codereview.chromium.org/440273002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@287793 0039d316-1c4b-4281-b951-d872f2087c98
Reason for revert:
Broke the WebRTC waterfall:
http://build.chromium.org/p/tryserver.webrtc/builders/win/builds/3958/steps/gclient%20revert/logs/stdio
Original issue's description:
> Add --no-history option to fetch and gclient for shallow clones.
>
> Many people* have complained on chromium-dev about the long times
> required to perform a full fetch over a DSL. This seems to be mostly
> due to the huge size of chromium's history (~9 GB). On the other side,
> not everybody is interested in downloading the full git history of
> the projects. The size of git packs required to fetch a working HEAD
> is one order of magnitude smaller (1.5 GB).
> This change makes it possible to perform a shallow fetch (in a way
> which is consistent with DEPS, leveraging git templates on clone),
> reducing fetch times by 80% for those not interested in the history.
>
> * See:
> [chromium-dev] "fetch chromium" keeps hanging/getting stuck on Windows 7
> [chromium-dev] Initial checkout with git taking long
> [chromium-dev] Trying to get latest source code fails when fetching
> [chromium-dev] Gclient sync takes too long
>
> BUG=228996
>
> Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=287606
TBR=iannucci@chromium.org,szager@chromium.org,wtc@chromium.org
NOTREECHECKS=true
NOTRY=true
BUG=228996
Review URL: https://codereview.chromium.org/440263002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@287637 0039d316-1c4b-4281-b951-d872f2087c98
Many people* have complained on chromium-dev about the long times
required to perform a full fetch over a DSL. This seems to be mostly
due to the huge size of chromium's history (~9 GB). On the other side,
not everybody is interested in downloading the full git history of
the projects. The size of git packs required to fetch a working HEAD
is one order of magnitude smaller (1.5 GB).
This change makes it possible to perform a shallow fetch (in a way
which is consistent with DEPS, leveraging git templates on clone),
reducing fetch times by 80% for those not interested in the history.
* See:
[chromium-dev] "fetch chromium" keeps hanging/getting stuck on Windows 7
[chromium-dev] Initial checkout with git taking long
[chromium-dev] Trying to get latest source code fails when fetching
[chromium-dev] Gclient sync takes too long
BUG=228996
Review URL: https://codereview.chromium.org/437903002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@287606 0039d316-1c4b-4281-b951-d872f2087c98
Aravind Vasudevan
Josip Sokcevic
Jamie Madill
Milad Farazmand
Edward Lemur
James Darpinian
Raul Tambre
Elly Fong-Jones
Torne (Richard Coles)
Aaron Gable
Dan Jacques
Dr Alex Gouaillard
iannucci@chromium.org
luqui@chromium.org
thestig@chromium.org
mmoss@chromium.org
sbc@chromium.org
iannucci@chromium.org
primiano@chromium.org
jochen@chromium.org
szager@chromium.org