In Python 3 the output of a subprocess is bytes, so they need to be decoded to be used as a string.
This previously caused the following error if there was an existing checkout during a fetch:
Traceback (most recent call last):
File "C:\Google\depot_tools\\fetch.py", line 318, in <module>
sys.exit(main())
File "C:\Google\depot_tools\\fetch.py", line 313, in main
return run(options, spec, root)
File "C:\Google\depot_tools\\fetch.py", line 299, in run
if not options.force and checkout.exists():
File "C:\Google\depot_tools\\fetch.py", line 98, in exists
return (os.path.exists(os.path.join(gclient_root, b'.gclient')) or
File "C:\Program Files\Python38\lib\ntpath.py", line 109, in join
genericpath._check_arg_types('join', path, *paths)
File "C:\Program Files\Python38\lib\genericpath.py", line 151, in _check_arg_types
raise TypeError("Can't mix strings and bytes in path components") from None
TypeError: Can't mix strings and bytes in path components
Bug: 939847
Change-Id: Ibf5b4923268595ba439586d688894f92696ecbb7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1701403
Auto-Submit: Raul Tambre <raul@tambre.ee>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Raul Tambre <raul@tambre.ee>
Bash and its descendants allow having $PATH entries containing literal tildes,
like:
...:~/depot_tools:...
If the user has their shell configured this way, their shell will be able to
invoke `fetch` and other depot_tools commands, but depot_tools will not in turn
be able to invoke subcommands via popen because popen does not have this
bash-specific behavior when not in shell mode.
This change has gclient detect this configuration problem and error out with a
descriptive error message. Not doing so leads to a puzzling condition where
gclient complains about being unable to find a program that (from the user's
perspective) is in their $PATH and invokable interactively.
It also has 'fetch' suppress its own failure stack trace if it runs gclient and
gclient fails; the 'fetch' stack trace pushes the messages from gclient (which
are actually diagnostic) off the screen.
Bug: chromium:952865
Change-Id: Ibba4d2fccee405aa68392ce141493f1de21ec018
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1573204
Commit-Queue: Elly Fong-Jones <ellyjones@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
The scripts still work with Python 2.
There are no intended behaviour changes.
Bug: 939847
Change-Id: Icada60c5b2cf351d62aead26b7364fcef2c2a3e5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1524486
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Raul Tambre <raul@tambre.ee>
After the initial gclient sync, "fetch --no-history" was adding a
fetchspec to the config for the main repository that causes all tags to
be downloaded in future. This means that after fetching a repository
with no history, the very next git pull (or other remote update
operation) will download all the tags, including all the history
referred to by those tags, leaving you with an almost-complete checkout
of the top-level repository that takes up much more space.
Fix this by skipping the "git config" command that requests the tags
when using no-history.
Change-Id: Ia31baa6a90df014ebc1e8c2ed6e05de3ad27d415
Reviewed-on: https://chromium-review.googlesource.com/909653
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Richard Coles <torne@chromium.org>
Add an option to not store stdout when running commands unless it's needed.
R=agable@chromium.org
Bug: 722686
Change-Id: I402c83767097d53e588ba3e8bca89291712a572f
Reviewed-on: https://chromium-review.googlesource.com/888584
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Several tools, including the "git" recipe module, hard-code a
checkout-relative "git.bat" path. Git is a feature that is provided by
the system, both to tooling and recipes:
1) For users, "depot_tools" must be on PATH, and during setup it will
have installed "git.bat", ensuring that Git tooling is available in
PATH.
2) For bots, the system is responsible for providing "git.bat" on PATH.
This is typically done at "/b/depot_tools/git.bat", which is sync'd
through the "update_scripts" step.
By formally treating Git as a system resource, we absolve Windows bots
and users from manually installing a depot_tools-local Git, bringing
them in line with other platforms.
BUG=chromium:590806
TEST=local
Change-Id: I93e89855cdd330a2ba7a8cfb8117a1789d1ab54e
Reviewed-on: https://chromium-review.googlesource.com/568694
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
This was breaking presubmit because the linked gae_ts_mon subdir was absent.
R=andybons@chromium.org, dsansome@chromium.org, dnj@chromium.org
BUG=608490
Review-Url: https://codereview.chromium.org/1959193002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@300511 0039d316-1c4b-4281-b951-d872f2087c98
We are making depot_tools a proper (chromium) recipe package, which assumes
that recipes are located in recipes/. So I need to move these other kinds of
recipes out of the way.
BUG=564920
R=dpranke@chromium.org, iannucci@chromium.org
Review URL: https://codereview.chromium.org/1494793002 .
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@297837 0039d316-1c4b-4281-b951-d872f2087c98
This reverts commit 78624b6839.
The reason is that install-build-deps.sh isn't meant to be run on mac
or windows, and this implementation ran it everywhere.
TBR=dpranke@chromium.org
BUG=229270
Review URL: https://codereview.chromium.org/1186893003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@295686 0039d316-1c4b-4281-b951-d872f2087c98
Handle KeyboardInterrupt gracefully rather the printing a
backtrace. Most users of these tools don't expect a
backtrace when then hit Ctrl-C.
Also, fix a few other inconsistencies found in the python
startup code of these different scripts:
- always call main function 'main' (rather than 'Main')
- always return 0 from main function
- if main takes args never include argv[0]
Review URL: https://codereview.chromium.org/955993006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294250 0039d316-1c4b-4281-b951-d872f2087c98
Original CL: https://codereview.chromium.org/440263002/
Many people* have complained on chromium-dev about the long times
required to perform a full fetch over a DSL. This seems to be mostly
due to the huge size of chromium's history (~9 GB). On the other side,
not everybody is interested in downloading the full git history of
the projects. The size of git packs required to fetch a working HEAD
is one order of magnitude smaller (1.5 GB).
This change makes it possible to perform a shallow fetch (in a way
which is consistent with DEPS, leveraging git templates on clone),
reducing fetch times by 80% for those not interested in the history.
* See:
[chromium-dev] "fetch chromium" keeps hanging/getting stuck on Windows 7
[chromium-dev] Initial checkout with git taking long
[chromium-dev] Trying to get latest source code fails when fetching
[chromium-dev] Gclient sync takes too long
BUG=228996
TBR=iannucci@chromium.org,szager@chromium.org,wtc@chromium.org
Review URL: https://codereview.chromium.org/440273002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@287793 0039d316-1c4b-4281-b951-d872f2087c98
Reason for revert:
Broke the WebRTC waterfall:
http://build.chromium.org/p/tryserver.webrtc/builders/win/builds/3958/steps/gclient%20revert/logs/stdio
Original issue's description:
> Add --no-history option to fetch and gclient for shallow clones.
>
> Many people* have complained on chromium-dev about the long times
> required to perform a full fetch over a DSL. This seems to be mostly
> due to the huge size of chromium's history (~9 GB). On the other side,
> not everybody is interested in downloading the full git history of
> the projects. The size of git packs required to fetch a working HEAD
> is one order of magnitude smaller (1.5 GB).
> This change makes it possible to perform a shallow fetch (in a way
> which is consistent with DEPS, leveraging git templates on clone),
> reducing fetch times by 80% for those not interested in the history.
>
> * See:
> [chromium-dev] "fetch chromium" keeps hanging/getting stuck on Windows 7
> [chromium-dev] Initial checkout with git taking long
> [chromium-dev] Trying to get latest source code fails when fetching
> [chromium-dev] Gclient sync takes too long
>
> BUG=228996
>
> Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=287606
TBR=iannucci@chromium.org,szager@chromium.org,wtc@chromium.org
NOTREECHECKS=true
NOTRY=true
BUG=228996
Review URL: https://codereview.chromium.org/440263002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@287637 0039d316-1c4b-4281-b951-d872f2087c98
Many people* have complained on chromium-dev about the long times
required to perform a full fetch over a DSL. This seems to be mostly
due to the huge size of chromium's history (~9 GB). On the other side,
not everybody is interested in downloading the full git history of
the projects. The size of git packs required to fetch a working HEAD
is one order of magnitude smaller (1.5 GB).
This change makes it possible to perform a shallow fetch (in a way
which is consistent with DEPS, leveraging git templates on clone),
reducing fetch times by 80% for those not interested in the history.
* See:
[chromium-dev] "fetch chromium" keeps hanging/getting stuck on Windows 7
[chromium-dev] Initial checkout with git taking long
[chromium-dev] Trying to get latest source code fails when fetching
[chromium-dev] Gclient sync takes too long
BUG=228996
Review URL: https://codereview.chromium.org/437903002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@287606 0039d316-1c4b-4281-b951-d872f2087c98
The git path is based on the git version, so it might change whenever git is
updated. Run git.bat instead, since that's updated with new git installs and
should always point to the right path.
Review URL: https://codereview.chromium.org/173393002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@252299 0039d316-1c4b-4281-b951-d872f2087c98
This typo was introduced in https://codereview.chromium.org/106403003.
Rename options.dryrun to options.dry_run.
R=agable@chromium.org,dpranke@chromium.org
BUG=none
TEST=none
Review URL: https://codereview.chromium.org/138623008
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@250802 0039d316-1c4b-4281-b951-d872f2087c98
'fetch chromium' will fail if the build dependencies are not already installed
on the host machine, because it runs 'gclient sync' which, by default, will run
hooks that will try to run build/gyp_chromium (which will then fail).
This introduces a --nohooks option that solve the problem by allowing the
user to do the following:
1) fetch --nohooks <recipe>
2) src/build/install-build-deps.sh
3) fetch <recipe>
BUG=325795
TEST=Run 'fetch --nohooks chromium' in a temporary 'clean' chroot directory.
R=maruel@chromium.org,jochen@chromium.org,scottmg@chromium.org,agable@chromium.org
Review URL: https://codereview.chromium.org/106403003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@240680 0039d316-1c4b-4281-b951-d872f2087c98
Follow the explicit iteration order of the spec keys array instead of relying
on the arbitrary iteration order of the hashtable.
BUG=
Review URL: https://chromiumcodereview.appspot.com/13878009
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@194973 0039d316-1c4b-4281-b951-d872f2087c98
to tell the user how to recover from partial checkouts and
what to do with an existing checkout.
R=iannucci@chromium.org
BUG=230691
Review URL: https://chromiumcodereview.appspot.com/13945020
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@194019 0039d316-1c4b-4281-b951-d872f2087c98
This allows non-committers to use fetch to checkout chromium or blink using
$ fetch chromium --nosvn=true
BUG=230357
R=dpranke@chromium.org
Review URL: https://codereview.chromium.org/13910005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@193713 0039d316-1c4b-4281-b951-d872f2087c98
Edward Lemur
James Darpinian
Raul Tambre
Elly Fong-Jones
Torne (Richard Coles)
Aaron Gable
Dan Jacques
Dr Alex Gouaillard
iannucci@chromium.org
luqui@chromium.org
thestig@chromium.org
mmoss@chromium.org
sbc@chromium.org
iannucci@chromium.org
primiano@chromium.org
jochen@chromium.org
szager@chromium.org
mmoss@chromium.org
wtc@chromium.org
digit@chromium.org
janx@chromium.org
dpranke@chromium.org
agable@chromium.org