You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
Go to file
Dirk Pranke ef579a1192 Revert "Reland "Update gsutil to use gsutil version 4.61, python3.""
This reverts commit e53a593956.

Reason for revert: Speculative revert for build failures on Mac11 bots: crbug.com/1219065.

Original change's description:
> Reland "Update gsutil to use gsutil version 4.61, python3."
>
> This reverts commit af121aeec9.
>
> Reason for revert: re-landing with a switch back to vpython to get the compiled C extension version of crcmod for performance.
>
> Original change's description:
> > Revert "Update gsutil to use gsutil version 4.61, python3."
> >
> > This reverts commit f059ec9368.
> >
> > Reason for revert: Reverting because we probably need to be using vpython and a compiled crcmod instead. See, e.g.,. b/188591640.
> >
> > Original change's description:
> > > Update gsutil to use gsutil version 4.61, python3.
> > >
> > > This CL updates the gsutil.py wrapper to download and use
> > > v4.61 of GCP's gsutil, which is Python3-compatible.
> > >
> > > v4.61 appears to be fully self-contained and have all of the
> > > packages it needs vendored into it. So, there's no reason to
> > > use vpython anymore, and this CL removes that.
> > >
> > > Also, this CL removes the 'fallback' option to gsutil and
> > > the ability to force a version switch, as this should no
> > > longer be necessary (it was added for a migration back in 2014
> > > but apparently this code was never removed afterwards).
> > >
> > > This CL also updates download_from_google_storage.py and
> > > upload_to_google_storage.py to similarly not have the version flags
> > > and to just use regular python3, not vpython3.
> > >
> > > Bug: 1184108
> > > Change-Id: I0d1a8351dba2d3ad1f927afa333fb10959f19443
> > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2898439
> > > Reviewed-by: Mike Frysinger <vapier@chromium.org>
> > > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> > > Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> > > Commit-Queue: Dirk Pranke <dpranke@google.com>
> >
> > Bug: 1184108
> > Change-Id: I8e21a9a40d81e4e185642f866855b6838f80f1c2
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2905904
> > Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> > Commit-Queue: Dirk Pranke <dpranke@google.com>
>
> Bug: 1184108
> Change-Id: I5d6d6d06842e08517488471c144972818fcf02ff
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2907155
> Reviewed-by: Mike Frysinger <vapier@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Dirk Pranke <dpranke@google.com>

Bug: 1184108
Change-Id: Ib0757e0873d6e6b39621d50d327d959f0fa43720
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2957473
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Dirk Pranke <dpranke@google.com>
4 years ago
bootstrap Roll new cpython(3) packages to depot_tools. 4 years ago
fetch_configs
git-templates
infra Roll recipe dependencies (trivial). 4 years ago
man Quick spelling update to man pages 4 years ago
python-bin Add python batch wrappers 4 years ago
python2-bin Add python batch wrappers 4 years ago
recipes bot_update: Don't collect metrics if not running as part of a build. 4 years ago
testing_support Set default branch in fake_repos 5 years ago
tests Revert "Reland "Update gsutil to use gsutil version 4.61, python3."" 4 years ago
third_party Revert "Improve git cl split" 5 years ago
win_toolchain Fix py3 compat issue in get_toolchain_if_necessary. 4 years ago
zsh-goodies
.cipd_impl.ps1 Try removing windows-386 from cipd_manifest.txt 4 years ago
.gitattributes Reland "repo: auto-update depot_tools when used" 4 years ago
.gitignore Ignore VsChromium config file 4 years ago
.style.yapf
.vpython Preliminary changes to git cl and presubmit_support 5 years ago
.vpython3 Preliminary changes to git cl and presubmit_support 5 years ago
CROS_OWNERS CROS_OWNERS: add more build peeps 4 years ago
DIR_METADATA [depot_tools] Migrate metadata to DIR_METADATA. 4 years ago
GOMA_OWNERS
LICENSE
LUCI_OWNERS tandrii: switch OWNERS to @google.com account. 5 years ago
OWNERS Revert "Remove ehmaldonado@chromium.org from OWNERS" 4 years ago
PRESUBMIT.py Add support for running PRESUBMIT.py checks under Python2 or 3. 4 years ago
README.gclient.md
README.git-cl.md Replace non-inclusive terms 4 years ago
README.md add clarification for Python 3 5 years ago
WATCHLISTS WATCHLISTS: CC chops-source-team@ on all depot_tools changes. 5 years ago
auth.py [auth] log errors from luci-auth token invocation. 4 years ago
autoninja Fix "expr: syntax error" on macOS 4 years ago
autoninja.bat Have autoninja.bat run depot_tools python3 4 years ago
autoninja.py Respect GOMA_DIR in autoninja instead of always assuming .cipd_bin. 4 years ago
bb Add revert instructions 4 years ago
bb.bat Add revert instructions 4 years ago
bootstrap_python3 [mac/arm] pull python3 binary in cipd bootstrap now that it exists 4 years ago
breakpad.py
cbuildbot
chrome_set_ver
cipd Default to arm64 cipd packages on arm macs 4 years ago
cipd.bat [cipd] Take cipd.ps1 out of %PATH%. 5 years ago
cipd_bin_setup.bat
cipd_bin_setup.sh
cipd_client_version [cipd] Update CIPD client to 2.5.2 4 years ago
cipd_client_version.digests [cipd] Update CIPD client to 2.5.2 4 years ago
cipd_manifest.txt [resultdb] Push rdb cli pin forward. 4 years ago
cipd_manifest.versions [resultdb] Push rdb cli pin forward. 4 years ago
cit
cit.bat
cit.py [subprocess2] Replace VOID with DEVNULL 4 years ago
clang-format
clang-format.bat
clang_format.py
clang_format_merge_driver
clang_format_merge_driver.bat
clang_format_merge_driver.py
codereview.settings update stale codereview.chromium.org links 4 years ago
compile_single_file
compile_single_file.bat
compile_single_file.py remove reference to jumbo build 5 years ago
cpplint.bat
cpplint.py CodeInclusion: Rename whitelist/blacklist -> allowlist/blocklist 5 years ago
cpplint_chromium.py
cros cros: raise python version requirement to 3.6 5 years ago
cros_sdk
crosjobs Add revert instructions 4 years ago
detect_host_arch.py gclient: Correctly set host_cpu to arm64 on arm macs 4 years ago
dirmd Add revert instructions 4 years ago
dirmd.bat Add revert instructions 4 years ago
download_from_google_storage Switch download_from_google_storage to py3 4 years ago
download_from_google_storage.bat Revert "Switch download_from_google_storage scripts to invoke python3." 4 years ago
download_from_google_storage.py Revert "Reland "Update gsutil to use gsutil version 4.61, python3."" 4 years ago
ensure_bootstrap
fetch Make Python 3 default on Windows 5 years ago
fetch.bat Abort fetch, gclient, gsutil.py when update_depot_tools fails on Windows 5 years ago
fetch.py Make fetch compatible with py 3.5 5 years ago
fix_encoding.py Revert "Modify fix_encodng to make it use fix_win_sys_argv for python2 only and" 4 years ago
gclient Make Python 3 default on Windows 5 years ago
gclient-new-workdir.py
gclient.bat Abort fetch, gclient, gsutil.py when update_depot_tools fails on Windows 5 years ago
gclient.py gclient: Report hook metrics 4 years ago
gclient_completion.sh
gclient_eval.py Revert "Remove use_relative_hooks now that all deprecated uses are removed." 5 years ago
gclient_paths.py Change FindGclientRoot to Use os.path.abspath instead of os.path.realpath 4 years ago
gclient_scm.py gclient: Stop running git under "arch" on macOS 4 years ago
gclient_utils.py Fix gclient_utils.FileRead() behavior for both 2 and 3. 4 years ago
gerrit_client.py Use py3 with gerrit_client.py 4 years ago
gerrit_util.py Implemented suppport for getrelatedchanges endpoint in gerrit module 4 years ago
git-cache Add quotes around paths in bash scripts 5 years ago
git-cl Add quotes around paths in bash scripts. 5 years ago
git-crrev-parse
git-drover Add quotes around paths in bash scripts 5 years ago
git-find-releases Add quotes around paths in bash scripts 5 years ago
git-footers Add quotes around paths in bash scripts 5 years ago
git-freeze Add quotes around paths in bash scripts 5 years ago
git-gs
git-hyper-blame Add quotes around paths in bash scripts 5 years ago
git-map Add quotes around paths in bash scripts 5 years ago
git-map-branches Add quotes around paths in bash scripts 5 years ago
git-mark-merge-base Add quotes around paths in bash scripts 5 years ago
git-migrate-default-branch Add git-migrate-default-branch 5 years ago
git-nav-downstream Add quotes around paths in bash scripts 5 years ago
git-nav-upstream
git-new-branch Add quotes around paths in bash scripts 5 years ago
git-number Add quotes around paths in bash scripts 5 years ago
git-rebase-update Add quotes around paths in bash scripts 5 years ago
git-rename-branch Add quotes around paths in bash scripts 5 years ago
git-reparent-branch Add quotes around paths in bash scripts 5 years ago
git-retry Add quotes around paths in bash scripts 5 years ago
git-runhooks Revert "Remove py batch wrappers from root dir" 4 years ago
git-squash-branch Add quotes around paths in bash scripts 5 years ago
git-thaw Add quotes around paths in bash scripts 5 years ago
git-upstream-diff Add quotes around paths in bash scripts 5 years ago
git_cache.py Reland "git-cache: Start collecting metrics." 4 years ago
git_cl.py Detect uploadvalidator error message 4 years ago
git_cl_completion.sh Add git completion helper to cl upload 5 years ago
git_common.py [git-rebase-update] Disallow `git gc` during git-rebase-update. 4 years ago
git_dates.py
git_drover.py git_drover: Replace with instructions on how to cherry-pick. 4 years ago
git_find_releases.py Add unit tests for git_find_releases 5 years ago
git_footers.py Replace non-inclusive terms 4 years ago
git_freezer.py
git_hyper_blame.py Add support for git_hyper_blame for Windows 5 years ago
git_map.py
git_map_branches.py Replace non-inclusive terms 4 years ago
git_mark_merge_base.py
git_migrate_default_branch.py Update origin HEAD when running git migration 4 years ago
git_nav_downstream.py depot_tools: Use gclient_utils.AskForData instead of raw_input. 5 years ago
git_new_branch.py Replace non-inclusive terms 4 years ago
git_number.py Run spellchecker on depot_tools 5 years ago
git_rebase_update.py [git-rebase-update] Disallow `git gc` during git-rebase-update. 4 years ago
git_rename_branch.py
git_reparent_branch.py
git_retry.py Replace non-inclusive terms 4 years ago
git_squash_branch.py
git_upstream_diff.py
gn Switch gn wrapper scripts to invoke python3. 4 years ago
gn.bat Switch gn wrapper scripts to invoke python3. 4 years ago
gn.py Switch gn wrapper scripts to invoke python3. 4 years ago
goma_auth Add revert instructions 4 years ago
goma_auth.bat Add revert instructions 4 years ago
goma_ctl Add revert instructions 4 years ago
goma_ctl.bat Add revert instructions 4 years ago
gsutil.py Revert "Reland "Update gsutil to use gsutil version 4.61, python3."" 4 years ago
gsutil.py.bat gsutil.py.bat: Skip updating depot_tools if DEPOT_TOOLS_UPDATE=0. 5 years ago
gsutil.vpython Revert "Reland "Update gsutil to use gsutil version 4.61, python3."" 4 years ago
led Add revert instructions 4 years ago
led.bat Add revert instructions 4 years ago
lockfile.py Reland "Use OS level locking in git_cache.py" 5 years ago
luci-auth Add revert instructions 4 years ago
luci-auth.bat Add revert instructions 4 years ago
lucicfg Add revert instructions 4 years ago
lucicfg.bat Add revert instructions 4 years ago
mac_toolchain
metrics.README.md Replace non-inclusive terms 4 years ago
metrics.py metrics: Wait until upload is complete on bots. 4 years ago
metrics_utils.py gclient: Upload metrics for sync'd deps. 4 years ago
my_activity.py my_activity: Add config file option to add additional gerrit instances 4 years ago
ninja
ninja-linux32
ninja-linux64
ninja-mac mac-arm64: Provide an arm64-native ninja-mac for builds in ¼ of the time 5 years ago
ninja.exe
ninjalog.README.md
ninjalog_uploader.py ninjalog_uploader: ignore http.client.RemoteDisconnected 4 years ago
ninjalog_uploader_wrapper.py ninjalog_uploader_wrapper: do not use binary read/write for json 4 years ago
owners.py Change to slash separator to fix a bug on Windows in owners.py 4 years ago
owners_client.py Reland "presubmit: Skip owners checks if code-owners plugin is enabled." 4 years ago
owners_finder.py owners_finder: Use code-owners plugin if available. 4 years ago
pinpoint Add revert instructions 4 years ago
pinpoint.bat Add revert instructions 4 years ago
post_build_ninja_summary.py autoninja: use pure python3 to get build id and for post_build_ninja_summary.py 4 years ago
presubmit_canned_checks.py Use input_api.sys instead of import sys 4 years ago
presubmit_canned_checks_test.py [depot_tools, chromium/src] mv CheckInclusiveLanguage into canned checks 4 years ago
presubmit_canned_checks_test_mocks.py [depot_tools, chromium/src] mv CheckInclusiveLanguage into canned checks 4 years ago
presubmit_support.py [py3] Decode stdout in presubmit runs 4 years ago
profile.xml
prpc Add revert instructions 4 years ago
prpc.bat Add revert instructions 4 years ago
pylint
pylint-1.5 pylint: roll six to 1.15.0. 4 years ago
pylint-1.6 pylint: roll six to 1.15.0. 4 years ago
pylint-1.7 pylint: roll six to 1.15.0. 4 years ago
pylint-1.8 pylint: roll six to 1.15.0. 4 years ago
pylint-1.9 pylint: roll six to 1.15.0. 4 years ago
pylint-2.6 pylint: roll six to 1.15.0. 4 years ago
pylint.bat
pylint_main.py Don't make depot_tools pylintrc override local pylint configs. 4 years ago
pylintrc CodeInclusion: Rename whitelist/blacklist -> allowlist/blocklist 5 years ago
python_runner.sh Make Python 3 default on Windows 5 years ago
rdb Add revert instructions 4 years ago
rdb.bat Add revert instructions 4 years ago
rdb_wrapper.py [resultdb] use requests.Session in rdb_wrapper 4 years ago
repo repo: restore fstrings for Python 3.6 4 years ago
repo_launcher repo: update to latest version 4 years ago
roll-dep Make Python 3 default on Windows 5 years ago
roll-dep.bat Make Python 3 default on Windows 5 years ago
roll_dep.py Support main branch as default branch in roll-dep 4 years ago
scm.py Use real default branch in gclient 4 years ago
setup_color.py setup_color: Disable colorama on pre-Windows-10 5 years ago
split_cl.py Add more users warning to git cl split 4 years ago
subcommand.py Run spellchecker on depot_tools 5 years ago
subprocess2.py [subprocess2] Replace VOID with DEVNULL 4 years ago
update_depot_tools Fetch all refs/heads/* on depot_tools update 4 years ago
update_depot_tools.bat Fetch all refs/heads/* on depot_tools update 4 years ago
update_depot_tools_toggle.py
upload_metrics.py metrics: Report bot metrics if DEPOT_TOOLS_REPORT_BUILD is set 4 years ago
upload_to_google_storage.py Revert "Reland "Update gsutil to use gsutil version 4.61, python3."" 4 years ago
vpython Add revert instructions 4 years ago
vpython.bat Add revert instructions 4 years ago
vpython3 depot_tools: Simplify MinGW check. 5 years ago
vpython3.bat Fix git-* commands when using python3 on Win 5 years ago
watchlists.py update stale codereview.chromium.org links 4 years ago
weekly Replace non-inclusive terms 4 years ago
whitespace.txt Change whitespace.txt 4 years ago
win32imports.py Reland "Use OS level locking in git_cache.py" 5 years ago
wtf
yapf Run yapf with vpython3 4 years ago
yapf.bat Run yapf with vpython3 4 years ago

README.md

depot_tools

Tools for working with Chromium development. It requires python 2.7 or 3.8 for python 3 support.

Tools

The most important tools are:

  • fetch: A gclient wrapper to checkout a project. Use fetch --help for more details.
  • gclient: A meta-checkout tool. Think repo or git submodules, except that it support OS-specific rules, e.g. do not checkout Windows only dependencies when checking out for Android. Use gclient help for more details and README.gclient.md.
  • git cl: A code review tool to interact with Rietveld or Gerrit. Use git cl help for more details and README.git-cl.md.
  • roll-dep: A gclient dependency management tool to submit a dep roll, updating a dependency to a newer revision.

There are a lot of git utilities included.

Updating

depot_tools updates itself automatically when running gclient tool. To disable auto update, set the environment variable DEPOT_TOOLS_UPDATE=0.

To update package manually, run update_depot_tools.bat on Windows, or ./update_depot_tools on Linux or Mac.

On Windows only, running gclient will install git and python.

Contributing

To contribute change for review:

git new-branch <somename>
# Hack
git add .
git commit -a -m "Fixes goat teleporting"
# find reviewers
git cl owners
git log -- <yourfiles>

# Request a review.
git cl upload -r reviewer1@chromium.org,reviewer2@chromium.org --send-mail

# Edit change description if needed.
git cl desc

# If change is approved, flag it to be committed.
git cl set-commit

# If change needs more work.
git rebase-update
...
git cl upload -t "Fixes goat teleporter destination to be Australia"

See also open bugs, open reviews, forum or report problems.

cpplint.py

Until 2018, our cpplint.py was a copy of the upstream version at https://github.com/google/styleguide/tree/gh-pages/cpplint. Unfortunately, that repository is not maintained any more. If you want to update cpplint.py in depot_tools, just upload a patch to do so. We will figure out a long-term strategy via issue https://crbug.com/916550.

Note that the cpplint.py here is also used by the Tricium analyzer, so if the cpplint.py here changes, we should also update the copy used there.