You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
The first call to gsutil causes a network call and makes the download script slow also in the most optimal cases. This CL refactors the download script and moves the first gsutil call after checking locally if sha1s match. 1) This turns the input acquisition into a generator and buffers the files and sha1s in a list before multithreading. 2) This sequentially checks the sha1s and files and bails out early if all match. In Chrome-land, we usually call this script with only one file. There are some cases with around 4. This could also be parallelized if the need arises. 3) The initial gsutil check, which ensures gsutil is updated, is moved right in front of the multithreaded downloads. The performance of one call to download_from_google_storage for an existing 500MB file is 2.3s before this CL and 1.2s after (most of the remaining time left is spent for making sha1sum). Example for full gclient runhooks (when everything is up-to-date): Chromium: 32s before, 12s after V8: 12s before, 3s after Bug: 776311 Change-Id: Ia7715a6af84b1b336455ea88494d399bdb050317 Reviewed-on: https://chromium-review.googlesource.com/897562 Commit-Queue: Michael Achenbach <machenbach@chromium.org> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org> Reviewed-by: Ryan Tseng <hinoka@chromium.org> |
7 years ago | |
---|---|---|
bootstrap/win | 7 years ago | |
fetch_configs | 7 years ago | |
git-templates | ||
infra | 7 years ago | |
man | 8 years ago | |
recipes | 7 years ago | |
support | ||
testing_support | 8 years ago | |
tests | 7 years ago | |
third_party | 8 years ago | |
win_toolchain | 8 years ago | |
zsh-goodies | ||
.gitattributes | 8 years ago | |
.gitignore | 8 years ago | |
.style.yapf | 10 years ago | |
LICENSE | ||
OWNERS | 8 years ago | |
PRESUBMIT.py | 7 years ago | |
README.gclient.md | 9 years ago | |
README.git-cl.md | 9 years ago | |
README.md | 8 years ago | |
README.testing | ||
WATCHLISTS | 9 years ago | |
annotated_gclient.py | ||
appengine_mapper.py | 9 years ago | |
apply_issue | ||
apply_issue.bat | 8 years ago | |
apply_issue.py | 8 years ago | |
auth.py | 8 years ago | |
autoninja | 8 years ago | |
autoninja.bat | 8 years ago | |
autoninja.py | 8 years ago | |
breakpad.py | 9 years ago | |
buildbucket.py | 10 years ago | |
cbuildbot | ||
checkout.py | 7 years ago | |
chrome_set_ver | ||
cipd | 8 years ago | |
cipd.bat | 8 years ago | |
cipd.ps1 | 8 years ago | |
cipd_bin_setup.bat | 8 years ago | |
cipd_bin_setup.sh | 8 years ago | |
cipd_client_version | 7 years ago | |
cipd_manifest.txt | 7 years ago | |
cit | 10 years ago | |
cit.bat | 8 years ago | |
cit.py | 7 years ago | |
clang-format | ||
clang-format.bat | 8 years ago | |
clang_format.py | 8 years ago | |
clang_format_merge_driver | 9 years ago | |
clang_format_merge_driver.bat | 8 years ago | |
clang_format_merge_driver.py | 9 years ago | |
codereview.settings | 9 years ago | |
compile_single_file | 8 years ago | |
compile_single_file.bat | 8 years ago | |
compile_single_file.py | 8 years ago | |
cpplint.bat | 8 years ago | |
cpplint.py | 8 years ago | |
cpplint_chromium.py | ||
cros | ||
cros_sdk | ||
dart_format.py | ||
depot-tools-auth | 10 years ago | |
depot-tools-auth.bat | 8 years ago | |
depot-tools-auth.py | 9 years ago | |
download_from_google_storage | ||
download_from_google_storage.bat | 8 years ago | |
download_from_google_storage.py | 7 years ago | |
ensure_bootstrap | 8 years ago | |
fetch | ||
fetch.bat | 8 years ago | |
fetch.py | 7 years ago | |
fix_encoding.py | 8 years ago | |
gclient | 9 years ago | |
gclient-new-workdir.py | 7 years ago | |
gclient.bat | 8 years ago | |
gclient.py | 7 years ago | |
gclient_completion.sh | ||
gclient_eval.py | 7 years ago | |
gclient_scm.py | 7 years ago | |
gclient_utils.py | 7 years ago | |
gerrit_client.py | 8 years ago | |
gerrit_util.py | 7 years ago | |
git-cache | 10 years ago | |
git-cl | 10 years ago | |
git-crrev-parse | 8 years ago | |
git-crsync | ||
git-drover | 10 years ago | |
git-find-releases | 10 years ago | |
git-footers | 10 years ago | |
git-freeze | 10 years ago | |
git-gs | 8 years ago | |
git-hyper-blame | 9 years ago | |
git-map | 10 years ago | |
git-map-branches | 10 years ago | |
git-mark-merge-base | 10 years ago | |
git-nav-downstream | 10 years ago | |
git-nav-upstream | ||
git-new-branch | 10 years ago | |
git-number | 10 years ago | |
git-rebase-update | 10 years ago | |
git-rename-branch | 10 years ago | |
git-reparent-branch | 10 years ago | |
git-retry | 10 years ago | |
git-runhooks | ||
git-squash-branch | 10 years ago | |
git-thaw | 10 years ago | |
git-upstream-diff | 10 years ago | |
git_cache.py | 7 years ago | |
git_cl.py | 7 years ago | |
git_cl_completion.sh | 9 years ago | |
git_common.py | 7 years ago | |
git_dates.py | 9 years ago | |
git_drover.py | 7 years ago | |
git_find_releases.py | 10 years ago | |
git_footers.py | 7 years ago | |
git_freezer.py | 10 years ago | |
git_hyper_blame.py | 8 years ago | |
git_map.py | 8 years ago | |
git_map_branches.py | 8 years ago | |
git_mark_merge_base.py | 10 years ago | |
git_nav_downstream.py | 10 years ago | |
git_new_branch.py | 9 years ago | |
git_number.py | 8 years ago | |
git_rebase_update.py | 8 years ago | |
git_rename_branch.py | 10 years ago | |
git_reparent_branch.py | 10 years ago | |
git_retry.py | 8 years ago | |
git_squash_branch.py | 8 years ago | |
git_upstream_diff.py | 7 years ago | |
gn | ||
gn.bat | 8 years ago | |
gn.py | 10 years ago | |
gsutil.py | 7 years ago | |
gsutil.vpython | 8 years ago | |
led | 8 years ago | |
led.bat | 8 years ago | |
mac_toolchain | 8 years ago | |
my_activity.py | 7 years ago | |
my_reviews.py | 8 years ago | |
ninja | 8 years ago | |
ninja-linux32 | 8 years ago | |
ninja-linux64 | 8 years ago | |
ninja-mac | 8 years ago | |
ninja.exe | 8 years ago | |
owners.py | 8 years ago | |
owners_finder.py | 8 years ago | |
patch.py | 8 years ago | |
presubmit_canned_checks.py | 7 years ago | |
presubmit_support.py | 7 years ago | |
profile.xml | ||
pylint | ||
pylint.py | 10 years ago | |
pylintrc | 8 years ago | |
python_runner.sh | 9 years ago | |
repo | 9 years ago | |
rietveld.py | 8 years ago | |
roll-dep | ||
roll-dep-svn | 9 years ago | |
roll-dep-svn.bat | 8 years ago | |
roll-dep.bat | 8 years ago | |
roll_dep.py | 7 years ago | |
roll_dep_svn.py | 8 years ago | |
scm.py | 7 years ago | |
setup_color.py | 8 years ago | |
split_cl.py | 7 years ago | |
subcommand.py | 8 years ago | |
subprocess2.py | 8 years ago | |
update_depot_tools | 8 years ago | |
update_depot_tools.bat | 8 years ago | |
update_depot_tools_toggle.py | 8 years ago | |
upload_to_google_storage.py | 8 years ago | |
vpython | 8 years ago | |
vpython.bat | 8 years ago | |
watchlists.py | 8 years ago | |
weekly | ||
wtf |
README.md
depot_tools
Tools for working with Chromium development. It requires python 2.7.
Tools
The most important tools are:
fetch
: Agclient
wrapper to checkout a project. Usefetch --help
for more details.gclient
: A meta-checkout tool. Think repo or git submodules, except that it support OS-specific rules, e.g. do not checkout Windows only dependencies when checking out for Android. Usegclient help
for more details and README.gclient.md.git cl
: A code review tool to interact with Rietveld or Gerrit. Usegit cl help
for more details and README.git-cl.md.roll-dep
: A gclient dependency management tool to submit a dep roll, updating a dependency to a newer revision.
There are a lot of git utilities included.
Updating
depot_tools
updates itself automatically when running gclient
tool. To
disable auto update, set the environment variable DEPOT_TOOLS_UPDATE=0
.
To update package manually, run update_depot_tools.bat
on Windows,
or ./update_depot_tools
on Linux or Mac.
On Windows only, running gclient
will install git
and python
.
Contributing
To contribute change for review:
git new-branch <somename>
# Hack
git add .
git commit -a -m "Fixes goat teleporting"
# find reviewers
git cl owners
git log -- <yourfiles>
# Request a review.
git cl upload -r reviewer1@chromium.org,reviewer2@chromium.org --send-mail
# Edit change description if needed.
git cl desc
# If change is approved, flag it to be committed.
git cl set-commit
# If change needs more work.
git rebase-update
...
git cl upload -t "Fixes goat teleporter destination to be Australia"
See also open bugs, open reviews, forum or report problems.
cpplint.py
To update cpplint.py, please submit the change upstream first at https://github.com/google/styleguide/tree/gh-pages/cpplint then copy it down.